ACCORDING TO A RECENT REPORT BY THE TOLLY GROUP, NETSCALER HAS OUTPERFORMED F5 IN A SERIES OF BENCHMARKING TESTS.
NetScaler commissioned The Tolly Group, a leading provider of third-party validation services for IT vendors, to conduct benchmarking tests comparing NetScaler VPX and F5 BIG-IP Virtual Edition (VE) virtualized software application delivery controllers (ADCs).
The results are clear:
NetScaler outperforms F5 in every scenario, offering better CPU efficiency, lower latency, and higher throughput. These improvements significantly impact application response time, end-user experience, and total cost of ownership (TCO).
Testing conditions and scenarios
Absolutely, ADCs (Application Delivery Controllers) have indeed come a long way from their origins as basic load balancers. They now play a crucial role in optimizing application performance, managing traffic, and ensuring security. However, it's essential that they do not introduce excessive latency, as this would counteract their primary purpose of enhancing performance.
Balancing these functions effectively is key to maintaining a smooth and efficient network. Have you encountered any specific issues with ADCs in your experience?
When testing ADC performance, it's crucial to conduct tests in simulated real-world scenarios to accurately reflect the performance customers can expect. To represent common internet traffic scenarios, all test traffic was encrypted with TLS 1.3. This is because analysis of data from web crawlers across the top 1 million websites showed that 73 percent of SSL traffic uses the latest TLS 1.3 protocol.
As many enterprises consolidate their application security workloads, such as web application firewalls (WAF), into their application delivery controllers (ADCs), tests were conducted to evaluate performance. These tests involved ADCs running various combinations of traditional load balancing (baseline test), content rewriting/manipulation (policy), and application security inspection (app sec), in increasing order of task complexity.
NetScaler outperforms F5 in all scenarios
NetScaler consistently outperforms F5 by using fewer CPU resources and introducing significantly lower latency across various scenarios and workload conditions. Overall, F5’s performance declines when carrying out multiple, common ADC tasks.
NetScaler runs as software in any environment and delivers the following advantages over F5 in a series of tests:
1. Superior Proxy Performance: NetScaler offers better proxy performance for TLS processing, resulting in an average latency advantage of approximately 100ms.
2. Lower Latency Under Stress: NetScaler VPX, a virtualized ADC, provides one-fifth the latency under stress and handles 10% more transactions than F5 BIG-IP.
3. Scalability: NetScaler is designed to be software-focused and hardware-agnostic, enabling dynamic scaling of internet traffic for hybrid and multi-cloud workloads.
4. Management and Flexibility: NetScaler's single management plane allows for consistent management across hybrid and multi-cloud environments, unlike F5, which requires multiple management systems.
These advantages make NetScaler a more efficient and flexible choice for application delivery.
NetScaler has been shown to achieve higher throughput with similar CPU usage compared to F5.
According to performance tests, NetScaler can handle high volumes of traffic more efficiently, providing multiple services such as traffic management, policy enforcement, and application security. This makes it a strong contender in the application delivery controller market.
NetScaler consistently shows higher throughput than F5 in various test scenarios. For example, when combining load balancing, content rewriting, and application security inspection, NetScaler achieved a throughput of 2,461 Mbps, which is three times higher than F5's 797 Mbps.
NetScaler has been shown to be 64% more efficient in CPU usage compared to F5 for a given throughput.
This means that NetScaler can handle the same amount of traffic while using significantly less CPU power, which can lead to better performance and lower operational costs.
NetScaler consistently demonstrates more efficient CPU usage than F5 across various test scenarios. For instance, when both load balancing and application security are enabled, NetScaler's CPU usage is approximately half that of F5.
It's important to note that during the test scenario involving load balancing, policy, and application security inspection, adjustments were necessary because F5 couldn't handle 2.3 Gbps of throughput without maxing out its CPU resources. As a result, the throughput was reduced to 1.2 Gbps to prevent F5 from exhausting all CPU resources. Despite this, NetScaler's CPU usage was 64% lower than F5's.
As shown in Figure 2, when the number of tasks increased, F5's CPU usage rose to nearly three times that of NetScaler, regardless of the throttled throughput.
F5 introduces up to 9x latency (application response delay) for similar throughput
In today’s application economy, the application experience is often synonymous with user experience. A significant increase in application response delay can greatly impact performance, resulting in a poor end-user experience and reduced service quality.
"53% of mobile users abandon sites that take longer than 3 seconds to load."
Delays in response times can cause impatience and frustration, leading users to perceive the application as unreliable and potentially reducing future engagement. Additionally, longer response times can increase user stress and dissatisfaction, further degrading the overall experience.
In a testing scenario with load balancing, policy, and application security enabled, and throughput throttled down to 1.2 Gbps (since F5 could not handle 2.3 Gbps without exhausting CPU resources), NetScaler demonstrated a significantly better performance. The application response delay per request for NetScaler was 89% less than that of F5.
NetScaler's one-pass architecture enables it to handle security functions and load balancing simultaneously, resulting in low latency even with security features active.
Achieving similar functionality and performance with F5 ADCs would require more or larger instances, leading to increased complexity and higher costs.
NetScaler consistently outperforms F5 in terms of latency, especially as the number of simultaneous connections increases.
According to performance benchmarking tests, NetScaler offers significantly lower latency and higher throughput compared to F5, making it a more efficient choice for handling increased traffic loads.
One of the key roles of ADCs is scalability, which is the ability to handle multiple simultaneous connections without affecting application response times.
This test measured P95 latency (the 95th percentile latency, indicating the delay for the worst 5 percent of traffic flows) at load levels of 200, 1,000, 2,000, and 4,000 simultaneous connections/users, with load balancing, policies, and application security capabilities enabled. As the load levels increased from 200 to 4,000 connections, the differences in latency between NetScaler and F5 were significant.
At 4,000 users, NetScaler's application response time was 96% faster than that of F5. F5 experienced degraded response times for individual sessions. With 4,000 concurrent users, typical for e-commerce applications, F5's latency was 26 times higher than NetScaler's (48.24 ms compared to 1.85 ms).
Impact: How ADC performance affects business outcomes
Analog-to-Digital Converters (ADCs) are crucial in various industries due to their ability to convert analog signals into digital data with high precision. Here are three industries that heavily rely on ADCs and how they apply this technology:
1. Medical Diagnostics: In medical imaging and diagnostics, high-resolution ADCs are essential for capturing detailed and accurate images from devices like MRI and CT scanners. These ADCs ensure that even the smallest variations in the analog signals are accurately converted to digital data, leading to precise diagnostics and better patient outcomes.
2. Industrial Automate: Modern production plants use ADCs to monitor and control various parameters such as pressure, temperature, and flow. High-performance ADCs are integrated into programmable logic controllers (PLCs) to handle multiple analog inputs from different sensors, ensuring robust and accurate data conversion even in harsh industrial environments.
3. Scientific Research: In fields like quantum mechanics and materials science, precise measurements are critical. High-resolution ADCs are used to convert analog signals from sensors into digital data with minimal noise and high fidelity. This allows researchers to obtain accurate data for their experiments and analyses.
These applications highlight the importance of ADCs in ensuring accuracy, reliability, and efficiency across different sectors.
E-commerce
High-performing e-commerce applications share a crucial feature: exceptionally fast response times. Customers expect instant connections, fast load times, and always-on availability. Failing to meet these expectations can lead to shopping cart abandonment or application deletion, resulting in lost revenue, poor customer experience, and diminished brand reputation and loyalty.
Tests indicate that NetScaler can handle more concurrent connections per ADC without users experiencing performance degradation.
"17% of shoppers abandoned their carts because the website crashed or had other errors."
Financial services
In the financial services industry, application response time is crucial. Slow response times can cause failed transactions, disrupt online banking services, and lead to significant customer dissatisfaction, which can harm the institution's reputation and financial performance. An ADC (Application Delivery Controller) with efficient CPU usage, lower latency, and higher throughput can significantly improve user experience and engagement, providing financial services companies with a competitive edge over their rivals.
Healthcare
High latency or slow application response can lead to delays in booking appointments, which in turn can delay diagnoses or medication, negatively impacting a patient’s health. For healthcare providers, it can result in delays in accessing electronic health records (EHR) and downtime, leading to delays in patient care. This can have a long-lasting impact on the patient’s health, as well as the healthcare provider’s reputation and revenue.
Summary of test results
The test results show that NetScaler can handle large amounts of traffic more efficiently than F5, using fewer CPU resources and achieving higher throughput. NetScaler's versatility in offering services like traffic management, policy enforcement, and application security makes it a crucial part of contemporary IT infrastructure.
Fewer, smaller ADCs required: NetScaler's efficiency allows it to handle the same workload with fewer and smaller ADC instances compared to F5. This can significantly reduce infrastructure complexity and costs.
Latency matters: Absolutely! Lower latency is crucial for enhancing application performance and improving the end-user experience. NetScaler's one-pass architecture is designed to minimize latency, ensuring faster application response times even when the system is under heavy load. This efficiency can make a significant difference in maintaining smooth and responsive applications.
Faster, more reliable application response: NetScaler significantly improves application performance and reliability by leveraging advanced load balancing, TCP optimization, and HTTP compression. These features reduce latency, minimize packet loss, and enhance data flow efficiency, ensuring a faster and more dependable user experience. This leads to increased productivity and satisfaction for end-users.
The tests conducted by The Tolly Group demonstrate that NetScaler not only outperforms F5 in CPU efficiency, latency, and throughput, but also delivers tangible benefits such as reduced infrastructure complexity and lower operational costs. These improvements can significantly impact business outcomes, leading to enhanced customer satisfaction, better user experience, and increased revenue.
NetScaler allows enterprises to achieve more efficient and faster application performance with fewer and/or smaller ADCs compared to F5. This results in reduced operational complexity and costs, as well as an optimal application end-user experience.
NetScaler’s superior performance in CPU efficiency, latency, and throughput makes NetScaler the clear choice for enterprises looking to optimize their application delivery and security.
Read the complete report: Performance Benchmarking NetScaler VPX vs. F5 BIG-IP Virtual Edition (no sign-up required)