Abstract

NetScreen Technologies, Inc. commissioned The Tolly Group to evaluate the performance of its NetScreen-5, an Internet security appliance integrating firewall and virtual private networking (VPN) in a SOHO environment. NetScreen requested that The Tolly Group evaluate the NetScreen-5 along with the following Internet appliances: a SonicWALL, Inc. SOHO/50 and a WatchGuard Technologies, Inc. SOHO. The Tolly Group conducted tests of devices as IPSec tunnels for application and zero-loss throughput. Tolly engineers also tested each device as a firewall and measured the zero-loss throughput when using UDP packets. For zero-loss performance tests, The Tolly Group measured steady-state throughput at 0.001%, the same metric The Tolly Group uses to test Layer 2 and Layer 3 networking devices. Testing was performed in July through November, 2000.

Summary: Tolly Group Evaluation of NetScreen-5 vs. SonicWALL SOHO/50 and WatchGuard SOHO (January 2001)

NetScreen Technologies commissioned The Tolly Group to evaluate the performance of its NetScreen-5 Internet security appliance against two competitors: SonicWALL SOHO/50 and WatchGuard SOHO. The test focused on Small Office/Home Office (SOHO) environments requiring integrated firewall and IPSec VPN security. Tests included application throughput, zero-loss packet forwarding in IPSec tunnels, and firewall performance under various packet sizes (64 to 1,518 bytes).

In IPSec application throughput tests, NetScreen-5 vastly outperformed its competitors, forwarding 5.6 Mbit/s of FTP traffic compared to SonicWALL’s 0.1 Mbit/s and WatchGuard’s 0.9 Mbit/s. For SAP R/3 traffic, NetScreen-5 achieved 3.9 Mbit/s, while SonicWALL and WatchGuard lagged behind at 0.1 Mbit/s and 0.7 Mbit/s, respectively. In zero-loss IPSec throughput tests, NetScreen-5 forwarded up to 45% of the theoretical maximum load with larger packets (1,024 bytes), whereas SonicWALL and WatchGuard struggled to exceed 5% and even failed to pass large packets (1,518 bytes) altogether due to packet fragmentation limitations.

As a firewall, NetScreen-5 demonstrated superior zero-loss throughput, achieving up to 85% of theoretical throughput with large packets, compared to SonicWALL’s 75% and WatchGuard’s 60%. Even in worst-case small packet tests (64 bytes), NetScreen-5 maintained 10% throughput without loss, outperforming the other devices. The study concluded that NetScreen-5’s performance was suitable for DSL- and cable-speed SOHO environments, providing enterprise-grade VPN encryption and firewalling without degrading network performance, unlike its competitors which were severely limited under high-load and encrypted conditions.