Cloud Network Firewall

Cloud network firewalls are considered to be the first line of defense when deployed in public cloud providers such as Amazon Web Services, Google Cloud Platform and Microsoft Azure. While the firewall market is one of the largest and most mature security technology segments, cloud network firewalls are a relatively new technology, deployed within a cloud service that by definition is constantly changing.

CyberRatings ran two cloud network firewall tests in 2024. The Q1 test consisted of eleven market leading cloud network firewalls where security effectiveness scores ranged from 5.39% to 100%. As part of that cloud firewall test, CyberRatings also discovered that many firewall evasion defenses were not on by default, potentially leaving customers at significant risk. In response, CyberRatings provided a policy and configuration guide to help enterprises ensure that their firewalls are configured properly.

In Q4, CyberRatings announced the results of a Cloud Service Provider (CSP) “Mini-Test” comparing native firewall solutions from Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP), assessing security effectiveness, usability, protection, and monitoring features. The Q4 test, using Keysight’s CyPerf v5.0, found that security effectiveness was notably low across these native solutions. The second part of this test will be published in Q1 2025.

For the Q1 publication, eleven market leading Cloud Network Firewall vendors were in this comparative test. Six products were Recommended, one product received a Neutral rating, and four received a Caution rating. Security Effectiveness scores ranged from 5.39% to 100%.

CyberRatings tested the cloud firewall products to determine how they handled TLS/SSL (authentication) 1.2 and 1.3 cipher suites (algorithms), how they defended against 984 exploits (attacks that take advantage of a software flaw or install malware), and whether any of 1,645 evasions could bypass protection. At all times the devices needed to remain stable under adverse conditions. To provide a more realistic rating based on modern network traffic, both clear text (HTTP) and encrypted traffic (HTTPS) were measured. Amazon Web Services (AWS) was the public cloud service chosen to run the test.

The test reports are provided in two ways: individual test reports for each vendor and a comparative report summarizing test results for all of the vendors in the test. The methodology and the comparative SVM are provided to the community at no cost, while the in-depth reports are available for purchase. Of the individual reports, those that received a “Caution” rating are available to all members free of charge. Security Effectiveness scores ranged from 5.39% to 100%.

The combined report for the Q4 test compares the results of native firewall solutions from Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). For this “Mini-Test” CyberRatings employed 522 attacks (exploits), focusing on exploit types that target servers and are typically relevant to cloud workload deployments. The exploits were from the last ten years with a severity of medium or higher.

The second part of this test will be published in Q1 2025.  Part two will include a higher number of exploits, along with evasions and malware. The second part of the test will also compare cloud service provider native solutions against market leading third-party cloud network firewall providers.

For the Q1 test, we would like to issue a special thank you to Keysight Technologies for providing their CyPerf and BreakingPoint tools for us to test the performance, TLS functionality, and stability of Cloud Network Firewalls. We would also like to thank TeraPackets for providing us with their Threat Replayer tool which enabled us to accurately replay exploits in a cloud environment.

For the Q4 test, the CyPerf v5.0 software testing platform was used exclusively.