Tag: Palo Alto Networks

SDxCentral: Palo Alto Networks and Fortinet given all clear after firewall hiccups

Posted on by CyberRatings.org

Palo Alto Networks and Fortinet have received a clean bill of health for their firewall protections, while the jury is still out on current Cisco defenses.

CyberRatings.org recommended both Palo Alto and Fortinet after new tests confirmed they had patched evasions previously discovered by the security testing firm.

In tests carried out at the start of the month by CyberRatings’ testing partner NSS Labs, researchers found they were able to bypass protection using Layer 4 TCP evasions in both Palo Alto’s PAN-OS (version 11.2.8-c537) and Fortinet’s IPS (v7.01154), as well as evading Layer 3 IP in the Palo Alto operating system.

Both firms reacted quickly, with Palo Alto developing an updated PAN-OS firmware package (PAN-OS 11.2.10-c37) and Fortinet deploying an updated IPS package (v7.01165 (33.00064) to fix the vulnerabilities.

Read the full article here.

CyberRatings.org and NSS Labs Announce Follow-On Enterprise Firewall Results

Posted on by CyberRatings.org

Austin, TX – November 25, 2025 – CyberRatings.org (CyberRatings), the non-profit organization dedicated to providing confidence in cybersecurity products and services through independent testing, today announced Follow-On Test Results for the Fortinet FortiGate-200G and Palo Alto Networks PA-1410 Enterprise Firewalls.

Both products have improved their ratings from Caution to Recommended following submissions to NSS Labs to retest after developing new builds to address their earlier evasion resistance deficiencies published on November 5, 2025.

“Both Fortinet and Palo Alto Networks responded quickly and transparently to our original findings, issuing updates within days and requesting immediate retesting,” said Vikram Phatak, CEO of NSS Labs. “The speed at which these vendors addressed and resolved critical issues shows their commitment to their customers’ security.”

Fortinet Follow-On Results

During the initial test of Fortinet’s v7.6.4 build3596 with IPS v7.01154 (33.00064), NSS Labs was able to bypass protection using Layer 4 TCP evasions. Fortinet responded quickly to develop an updated IPS signature package. After retesting, NSS Labs confirmed that the update addressed all exploit evasion resistance deficiencies.

Exploit evasion resistance increased from 60% to 100%, elevating the overall Security Effectiveness from 79.24% to 99.24%. Organizations running IPS version v7.01154 (33.00064) or earlier should upgrade immediately to v7.01165 (33.00064) to ensure protection against evasion techniques as detailed in the November 5 publication.

Palo Alto Networks Follow-On Results

During the initial test of PAN-OS 11.2.8-c537, NSS Labs was able to bypass protection using Layer 3 IP and Layer 4 TCP evasions. Palo Alto Networks responded quickly to develop an updated PAN-OS firmware package (PAN-OS 11.2.10-c37) to ensure that the problem had been fixed. After retesting, NSS Labs confirmed that the updated firmware addressed all exploit evasion resistance deficiencies, providing substantial improvements in protection.

Exploit evasion resistance increased dramatically from 0% to 100%, elevating the overall Security Effectiveness from 46.37% to 96.07%.

NSS Labs notes that it is not unusual for vendors to submit pre-release software or firmware intended for imminent release, which NSS Labs requires to be scheduled for general availability within 90 days following a test. Palo Alto Networks confirmed that PAN-OS version 11.2.10-c37 was provided as a pre-release and will be designated as PAN-OS 11.2.10 upon reaching general availability.

Organizations running PAN-OS 11.2.8-c537 or earlier should immediately request PAN-OS 11.2.10 to ensure protection against evasion techniques as detailed in the November 5 publication.

Context and Vendor Accountability

These follow-on results reaffirm the importance of independent testing and vendor accountability. Both vendors’ prompt response demonstrates how transparency and rapid engineering benefit customers.

To accompany these follow-on reports, NSS Labs published a blog titled When Firewalls Fail Gracefully: Why Vendor Responsiveness Matters as Much as Security Effectiveness, highlighting the importance of transparency and quick remediation in cybersecurity engineering.

Testing Methodology

The follow-on tests were conducted using the same methodology and datasets employed in the original Q4 2025 Enterprise Firewall Comparative Report, which evaluated seven leading products under real-world conditions. The updated results now place Fortinet and Palo Alto Networks in the Recommended category alongside Check Point, Juniper Networks, and Versa Networks.

Tests were conducted by NSS Labs developed technologies and Keysight’s CyPerf tool to evaluate security, performance, TLS functionality, and stability. The updated test reports are available at no cost on the CyberRatings.org website.

When Firewalls Fail Gracefully

Posted on by CyberRatings.org

The latest NSS Labs Enterprise Firewall Comparative Report was published this month and, as usual, provided a deep insight into the state of the enterprise firewall market.

Seven of the most widely deployed products were tested using real-world attack scenarios, enterprise-grade workloads, and adversarial evasion techniques to measure their resilience, reliability, and performance.

The results reveal a security landscape that remains uneven: most products blocked the majority of exploits and malware, but a few stumbled when exposed to modern, and not so modern, evasion techniques.

However, the story doesn’t end with the Comparative Security Map – it is also a case study in vendor accountability. How vendors respond when weaknesses are exposed in independent tests such as this tells us a lot about how they are likely to support their enterprise customers in a pinch. It also tells us how seriously they take engineering challenges that could result in serious failures, or even breaches, when installed in live environments.

Palo Alto Networks and Fortinet, though not the highest-scoring participants, stand out precisely because they treated the findings as an opportunity to rectify shortcomings in their products that could have a serious impact on their customers. Within days of publication, both vendors confirmed patches for the issues identified and scheduled retests for the affected products. That kind of responsiveness deserves as much attention as raw test scores.

Read the full blog from NSS Labs: https://nsslabs.com/media/blog/when-firewalls-fail-gracefully/

SDxCentral: SSE protection found uneven across major vendors

Posted on by CyberRatings.org

Researchers reported major disparity in security effectiveness of security service edge (SSE) protection across major vendors.

Non-profit cyberratings.org/ (CyberRatings) found security effectiveness ranged from less than 3% to 100% in its testing of vendor products, with only Fortinet, Palo Alto Networks, Versa Networks, and Zscaler earning a “recommended” rating.

In contrast, SSE products from Cisco, Cloudflare, and Skyhigh were tagged with a “caution” label, indicating “below-average” security effectiveness with the recommendation that end users “should consider seeking other solutions.” The ratings were put down due to “failures in critical tests.”

Read the full article here.

CyberRatings.org Test Results Reveal Critical Failures in SSE

Posted on by CyberRatings.org

Austin, TX – July 16, 2025 – CyberRatings.org (CyberRatings), the non-profit organization dedicated to providing insight into the capabilities of cybersecurity products and services through independent testing, today announced the comparative results of its latest Security Service Edge (SSE) evaluation. The findings expose a striking disparity in product performance: Security Effectiveness ranged from 2.95% to 100%, underscoring just how uneven SSE protection remains across vendors.

Only Fortinet, Palo Alto Networks, Versa Networks, and Zscaler earned a Recommended rating, while products from Cisco, Cloudflare, and Skyhigh were rated Caution due to failures in critical tests.

Despite meeting our inclusion criteria and high market interest, we were unable to include Cato Networks and Netskope in this test. Netskope’s high entry level licensing cost and their lack of responsiveness to our inquiries to purchase their product rendered it inaccessible. Cato was explicit in their refusal to engage with us or allow us to procure licensing for any form of independent third-party validation.

“With cloud-delivered products rapidly evolving through continuous integration and deployment, customers have little visibility into what changes under the hood,” said Vikram Phatak, CEO of CyberRatings.org. “Only by conducting regular independent testing can enterprises ensure they’re not left vulnerable to silent failures that could go unnoticed for months.”

Of all the SSE test criteria, blocking evasions had the most impact on security effectiveness. Evasion techniques are used by threat actors to disguise or modify attacks, so they slip past defenses. While most products excelled at blocking known malware and exploits, three failed to stop evasions — exposing organizations to entire classes of undetected attacks.

These independent tests uniquely stress real-world evasion techniques that standard evaluations often overlook — the techniques cybercriminals rely on to bypass security measures.

The SSE evaluation was designed to reflect modern, adversarial conditions and covered:

  • Malware: 6,184 malware samples in active use by global threat actors.
  • Exploits: 205 exploits of known vulnerabilities.
  • Evasions: 1,154 evasions spanning 37 categories of techniques.
  • False Positives: 1,514 legitimate files and applications, verifying security measures do not impact users and operations.
  • TLS/SSL: Encrypted attacks using cipher suites that represent ~97% of real-world HTTPS traffic.

Security Service Edge is inherently complex — a multi-layered technology stacked atop ever-changing cloud environments. Customers typically have minimal visibility into how these systems operate and testing them independently is challenging. This double-layered opacity makes third-party validation essential to diagnose performance issues, fine-tune policy enforcement, and ensure real security outcomes. CyberRatings strongly urges organizations to adopt periodic or ongoing third-party testing to ensure consistent protection and compliance.

NSS Labs is the Official Testing Partner of CyberRatings. Keysight’s CyPerf tool was used for performance and TLS/SSL functionality, and TeraPackets Threat Replayer tool was used for exploit replay validation.

CyberRatings.org Announces Test Results for Cisco Umbrella and Palo Alto Networks Prisma Access

Posted on by CyberRatings.org

Austin, TX – May 15, 2025 – cyberratings.org/ (CyberRatings), the non-profit organization dedicated to providing insight into the capabilities of cybersecurity products and services through independent testing, has released additional results from its Security Service Edge (SSE) testing. These latest tests focused on two leading products: Cisco Umbrella and Palo Alto Networks Prisma Access.

Palo Alto Networks Prisma achieved a Security Effectiveness score of 98.89%, successfully blocking 100% of evasions. In contrast, Cisco Umbrella scored 12.44%, primarily due to its failure to detect evasive threats. Full test reports detail product performance across multiple threat categories, with scoring weighted by attack severity.

The evaluation covered:

  • TLS/SSL: Top 5 Ciphers used (accounts for ~97% of HTTPS traffic).
  • Malware: 6,184 attack samples sourced from current malware campaigns.
  • Exploits: 205 attack samples from widely exploited vulnerabilities in enterprise environments.
  • Evasions: 1,154 attacks spanning 37 evasion techniques.
  • False Positives: 1,514 samples from various business-critical files and applications, ensuring security measures did not disrupt legitimate traffic.

Evasion techniques are used by attackers to disguise or obfuscate attacks so that they bypass detection. SSE products must not be tricked by evasions—failure exposes organizations to entire classes of (undetected) threats.

“Missing just one type of evasion allows attackers to use entire categories of malware or exploits undetected,” said Vikram Phatak, CEO of cyberratings.org/.

Security Service Edge is a complex multi-layered security technology built on top of complex, ever-changing cloud technologies. Customers have minimal visibility into their operation and architecture, and testing is challenging. This double-layered opacity limits an organization’s ability to diagnose performance issues, fine-tune policy enforcement, or validate security outcomes.

“These are closed systems—what I think of as a black box in a black box—that force executives to make risk decisions based on trust rather than evidence,” Phatak added. “That’s why it is critical that independent testing provides evidence-based data on which executives can make decisions.”

CyberRatings is on track to test several other SSE vendors for Threat Protection along with a Comparative Report to be published this summer.

In addition to in-house testing technologies, CyberRatings used Keysight’s CyPerf tool to test performance and TLS/SSL functionality as well as TeraPackets Threat Replayer tool for exploit packet capture replay.

Futuriom: Cloud Firewalls Have Gaping Holes

Posted on by CyberRatings.org

In the release of remarkable firewall test results today, independent nonprofit testing firm CyberRatings.org revealed wild variability in network and cloud firewall efficacy, with special concerns about the firewall instances running in the major public clouds, which seemed not to work very well at all.

In the release of the CyberRatings Q1 2025 Comparative Test Report on Cloud Network Firewalls (CNFWs), many traditional firewalls performed quite well with efficacy ranging at almost 100%. Third-party firewalls from Check Point, Fortinet, Juniper Networks, Palo Alto Networks, and Versa Networks demonstrated the highest security effectiveness blocking exploits and evasion tactics. Results ranged from 99.61% to 100%.

But move into the public cloud, and you get a different story. Some native firewalls from Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure received a 0% Security Effectiveness score as they allowed attacks to bypass existing defenses. In addition, Cisco’s Secure Firewall Defense didn’t receive high ratings, with a 54.5% effectiveness rating and the highest costs per bit of traffic in the bunch.

Read the full article here.

CyberRatings.org Publishes Test Results on Cloud Network Firewalls

Posted on by CyberRatings.org

Austin, TX – April 2, 2025 – CyberRatings.org (CyberRatings), the non-profit entity dedicated to providing confidence in cybersecurity product efficacy, today released its Q1 2025 Comparative Test Report on Cloud Network Firewalls (CNFW), along with separate, in-depth reports for each of the ten cloud firewall solutions tested. Security effectiveness results ranged from 0% to 100%.

Key Findings:

  • Third-party firewalls from Check Point, Fortinet, Juniper Networks, Palo Alto Networks, and Versa Networks demonstrated the highest security effectiveness blocking exploits and evasion tactics. Results ranged from 99.61% to 100%.
  • Native cloud firewalls from Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure offer a convenient alternative, but all received 0% Security Effectiveness as they allowed attacks to bypass existing defenses.
  • Google Cloud Platform’s Next Generation Firewall (NGFW) service leverages Palo Alto Networks technology. We attribute the differences in security effectiveness and performance results between the two platforms to each provider independently selecting and deploying different software versions based on their own criteria.
  • A total of six firewall solutions were Recommended and four received Caution ratings.

In the Cloud Service Provider Native Firewall test from November 2024 only 522 exploits were used in the Part 1 “Mini-Test”, but not evasions. For this round of testing, a greater number of exploits were deployed, and evasions were introduced to the test samples:

  • False Positives: 2,760 samples from various business-critical files and applications, ensuring security measures did not disrupt legitimate traffic.
  • Exploits: 2,028 attack samples from widely exploited vulnerabilities in enterprise environments.
  • Evasion Techniques: 2,500 attacks spanning 27 evasion techniques tested across multiple network layers to bypass firewall defenses.
  • Performance Metrics: 46 different stress and capacity tests under diverse workloads.
  • Stability & Reliability: Seven extended tests simulating prolonged real-world attack and operational scenarios.

CyberRatings evaluated firewall security by testing for evasion detection at three separate layers of the Open Systems Interconnection (OSI) model, specifically Layers 3, 4, and 7. Missing lower-layer evasions had the greatest impact on the overall score because these layers form the foundation of firewall security at the fundamental networking level, and when these lower layers are compromised, the firewall’s primary protective function is undermined. Points were deducted based on the firewall’s ability—or inability—to detect evasions:

  • A missed evasion from the Layer 3 level resulted in a 50% deduction per category, up to a potential category maximum reduction of 100%.
  • Missing a Layer 4 evasion led to a 20% deduction per category, up to a potential category maximum reduction of 60%.
  • A miss at Layer 7 incurred only a 1% deduction per category, up to a potential category maximum reduction of 10%.

Layers 3 and 4 evasions are particularly concerning since all modern applications rely on IP and TCP. Vulnerabilities at these layers can be exploited across a wide range of systems—from cloud services to enterprise applications.

“Until cloud service provider native firewalls provide better protection, customers should be looking to third parties for their cloud security needs,” said Vikram Phatak, CEO of CyberRatings.org. “Traditional third-party security vendors have demonstrated that they bring significant value to customers.”

Below is a summary of the Ratings:

The cloud firewalls were tested using Keysight’s CyPerf v5.0 software testing platform in addition to CyberRatings’ in-house developed test tools. Enterprises can easily perform similar testing with a 2-week free trial from Keysight. Further details of the CyPerf strike library can be found here: https://www.keysight.com/us/en/products/network-test/cloud-test/cyperf.html

Exploring Cloud Service Provider Native Firewalls

Posted on by CyberRatings.org

In April 2024, we published the results of our annual Cloud Network Firewall test. In that test, the AWS Network Firewall exhibited a mere 5.39% Security Effectiveness score, the lowest result in our comparison. That extremely low Security Effectiveness score was not considered to meet any reasonably acceptable standard, and it was concerning enough that we decided to re-evaluate AWS’ offering six months later to see if any improvements might have been made.

We expanded our testing to include Microsoft Azure Firewall and Google Cloud Platform (GCP) Cloud NGFW and published those results on November 26, 2024. Combined, the “Big Three” now account for two-thirds of the growing cloud provider market. Our goal was to assess their native firewall capabilities and understand their strengths and limitations against a variety of attacks.

Part One of a Two-Part Test

This testing was conducted as part one of a two-part series examining cloud network firewalls. It was limited to a subset1 of exploits from Keysight’s CyPerf v5.0 software testing platform. In part two, we will increase the difficulty of the test by widening the scope and depth of testing. The second part of the test will also compare cloud service provider native offerings against market leading third-party cloud network firewall providers. Publication of part 2 will be in Q1 of 2025.

Retesting AWS Network Firewall

Here’s a rundown of our journey with the AWS Network Firewall, including our steps and what we discovered along the way. We decided to publish these additional details so that customers of AWS Firewalls can investigate how the firewall would work in their environments.

Setting Up and Initial Testing

First, we set up the AWS Network Firewall with its basic firewall features, such as access control rules. We wanted to ensure that the fundamental functionalities were working correctly before diving deeper. During our initial testing, these features functioned as expected, effectively controlling access according to our configurations. Then, we executed a series of attacks to test the firewall’s threat detection capabilities. Surprisingly, the AWS Network Firewall only blocked two out of all the attacks we launched. This unexpected result prompted us to question whether the firewall was correctly configured or if there was an issue with our setup.

Double-Checking the Configuration

To rule out any misconfigurations on our part, we carefully double-checked our firewall settings against the official AWS documentation. We reviewed each configuration parameter, ensuring everything matched exactly as prescribed. Our review confirmed that the configuration was indeed correct.

Validating the Firewall Deployment

We conducted additional tests to verify that the AWS Network Firewall was deployed correctly and that all network traffic was routed through it rather than bypassing the firewall and going directly to the internet. We transmitted both normal and malicious traffic across the firewall to observe its behavior.

All traffic forwarded or blocked by the firewall rules was accurately logged in the AWS flow and alert logs via AWS CloudWatch. This validation step reassured us that the firewall was operational and actively monitoring the network traffic as intended.

Diving Deeper with Detailed Analysis

Next, we tested specific threat detection capabilities to confirm that the firewall was functioning correctly. We selected three Suricata rules from the AWS-managed ruleset and generated traffic that matched the malicious content described in those rules.

Case Study: The daddy.linkpc.net Domain

One of the domains we tested was daddy.linkpc.net. The Suricata signature associated with this domain is designed to drop any traffic attempting to access it. Here were the steps we took:

  1. Attempted to Resolve the Domain: We tried to resolve daddy.linkpc.net, mimicking a user or system attempting to access a malicious domain.
  2. Firewall Detection: The AWS Network Firewall detected this malicious activity using the predefined security signature (Signature ID 2853242).
  3. Action Taken: The firewall blocked the DNS query, effectively preventing any communication with the malicious domain.
  4. Observation: The alert triggered by this activity was visible in the AWS Alert Logs, confirming that the firewall’s detection mechanisms were working as expected for this case.

At this stage, we theorized that most of the attacks bypassed the firewall because it lacked specific rules for those particular attacks. To investigate this possibility, we decided to delve deeper into the firewall’s ruleset.

Matching CVEs from AWS Suricata Rules to CyPerf Attacks

Suricata is a high-performance, open-source network analysis and threat detection software. It’s widely used by both private and public organizations and is embedded by major vendors to protect their assets. Suricata uses a signature-based approach to detect threats, which relies on predefined rules and patterns.

CVE stands for Common Vulnerabilities and Exposures. It is a glossary that classifies vulnerabilities. Its purpose is to identify, define, and catalog publicly disclosed cybersecurity vulnerabilities.

To test our theory, we needed to determine whether the AWS Network Firewall had rules corresponding to the attacks Cyperf was using. Here’s how we approached it:

  1. Extracted CVEs from Suricata Rules: We extracted all the CVEs listed in the Suricata rules used by the AWS Network Firewall.
  2. Matched CVEs with CyPerf Attacks: We compared these CVEs with the attacks available in CyPerf to identify those that matched.
  3. Executed Matched Attacks: We then executed only the attacks with corresponding CVEs in the firewall’s ruleset.

The Result: We found very few rules matched CVEs. This is because many Suricata rules did not contain CVE details, and out of those that did, the AWS Firewall only blocked two attacks. Further, most of the signatures were not relevant for cloud environments.

Observations and Concerns

Through our analysis, we observed the following about AWS Network Firewall’s capabilities:

  • Reliance on Basic Regular Expressions: Many signatures in the firewall’s ruleset rely on simple regular expressions. While effective for straightforward patterns, this approach can be insufficient against more sophisticated or obfuscated attacks.
  • Design Focus: Many signatures appear to be designed for home or small office environments rather than for cloud or server workloads. This could limit the firewall’s effectiveness in enterprise or cloud-native scenarios.
  • Total Signatures and Limitations: As of October 2024, the signatures contained over 21,500 preconfigured rules. These signatures are organized into groups, each containing thousands of signatures. Therefore, to enable one, you must enable all in the group. The AWS Firewall also limits the number of signatures you can enable. The maximum number of signatures is 30,000 for stateful and stateless firewall
  • Distribution of Rules:
    • Approximately 10% of the rules protect web browsers.
    • Around 17,500 rules monitor outbound connections (post-infection activities).
    • Only about 4,000 rules focus on inbound traffic.
    • Lack of CVE Information: Most signatures do not contain CVE details, making it challenging to assess the firewall’s coverage against known vulnerabilities.

Assessing the Severity of Limitations

To illustrate the potential impact of these limitations, let’s consider an example involving domain-based signatures:

Example: Circumventing Domain-Based Signatures

  • Signature Limitation: Suppose there’s a signature designed to block traffic to cnn.com.
  • Potential Circumvention: An attacker could easily bypass this signature by slightly altering the domain name to cnn.com, using a subdomain like news.cnn.com, or adding a character before or after cnn.com.
  • Issue: If the signature only matches the exact domain cnn.com or tries to match a precise number of characters, it fails to detect these variations, allowing malicious traffic to pass undetected.
  • Implication: This example demonstrates how attackers can exploit simple signatures by making minor changes, highlighting the need for more robust and comprehensive detection mechanisms.

Our Journey with Other Cloud Service Provider Firewalls

Azure

Unfortunately, we came into this test blind; Azure seems to use a propriety, closed-source security stack. It provides little visibility into how it works or how to change anything. You pay for it, turn it on, and it does “stuff.” This limitation is frustrating, given the results, as it removes the user’s ability to adjust protection based on their unique deployment.

The lack of visibility into rules and/or related CVE information limits the user’s ability to understand how well the protection aligns with their environment, applications, and workloads.

Google Cloud Firewall (GCP)

Like Azure, this is also closed-source security software. However, we discovered that GCP’s firewall is running software from Palo Alto Networks. We have worked with Palo Alto Networks for many years and expected much better results than we saw reflected in the scores. GCP directs you to the Palo Alto Networks website for information on the deployed protection.
Recommendations:

  • Third-Party Firewall: Until Cloud Service Provider’s Native Firewalls offer adequate protection, we recommend using a third-party cloud firewall (offered through the respective CSP’s marketplace).
  • Custom Rule Creation: Consider creating custom rules tailored to your specific environment and attack surface.
  • Supplementary Security Measures: Consider using additional security technologies to complement the Cloud Network
  • Update & Test: Update your firewall regularly and test each update. Attackers are continuously innovating; defenses that work today may not work tomorrow. It is important that your defenses are current and that you verify they work as expected.
  • Supplementary Security Measures: Consider using additional security technologies to complement the Cloud Network Firewalls based on test results.

Our journey highlighted the importance of not solely relying on default configurations and the need for a proactive approach to network security. By understanding the capabilities and limitations of tools like Cloud Service Provider Firewalls, organizations can better protect their assets and respond to evolving threats.

For further information on how to best configure these products, please reference Best Practices for Cloud Network Firewall Deployment in 2024: Cloud Service Providers (CSP).

[1] We limited the scope of exploits to those that: 1) targeted servers, 2) were applicable to applications or workloads that could be run on a cloud virtual machine or bare metal platform, 3) were for known exploits over the last 10 years with a CVSS score of medium to high.

 

Best Practices for Cloud Network Firewall Deployment in 2024: Cloud Service Providers (CSP)

Posted on by CyberRatings.org

This guide supports the Cloud Network Firewall (CNFW) mini-test, which compares the security effectiveness of native firewall solutions from Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP).

This guide should supplement what vendors already provide to their customers. Please see the links below for the best practices and guides for each product we tested. We have also included information for Keysight’s CyPerf v5.0 software testing platform, enabling enterprises to easily replicate our results.

Cloud Network Firewall Test Topology

AWS Network Firewall

Product, best practices, and documentation:

Following AWS’s documentation, the CyberRatings team deployed the AWS Network Firewall instance in routing mode to inspect both inbound and outbound traffic. We set up and configured our threat testing harness using CyPerf, which was installed using the AWS Marketplace.

Deployment Steps:

  1. Route Tables: The route tables must be configured to properly route traffic across the AWS Firewall (AWS FW). Three necessary routing tables are required, and they are described below.
  2. Control Subnet: routes traffic from and to the Control Subnet, the subnet on which the AWS FW endpoint has been deployed.
  3. Customer Subnet: routes traffic from and to the Customer Subnet, which is the subnet on which the trusted clients and servers are deployed, i.e., the LAN side of the Firewall.
  4. Public Internet Gateway: routes traffic from and to the Customer Subnet, which is the subnet on which the trusted clients and servers are deployed, i.e., the WAN side of the Firewall.
  5. Firewall Policies: create rule groups for various traffic types, including allowed and blocked IPs and protocols.
  6. Firewall Subnet: configure in each VPC to direct traffic through the firewall.
  7. Logging & Monitoring: Lastly, we enabled logging to store data in CloudWatch for auditing and monitoring purposes (S3 can also be utilized).

For AWS Network Firewall, multiple steps are required:

  1. Enable logging (both flow and alert logs)
  2. Forward logging output to one or more AWS services (CloudWatch, S3 Storage, etc.)
  3. Then, we analyzed the logs (in JSON format). They can also be exported to multiple formats to be viewed locally.

CyPerf: We used AWS Marketplace and installation was straightforward.

Google Cloud NGFW Enterprise Firewall

Product, best practices, and documentation:

We followed their best practices and documentation and deployed the instance in routing mode to inspect both inbound and outbound traffic. We set up and configured our threat testing harness using CyPerf. Using the provided instructions and documentation, it was easy to deploy.

Cloud NGFW’s threat detection and prevention capabilities are powered by Palo Alto Networks threat prevention technologies[1]. To help protect your network, Cloud NGFW supports a default set of threat signatures with predefined severity levels. Users can view all the threat signatures configured in Cloud NGFW in the threat vault.

Firewall Endpoints:

Firewall Policy:

CyPerf: The CyPerf agents were deployed on GCP using Terraform. The procedure for deploying using Terraform is shown below.

Installation of Google Cloud Software Development Kit (SDK):

  1. Reference: Install the gcloud CLI  |  Google Cloud CLI Documentation
  2. Next step: https://github.com/Keysight/cyperf/tree/main/deployment/gcp/terraform
  3. Once the GCP SDK has been installed successfully, install the agents using Terraform. (These were obtained from Cyperf): https://github.com/Keysight/cyperf/tree/main/deployment/gcp/terraform

Microsoft Azure Firewall Premium

Product, best practices, and documentation:

We followed their best practices and documentation for Microsoft Azure Premium Firewall and deployed the instance in routing mode to inspect both inbound and outbound traffic. We set up and configured our threat testing harness using CyPerf.

Microsoft Azure Firewall Premium uses Microsoft’s closed-source signatures. As of October 2024, its ruleset contained over 67,000 rules in over 50 categories.

Deployment Steps:

  1. First, we installed Terraform: https://learn.hashicorp.com/tutorials/terraform/install-cli
  2. Then we installed Azure CLI: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli
  3. Firewall Policy:
  4. Firewall IDPS Policy:
  5. Firewall Threat Intelligence Policy:
  6. Lastly, we set up logging and forwarded logs to NetWatcher, another service in Azure. The logs could then be analyzed in multiple formats.

CyPerf: https://github.com/Keysight/cyperf/tree/main/deployment/azure/terraform/controller_and_agent_pair

[1] https://cloud.google.com/firewall/docs/about-threats