Tag: Fortinet

SDxCentral: Palo Alto Networks and Fortinet given all clear after firewall hiccups

Posted on by CyberRatings.org

Palo Alto Networks and Fortinet have received a clean bill of health for their firewall protections, while the jury is still out on current Cisco defenses.

CyberRatings.org recommended both Palo Alto and Fortinet after new tests confirmed they had patched evasions previously discovered by the security testing firm.

In tests carried out at the start of the month by CyberRatings’ testing partner NSS Labs, researchers found they were able to bypass protection using Layer 4 TCP evasions in both Palo Alto’s PAN-OS (version 11.2.8-c537) and Fortinet’s IPS (v7.01154), as well as evading Layer 3 IP in the Palo Alto operating system.

Both firms reacted quickly, with Palo Alto developing an updated PAN-OS firmware package (PAN-OS 11.2.10-c37) and Fortinet deploying an updated IPS package (v7.01165 (33.00064) to fix the vulnerabilities.

Read the full article here.

CyberRatings.org and NSS Labs Announce Follow-On Enterprise Firewall Results

Posted on by CyberRatings.org

Austin, TX – November 25, 2025 – CyberRatings.org (CyberRatings), the non-profit organization dedicated to providing confidence in cybersecurity products and services through independent testing, today announced Follow-On Test Results for the Fortinet FortiGate-200G and Palo Alto Networks PA-1410 Enterprise Firewalls.

Both products have improved their ratings from Caution to Recommended following submissions to NSS Labs to retest after developing new builds to address their earlier evasion resistance deficiencies published on November 5, 2025.

“Both Fortinet and Palo Alto Networks responded quickly and transparently to our original findings, issuing updates within days and requesting immediate retesting,” said Vikram Phatak, CEO of NSS Labs. “The speed at which these vendors addressed and resolved critical issues shows their commitment to their customers’ security.”

Fortinet Follow-On Results

During the initial test of Fortinet’s v7.6.4 build3596 with IPS v7.01154 (33.00064), NSS Labs was able to bypass protection using Layer 4 TCP evasions. Fortinet responded quickly to develop an updated IPS signature package. After retesting, NSS Labs confirmed that the update addressed all exploit evasion resistance deficiencies.

Exploit evasion resistance increased from 60% to 100%, elevating the overall Security Effectiveness from 79.24% to 99.24%. Organizations running IPS version v7.01154 (33.00064) or earlier should upgrade immediately to v7.01165 (33.00064) to ensure protection against evasion techniques as detailed in the November 5 publication.

Palo Alto Networks Follow-On Results

During the initial test of PAN-OS 11.2.8-c537, NSS Labs was able to bypass protection using Layer 3 IP and Layer 4 TCP evasions. Palo Alto Networks responded quickly to develop an updated PAN-OS firmware package (PAN-OS 11.2.10-c37) to ensure that the problem had been fixed. After retesting, NSS Labs confirmed that the updated firmware addressed all exploit evasion resistance deficiencies, providing substantial improvements in protection.

Exploit evasion resistance increased dramatically from 0% to 100%, elevating the overall Security Effectiveness from 46.37% to 96.07%.

NSS Labs notes that it is not unusual for vendors to submit pre-release software or firmware intended for imminent release, which NSS Labs requires to be scheduled for general availability within 90 days following a test. Palo Alto Networks confirmed that PAN-OS version 11.2.10-c37 was provided as a pre-release and will be designated as PAN-OS 11.2.10 upon reaching general availability.

Organizations running PAN-OS 11.2.8-c537 or earlier should immediately request PAN-OS 11.2.10 to ensure protection against evasion techniques as detailed in the November 5 publication.

Context and Vendor Accountability

These follow-on results reaffirm the importance of independent testing and vendor accountability. Both vendors’ prompt response demonstrates how transparency and rapid engineering benefit customers.

To accompany these follow-on reports, NSS Labs published a blog titled When Firewalls Fail Gracefully: Why Vendor Responsiveness Matters as Much as Security Effectiveness, highlighting the importance of transparency and quick remediation in cybersecurity engineering.

Testing Methodology

The follow-on tests were conducted using the same methodology and datasets employed in the original Q4 2025 Enterprise Firewall Comparative Report, which evaluated seven leading products under real-world conditions. The updated results now place Fortinet and Palo Alto Networks in the Recommended category alongside Check Point, Juniper Networks, and Versa Networks.

Tests were conducted by NSS Labs developed technologies and Keysight’s CyPerf tool to evaluate security, performance, TLS functionality, and stability. The updated test reports are available at no cost on the CyberRatings.org website.

When Firewalls Fail Gracefully

Posted on by CyberRatings.org

The latest NSS Labs Enterprise Firewall Comparative Report was published this month and, as usual, provided a deep insight into the state of the enterprise firewall market.

Seven of the most widely deployed products were tested using real-world attack scenarios, enterprise-grade workloads, and adversarial evasion techniques to measure their resilience, reliability, and performance.

The results reveal a security landscape that remains uneven: most products blocked the majority of exploits and malware, but a few stumbled when exposed to modern, and not so modern, evasion techniques.

However, the story doesn’t end with the Comparative Security Map – it is also a case study in vendor accountability. How vendors respond when weaknesses are exposed in independent tests such as this tells us a lot about how they are likely to support their enterprise customers in a pinch. It also tells us how seriously they take engineering challenges that could result in serious failures, or even breaches, when installed in live environments.

Palo Alto Networks and Fortinet, though not the highest-scoring participants, stand out precisely because they treated the findings as an opportunity to rectify shortcomings in their products that could have a serious impact on their customers. Within days of publication, both vendors confirmed patches for the issues identified and scheduled retests for the affected products. That kind of responsiveness deserves as much attention as raw test scores.

Read the full blog from NSS Labs: https://nsslabs.com/media/blog/when-firewalls-fail-gracefully/

SDxCentral: SSE protection found uneven across major vendors

Posted on by CyberRatings.org

Researchers reported major disparity in security effectiveness of security service edge (SSE) protection across major vendors.

Non-profit cyberratings.org/ (CyberRatings) found security effectiveness ranged from less than 3% to 100% in its testing of vendor products, with only Fortinet, Palo Alto Networks, Versa Networks, and Zscaler earning a “recommended” rating.

In contrast, SSE products from Cisco, Cloudflare, and Skyhigh were tagged with a “caution” label, indicating “below-average” security effectiveness with the recommendation that end users “should consider seeking other solutions.” The ratings were put down due to “failures in critical tests.”

Read the full article here.

CyberRatings.org Test Results Reveal Critical Failures in SSE

Posted on by CyberRatings.org

Austin, TX – July 16, 2025 – CyberRatings.org (CyberRatings), the non-profit organization dedicated to providing insight into the capabilities of cybersecurity products and services through independent testing, today announced the comparative results of its latest Security Service Edge (SSE) evaluation. The findings expose a striking disparity in product performance: Security Effectiveness ranged from 2.95% to 100%, underscoring just how uneven SSE protection remains across vendors.

Only Fortinet, Palo Alto Networks, Versa Networks, and Zscaler earned a Recommended rating, while products from Cisco, Cloudflare, and Skyhigh were rated Caution due to failures in critical tests.

Despite meeting our inclusion criteria and high market interest, we were unable to include Cato Networks and Netskope in this test. Netskope’s high entry level licensing cost and their lack of responsiveness to our inquiries to purchase their product rendered it inaccessible. Cato was explicit in their refusal to engage with us or allow us to procure licensing for any form of independent third-party validation.

“With cloud-delivered products rapidly evolving through continuous integration and deployment, customers have little visibility into what changes under the hood,” said Vikram Phatak, CEO of CyberRatings.org. “Only by conducting regular independent testing can enterprises ensure they’re not left vulnerable to silent failures that could go unnoticed for months.”

Of all the SSE test criteria, blocking evasions had the most impact on security effectiveness. Evasion techniques are used by threat actors to disguise or modify attacks, so they slip past defenses. While most products excelled at blocking known malware and exploits, three failed to stop evasions — exposing organizations to entire classes of undetected attacks.

These independent tests uniquely stress real-world evasion techniques that standard evaluations often overlook — the techniques cybercriminals rely on to bypass security measures.

The SSE evaluation was designed to reflect modern, adversarial conditions and covered:

  • Malware: 6,184 malware samples in active use by global threat actors.
  • Exploits: 205 exploits of known vulnerabilities.
  • Evasions: 1,154 evasions spanning 37 categories of techniques.
  • False Positives: 1,514 legitimate files and applications, verifying security measures do not impact users and operations.
  • TLS/SSL: Encrypted attacks using cipher suites that represent ~97% of real-world HTTPS traffic.

Security Service Edge is inherently complex — a multi-layered technology stacked atop ever-changing cloud environments. Customers typically have minimal visibility into how these systems operate and testing them independently is challenging. This double-layered opacity makes third-party validation essential to diagnose performance issues, fine-tune policy enforcement, and ensure real security outcomes. CyberRatings strongly urges organizations to adopt periodic or ongoing third-party testing to ensure consistent protection and compliance.

NSS Labs is the Official Testing Partner of CyberRatings. Keysight’s CyPerf tool was used for performance and TLS/SSL functionality, and TeraPackets Threat Replayer tool was used for exploit replay validation.

Futuriom: Cloud Firewalls Have Gaping Holes

Posted on by CyberRatings.org

In the release of remarkable firewall test results today, independent nonprofit testing firm CyberRatings.org revealed wild variability in network and cloud firewall efficacy, with special concerns about the firewall instances running in the major public clouds, which seemed not to work very well at all.

In the release of the CyberRatings Q1 2025 Comparative Test Report on Cloud Network Firewalls (CNFWs), many traditional firewalls performed quite well with efficacy ranging at almost 100%. Third-party firewalls from Check Point, Fortinet, Juniper Networks, Palo Alto Networks, and Versa Networks demonstrated the highest security effectiveness blocking exploits and evasion tactics. Results ranged from 99.61% to 100%.

But move into the public cloud, and you get a different story. Some native firewalls from Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure received a 0% Security Effectiveness score as they allowed attacks to bypass existing defenses. In addition, Cisco’s Secure Firewall Defense didn’t receive high ratings, with a 54.5% effectiveness rating and the highest costs per bit of traffic in the bunch.

Read the full article here.

CyberRatings.org Publishes Test Results on Cloud Network Firewalls

Posted on by CyberRatings.org

Austin, TX – April 2, 2025 – CyberRatings.org (CyberRatings), the non-profit entity dedicated to providing confidence in cybersecurity product efficacy, today released its Q1 2025 Comparative Test Report on Cloud Network Firewalls (CNFW), along with separate, in-depth reports for each of the ten cloud firewall solutions tested. Security effectiveness results ranged from 0% to 100%.

Key Findings:

  • Third-party firewalls from Check Point, Fortinet, Juniper Networks, Palo Alto Networks, and Versa Networks demonstrated the highest security effectiveness blocking exploits and evasion tactics. Results ranged from 99.61% to 100%.
  • Native cloud firewalls from Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure offer a convenient alternative, but all received 0% Security Effectiveness as they allowed attacks to bypass existing defenses.
  • Google Cloud Platform’s Next Generation Firewall (NGFW) service leverages Palo Alto Networks technology. We attribute the differences in security effectiveness and performance results between the two platforms to each provider independently selecting and deploying different software versions based on their own criteria.
  • A total of six firewall solutions were Recommended and four received Caution ratings.

In the Cloud Service Provider Native Firewall test from November 2024 only 522 exploits were used in the Part 1 “Mini-Test”, but not evasions. For this round of testing, a greater number of exploits were deployed, and evasions were introduced to the test samples:

  • False Positives: 2,760 samples from various business-critical files and applications, ensuring security measures did not disrupt legitimate traffic.
  • Exploits: 2,028 attack samples from widely exploited vulnerabilities in enterprise environments.
  • Evasion Techniques: 2,500 attacks spanning 27 evasion techniques tested across multiple network layers to bypass firewall defenses.
  • Performance Metrics: 46 different stress and capacity tests under diverse workloads.
  • Stability & Reliability: Seven extended tests simulating prolonged real-world attack and operational scenarios.

CyberRatings evaluated firewall security by testing for evasion detection at three separate layers of the Open Systems Interconnection (OSI) model, specifically Layers 3, 4, and 7. Missing lower-layer evasions had the greatest impact on the overall score because these layers form the foundation of firewall security at the fundamental networking level, and when these lower layers are compromised, the firewall’s primary protective function is undermined. Points were deducted based on the firewall’s ability—or inability—to detect evasions:

  • A missed evasion from the Layer 3 level resulted in a 50% deduction per category, up to a potential category maximum reduction of 100%.
  • Missing a Layer 4 evasion led to a 20% deduction per category, up to a potential category maximum reduction of 60%.
  • A miss at Layer 7 incurred only a 1% deduction per category, up to a potential category maximum reduction of 10%.

Layers 3 and 4 evasions are particularly concerning since all modern applications rely on IP and TCP. Vulnerabilities at these layers can be exploited across a wide range of systems—from cloud services to enterprise applications.

“Until cloud service provider native firewalls provide better protection, customers should be looking to third parties for their cloud security needs,” said Vikram Phatak, CEO of CyberRatings.org. “Traditional third-party security vendors have demonstrated that they bring significant value to customers.”

Below is a summary of the Ratings:

The cloud firewalls were tested using Keysight’s CyPerf v5.0 software testing platform in addition to CyberRatings’ in-house developed test tools. Enterprises can easily perform similar testing with a 2-week free trial from Keysight. Further details of the CyPerf strike library can be found here: https://www.keysight.com/us/en/products/network-test/cloud-test/cyperf.html

CyberRatings.org Announces Test Results for Fortinet Unified Secure Access Service Edge (FortiSASE)

Posted on by CyberRatings.org

Austin, TX – December 4, 2024 – CyberRatings.org (CyberRatings), the non-profit entity dedicated to providing confidence in cybersecurity products and services through its research and testing programs, has completed an independent test of Fortinet Unified Secure Access Service Edge (FortiSASE).

The FortiSASE was tested for Security Service Edge (SSE) Threat Protection, and measured on how it defended against 205 exploits, 7,140 wild malware samples and whether any of 1,124 evasions could bypass its protection. The product was also tested on how it handled TLS/SSL 1.2 and 1.3 cipher suites.

Threat actors apply evasion techniques to disguise and modify attacks to avoid detection by security products. Therefore, it is imperative that an SSE correctly handles evasions. An attacker can bypass protection if an SSE fails to detect a single form of evasion. Fortinet resisted 1,124 out of 1,124 evasions.

FortiSASE received a “AAA” rating after achieving a 98.53% Protection Rate for blocking 99.02% of Exploits, 99.50% of Malware and 100% of Evasions. TLS/SSL Functionality scored at 100%.

The combined measurements to determine the overall Protection Rate also included false positives, which is a key to correctly identifying and allowing legitimate traffic while protecting against malware, exploits, and phishing attacks. False positive tests assessed Fortinet’s ability to block attacks while permitting legitimate traffic, achieving 100% for browsing and 99.83% for file downloads.

FortiSASE also received a “AAA” rating for Zero Trust Network Access (ZTNA). Authentication & Identity were 100%, Resource Access achieved 100%, Routing & Policy Enforcement tested at 95% and TLS/SSL Functionality scored at 100%.

“Fortinet handled our variety of use cases with ease and demonstrated that they could block attacks under a wide range of conditions. Their offering should be on everyone’s short list,” said Vikram Phatak, CEO of CyberRatings.org.

CyberRatings is on track to test several other SSE vendors for Threat Protection along with Software-Defined Wide Area Network (SD-WAN), and Zero Trust Network Access (ZTNA) bringing together the Secure Access Service Edge (SASE) package of test results to be published in the coming months.

Keysight provided its CyPerf tool to test performance and TLS/SSL functionality. TeraPackets provided its Threat Replayer tool for exploit packet capture replay.

MEF: 15 Leading Technology and Service Providers Achieve SASE Certification in Industry’s Only Independent Certification Program

Posted on by CyberRatings.org

DALLAS, Texas, October 29, 2024 – MEF, a global consortium of network, cloud, security, and technology providers driving enterprise digital transformation, today announced significant advancements in its MEF 3.0 Secure Access Service Edge (SASE) Certification Program. Technology providers Fortinet and Versa have achieved full SASE certification, while service providers AT&T, BT, Colt, Comcast Business, Console Connect, Liberty Latin America, Lumen, Orange Business, TPG, and Verizon have also earned full SASE certification. Additionally, technology providers Broadcom Inc. and Palo Alto Networks, and service provider Sparkle, are expected to achieve full SASE certification shortly. Organizations that achieve SASE certification through MEF’s rigorous independent program receive a rating on product effectiveness and are listed in MEF’s registry of certified companies. SASE certification is now available to all MEF members.

Read the full press release here.

MEF: Leading Technology Providers Achieve First Milestone in SASE Certification Program

Posted on by CyberRatings.org

LOS ANGELES, Calif., August 28, 2024 – MEF, a global consortium of network, cloud, security, and technology providers accelerating enterprise digital transformation, today announced new certifications in the first module of its Secure Access Service Edge (SASE) products and services certification program. SD-WAN certification was achieved by Broadcom, Inc., Fortinet, and Versa. Palo Alto Networks is expected to achieve its SD-WAN certification shortly. Certified technology providers have received a rating on product effectiveness and will be listed in MEF’s registry of certified organizations. MEF’s SASE certification program is now generally available to technology and service provider members.

Enterprises worldwide have adopted SD-WAN to enable digital transformation, addressing changing workforce needs and cloud migration. SD-WAN offers improved application performance, centralized management, optimized connectivity, agility, security, cost, and other benefits. As a key component of SASE, SD-WAN has become increasingly integrated with cybersecurity solutions to protect distributed environments.

Read the full press release here.