Tag: SSE

InsideCybersecurity: Cyber assessment firm identifies evasion vulnerabilities in enterprise firewall products

Posted on by CyberRatings.org

A nonprofit cyber assessment firm found vulnerabilities in the ability of widely used enterprise firewall products to block transport and network-layer evasions commonly deployed by cyber attackers, in a report examining the effectiveness of security offerings.

“Enterprise Firewalls are constantly evolving to combat new attacker techniques and tools but sometimes that evolution takes a wrong turn. A vendor can have a near-perfect detection engine but if attackers can bypass that engine it gives them a clear path through your defenses,” CyberRatings.org CEO Vikram Phatak sad in a Nov. 5 release.

CyberRatings is a nonprofit organization conducting independent testing of cybersecurity products through its testing partner firm, NSS Labs.

CyberRatings evaluated the “security effectiveness” of seven firewall products in 55 performance tests using 3,326 exploits, 11,311 malware samples, 5,752 evasion techniques in 53 evasion categories and 6,481 false-positive samples,” according to the report.

Read the full article here.

CyberRatings.org and NSS Labs Announce Follow-On Enterprise Firewall Results

Posted on by CyberRatings.org

Austin, TX – November 25, 2025 – CyberRatings.org (CyberRatings), the non-profit organization dedicated to providing confidence in cybersecurity products and services through independent testing, today announced Follow-On Test Results for the Fortinet FortiGate-200G and Palo Alto Networks PA-1410 Enterprise Firewalls.

Both products have improved their ratings from Caution to Recommended following submissions to NSS Labs to retest after developing new builds to address their earlier evasion resistance deficiencies published on November 5, 2025.

“Both Fortinet and Palo Alto Networks responded quickly and transparently to our original findings, issuing updates within days and requesting immediate retesting,” said Vikram Phatak, CEO of NSS Labs. “The speed at which these vendors addressed and resolved critical issues shows their commitment to their customers’ security.”

Fortinet Follow-On Results

During the initial test of Fortinet’s v7.6.4 build3596 with IPS v7.01154 (33.00064), NSS Labs was able to bypass protection using Layer 4 TCP evasions. Fortinet responded quickly to develop an updated IPS signature package. After retesting, NSS Labs confirmed that the update addressed all exploit evasion resistance deficiencies.

Exploit evasion resistance increased from 60% to 100%, elevating the overall Security Effectiveness from 79.24% to 99.24%. Organizations running IPS version v7.01154 (33.00064) or earlier should upgrade immediately to v7.01165 (33.00064) to ensure protection against evasion techniques as detailed in the November 5 publication.

Palo Alto Networks Follow-On Results

During the initial test of PAN-OS 11.2.8-c537, NSS Labs was able to bypass protection using Layer 3 IP and Layer 4 TCP evasions. Palo Alto Networks responded quickly to develop an updated PAN-OS firmware package (PAN-OS 11.2.10-c37) to ensure that the problem had been fixed. After retesting, NSS Labs confirmed that the updated firmware addressed all exploit evasion resistance deficiencies, providing substantial improvements in protection.

Exploit evasion resistance increased dramatically from 0% to 100%, elevating the overall Security Effectiveness from 46.37% to 96.07%.

NSS Labs notes that it is not unusual for vendors to submit pre-release software or firmware intended for imminent release, which NSS Labs requires to be scheduled for general availability within 90 days following a test. Palo Alto Networks confirmed that PAN-OS version 11.2.10-c37 was provided as a pre-release and will be designated as PAN-OS 11.2.10 upon reaching general availability.

Organizations running PAN-OS 11.2.8-c537 or earlier should immediately request PAN-OS 11.2.10 to ensure protection against evasion techniques as detailed in the November 5 publication.

Context and Vendor Accountability

These follow-on results reaffirm the importance of independent testing and vendor accountability. Both vendors’ prompt response demonstrates how transparency and rapid engineering benefit customers.

To accompany these follow-on reports, NSS Labs published a blog titled When Firewalls Fail Gracefully: Why Vendor Responsiveness Matters as Much as Security Effectiveness, highlighting the importance of transparency and quick remediation in cybersecurity engineering.

Testing Methodology

The follow-on tests were conducted using the same methodology and datasets employed in the original Q4 2025 Enterprise Firewall Comparative Report, which evaluated seven leading products under real-world conditions. The updated results now place Fortinet and Palo Alto Networks in the Recommended category alongside Check Point, Juniper Networks, and Versa Networks.

Tests were conducted by NSS Labs developed technologies and Keysight’s CyPerf tool to evaluate security, performance, TLS functionality, and stability. The updated test reports are available at no cost on the CyberRatings.org website.

When Firewalls Fail Gracefully

Posted on by CyberRatings.org

The latest NSS Labs Enterprise Firewall Comparative Report was published this month and, as usual, provided a deep insight into the state of the enterprise firewall market.

Seven of the most widely deployed products were tested using real-world attack scenarios, enterprise-grade workloads, and adversarial evasion techniques to measure their resilience, reliability, and performance.

The results reveal a security landscape that remains uneven: most products blocked the majority of exploits and malware, but a few stumbled when exposed to modern, and not so modern, evasion techniques.

However, the story doesn’t end with the Comparative Security Map – it is also a case study in vendor accountability. How vendors respond when weaknesses are exposed in independent tests such as this tells us a lot about how they are likely to support their enterprise customers in a pinch. It also tells us how seriously they take engineering challenges that could result in serious failures, or even breaches, when installed in live environments.

Palo Alto Networks and Fortinet, though not the highest-scoring participants, stand out precisely because they treated the findings as an opportunity to rectify shortcomings in their products that could have a serious impact on their customers. Within days of publication, both vendors confirmed patches for the issues identified and scheduled retests for the affected products. That kind of responsiveness deserves as much attention as raw test scores.

Read the full blog from NSS Labs: https://nsslabs.com/media/blog/when-firewalls-fail-gracefully/

Inside Cybersecurity: Cyber Assessment Firm Identifies Evasion Vulnerabilities in Enterprise Firewall Products

Posted on by CyberRatings.org

A nonprofit cyber assessment firm found vulnerabilities in the ability of widely used enterprise firewall products to block transport and network-layer evasions commonly deployed by cyber attackers, in a report examining the effectiveness of security offerings.

“Enterprise Firewalls are constantly evolving to combat new attacker techniques and tools but sometimes that evolution takes a wrong turn. A vendor can have a near-perfect detection engine but if attackers can bypass that engine it gives them a clear path through your defenses,” CyberRatings.org CEO Vikram Phatak said in a Nov. 5 release.

Read the full article here.

CyberRatings.org and NSS Labs Announce 2025 Enterprise Firewall Test Results

Posted on by CyberRatings.org

Austin, TX – November 5, 2025 – CyberRatings.org (CyberRatings), the non-profit organization dedicated to providing confidence in cybersecurity products and services through independent testing, today announced the results of its latest Enterprise Firewall (EFW) evaluation.  Tests were conducted by NSS Labs and are now available at no cost on the CyberRatings.org website.

NSS Labs performed independent evaluations of seven leading Enterprise Firewall products using the Enterprise Firewall Test Methodology v3.0. The testing revealed a striking disparity in performance — Security Effectiveness ranged from 46.37% to 99.59%.

Firewalls were tested under encrypted enterprise-grade workloads using 3,326 exploits, 11,311 malware samples, 5,752 evasion techniques spanning 53 evasion categories, 6,481 false-positive samples, and 55 performance tests. Each firewall was required to maintain operational stability throughout testing.

Key Findings

  • Attackers Are Bypassing Defenses: While average exploit and malware block rates exceeded 96%, three widely deployed vendors failed critical evasion tests that significantly reduced their effectiveness. Only three of seven products earned aRecommended
  • Evasion Vulnerabilities: Common transport and network-layer evasions, techniques that can be applied to nearly every attack, bypassed some of the world’s most widely used firewalls.
  • Encrypted Threats: More than 95% of global web traffic is encrypted. Detecting attacks hidden within TLS/SSL sessions remains a crucial differentiator; some products showed marked performance degradation when inspecting encrypted traffic.
  • Accuracy Matters: One product recorded only 80% false-positive accuracy, potentially increasing operational costs and reducing trust in security alerts as customers disable protections to reduce noise.

“Enterprise Firewalls are constantly evolving to combat new attacker techniques and tools but sometimes that evolution takes a wrong turn,” said Vikram Phatak, CEO of CyberRatings.org. “A vendor can have a near-perfect detection engine but if attackers can bypass that engine it gives them a clear path through your defenses.”

The test results are as follows:

NSS Labs is the Official Testing Partner of CyberRatings, generating the test results and reports for CyberRatings publications. NSS Labs developed tools and Keysight’s CyPerf tool were used to test the security, performance, TLS functionality, and stability of Enterprise Firewalls.

The Enterprise Firewall Test Reports, Comparative Report and Security Map are available at CyberRatings.org.

CyberRatings.org Removes Paywall, Making All Cybersecurity Product Test Reports Free to Access

Posted on by CyberRatings.org

AUSTIN, Texas – November 4, 2025 — CyberRatings.org, the nonprofit member organization dedicated to promoting transparency and accountability in cybersecurity products, today announced a major step forward in its mission to empower organizations with trusted, independent data. All cybersecurity product test reports, previously available only through paid access, are now free to view and download on the CyberRatings.org website.

This initiative reflects CyberRatings’ unwavering commitment to transparency and its belief that informed decisions are the cornerstone of effective cybersecurity. By removing the paywall, CyberRatings ensures that enterprises, vendors, analysts, and the broader cybersecurity community can freely access vital data and insights.

“A rising tide lifts all boats,” said Vikram Phatak, CEO of CyberRatings.org. “By opening access to our reports, we’re inviting the global cybersecurity community to learn from our data, compare results, and collectively improve defenses.”

The decision reflects CyberRatings.org’s belief that greater transparency and access to independent data strengthen the entire cybersecurity community. Through rigorous evaluations of products and services—spanning enterprise, cloud, and small business firewalls; security service edge; software-defined wide area networks (SD-WAN); AI Protection; and more—CyberRatings delivers unbiased data that helps organizations understand real-world performance.

Since its inception, CyberRatings.org has worked to build trust through independence and openness. Earlier this year, the organization named NSS Labs as its official testing partner, reinforcing its dedication to credible, data-driven assessments that serve the public interest.

Visitors can now access all reports at cyberratings.org free of charge.

SDxCentral: SSE protection found uneven across major vendors

Posted on by CyberRatings.org

Researchers reported major disparity in security effectiveness of security service edge (SSE) protection across major vendors.

Non-profit cyberratings.org/ (CyberRatings) found security effectiveness ranged from less than 3% to 100% in its testing of vendor products, with only Fortinet, Palo Alto Networks, Versa Networks, and Zscaler earning a “recommended” rating.

In contrast, SSE products from Cisco, Cloudflare, and Skyhigh were tagged with a “caution” label, indicating “below-average” security effectiveness with the recommendation that end users “should consider seeking other solutions.” The ratings were put down due to “failures in critical tests.”

Read the full article here.

Inside Cybersecurity: Cyber assessment firm finds major security gaps in cloud service edge products from leading tech companies

Posted on by CyberRatings.org

A nonprofit cyber assessment firm highlights the importance of evasion testing in a recent report evaluating the cybersecurity of seven “security service edge” products from tech companies.

“While most products excelled at blocking known malware exploits, three failed to stop evasions—exposing organizations to entire classes of undetected attacks. These independent tests uniquely stress real-world evasion techniques that standard evaluations often overlook—the techniques cybercriminals rely on to bypass security measures,” according to a July 16 release on the latest report…

Read the full article here.

CyberRatings.org Test Results Reveal Critical Failures in SSE

Posted on by CyberRatings.org

Austin, TX – July 16, 2025 – CyberRatings.org (CyberRatings), the non-profit organization dedicated to providing insight into the capabilities of cybersecurity products and services through independent testing, today announced the comparative results of its latest Security Service Edge (SSE) evaluation. The findings expose a striking disparity in product performance: Security Effectiveness ranged from 2.95% to 100%, underscoring just how uneven SSE protection remains across vendors.

Only Fortinet, Palo Alto Networks, Versa Networks, and Zscaler earned a Recommended rating, while products from Cisco, Cloudflare, and Skyhigh were rated Caution due to failures in critical tests.

Despite meeting our inclusion criteria and high market interest, we were unable to include Cato Networks and Netskope in this test. Netskope’s high entry level licensing cost and their lack of responsiveness to our inquiries to purchase their product rendered it inaccessible. Cato was explicit in their refusal to engage with us or allow us to procure licensing for any form of independent third-party validation.

“With cloud-delivered products rapidly evolving through continuous integration and deployment, customers have little visibility into what changes under the hood,” said Vikram Phatak, CEO of CyberRatings.org. “Only by conducting regular independent testing can enterprises ensure they’re not left vulnerable to silent failures that could go unnoticed for months.”

Of all the SSE test criteria, blocking evasions had the most impact on security effectiveness. Evasion techniques are used by threat actors to disguise or modify attacks, so they slip past defenses. While most products excelled at blocking known malware and exploits, three failed to stop evasions — exposing organizations to entire classes of undetected attacks.

These independent tests uniquely stress real-world evasion techniques that standard evaluations often overlook — the techniques cybercriminals rely on to bypass security measures.

The SSE evaluation was designed to reflect modern, adversarial conditions and covered:

  • Malware: 6,184 malware samples in active use by global threat actors.
  • Exploits: 205 exploits of known vulnerabilities.
  • Evasions: 1,154 evasions spanning 37 categories of techniques.
  • False Positives: 1,514 legitimate files and applications, verifying security measures do not impact users and operations.
  • TLS/SSL: Encrypted attacks using cipher suites that represent ~97% of real-world HTTPS traffic.

Security Service Edge is inherently complex — a multi-layered technology stacked atop ever-changing cloud environments. Customers typically have minimal visibility into how these systems operate and testing them independently is challenging. This double-layered opacity makes third-party validation essential to diagnose performance issues, fine-tune policy enforcement, and ensure real security outcomes. CyberRatings strongly urges organizations to adopt periodic or ongoing third-party testing to ensure consistent protection and compliance.

NSS Labs is the Official Testing Partner of CyberRatings. Keysight’s CyPerf tool was used for performance and TLS/SSL functionality, and TeraPackets Threat Replayer tool was used for exploit replay validation.

CyberRatings.org Names NSS Labs as Official Testing Partner

Posted on by CyberRatings.org

AUSTIN, TX – July 9, 2025 — cyberratings.org/ (CyberRatings), the non-profit dedicated to insight into the capabilities of cybersecurity products and services through independent testing, today announced that NSS Labs has been named its Official Testing Partner. This collaboration marks a significant step forward in delivering unbiased, high-impact cybersecurity assessments to enterprises, governments, and the public sector.

By partnering with NSS Labs, a newly re-launched and revitalized leader in independent security testing, CyberRatings reinforces its mission to bring greater clarity, accountability, and objectivity to an increasingly complex cybersecurity landscape.

“Our mission is to empower stakeholders with the data they need to make confident security decisions,” said Ian Foo, CTO and EVP of Product for NSS Labs. “By serving as the Official Testing Partner to CyberRatings, NSS Labs will help extend that mission, combining our lab’s deep technical expertise with CyberRatings’ commitment to transparency.”

Partnership Highlights

  • Methodology-Driven Testing: CyberRatings will continue to develop robust, real-world test methodologies and will contract with NSS Labs to execute independent testing on behalf of CyberRatings.
  • Publication of Results: The test results will be published by CyberRatings, offering end users unparalleled visibility into how products perform against sophisticated, evolving threats.
  • Thought Leadership and Education: In addition to product testing, NSS Labs will contribute to educational initiatives from CyberRatings—authoring comparative studies, providing expert commentary, and participating in forums that advance cybersecurity best practices.

A Shared Vision for Cybersecurity Assurance

NSS Labs, newly relaunched as “NSS Labs 2.0,” brings a rich heritage as the gold standard for cybersecurity product testing, now enhanced by interactive tools, expanded testing of advanced technologies (including AI/ML-powered defenses and post-quantum cryptography), and a leadership team deeply experienced in cybersecurity strategy and technical validation.

CyberRatings will continue to set the benchmark for transparency in cybersecurity by openly publishing the results of these rigorous tests. This shared commitment to integrity and openness ensures that organizations worldwide can make more informed decisions about cybersecurity investments—backed by credible, real-world data.