Austin, TX – May 15, 2025 – CyberRatings.org (CyberRatings), the non-profit organization dedicated to providing insight into the capabilities of cybersecurity products and services through independent testing, has released additional results from its Security Service Edge (SSE) testing. These latest tests focused on two leading products: Cisco Umbrella and Palo Alto Networks Prisma Access.
Palo Alto Networks Prisma achieved a Security Effectiveness score of 98.89%, successfully blocking 100% of evasions. In contrast, Cisco Umbrella scored 12.44%, primarily due to its failure to detect evasive threats. Full test reports detail product performance across multiple threat categories, with scoring weighted by attack severity.
The evaluation covered:
- TLS/SSL: Top 5 Ciphers used (accounts for ~97% of HTTPS traffic).
- Malware: 6,184 attack samples sourced from current malware campaigns.
- Exploits: 205 attack samples from widely exploited vulnerabilities in enterprise environments.
- Evasions: 1,154 attacks spanning 37 evasion techniques.
- False Positives: 1,514 samples from various business-critical files and applications, ensuring security measures did not disrupt legitimate traffic.
Evasion techniques are used by attackers to disguise or obfuscate attacks so that they bypass detection. SSE products must not be tricked by evasions—failure exposes organizations to entire classes of (undetected) threats.
“Missing just one type of evasion allows attackers to use entire categories of malware or exploits undetected,” said Vikram Phatak, CEO of CyberRatings.org.
Security Service Edge is a complex multi-layered security technology built on top of complex, ever-changing cloud technologies. Customers have minimal visibility into their operation and architecture, and testing is challenging. This double-layered opacity limits an organization’s ability to diagnose performance issues, fine-tune policy enforcement, or validate security outcomes.
“These are closed systems—what I think of as a black box in a black box—that force executives to make risk decisions based on trust rather than evidence,” Phatak added. “That’s why it is critical that independent testing provides evidence-based data on which executives can make decisions.”
CyberRatings is on track to test several other SSE vendors for Threat Protection along with a Comparative Report to be published this summer.
In addition to in-house testing technologies, CyberRatings used Keysight’s CyPerf tool to test performance and TLS/SSL functionality as well as TeraPackets Threat Replayer tool for exploit packet capture replay.