Cyentia Institute collaborated with Cobalt on the latest State of Pentesting Report, which was released last week. The thing I found most interesting in this edition is the organization-level performance benchmarks we derived for the first time. We measured 4 key metrics and compared them across thousands of organizations:
- Ratio of pentest findings rated high-risk
- Proportion of high-risk findings resolved
- Mean-time-to-resolution (MTTR)
- Half-life of high-risk findings using survival analysis
This essentially answers “How exposed are you, how complete is your remediation, and how quickly do you get it done?
We used the half-life of high-risk findings as the prime factor differentiating leaders, laggards, and everyone in between. Why half-life? It’s the best all-around measure of how the organization is focusing on high-risk findings that matter most and resolving them in a fast AND comprehensive manner.
The chart here shows the half-life of high-risk findings for all organizations. Leaders have the shortest half-life (the top 10%). They resolve half of their high-risk pentest findings in 10 days or less. On the other end of the spectrum, it takes the laggards 25 times longer to reach that goal.
The obvious question here is what distinguishes leaders from laggards? The first hypothesis I had during the analysis was whether the “leaders” simply didn’t have many high-risk findings to contend with. That might be because they eliminated them before the pentest, because the assets in scope weren’t highly critical/sensitive, or for any number of other reasons. Bottom line: maybe leaders just got a “head start” and reached the halfway mark earlier.
The analysis, depicted in this chart, proved that to not be the case.
Like last time, each dot in the chart represents an organization. The axis measures the proportion of each org’s pentest findings rated as high risk (more on this rating in a future post). Leaders are indicated in blue; laggards in pink.
The fascinating thing is that leaders and laggards are dispersed throughout the range. This clearly disproves the hypothesis that the primary success factor of leading organizations is a low rate of high-risk findings.
This is not just a “noteworthy analytical outcome” that excites data nerds like me. The takeaway is really important for teams struggling to keep up with the many security issues vying for remediation priority.
It shows that high remediation performance is achievable even in a high-risk environment. Case in point – that rightmost organization facing a 60%+ rate of high-risk findings that nevertheless achieved leader status by quickly resolving those issues. Conversely, the chart warns that orgs with fewer risky exposures should not rest on their laurels.
The lesson: High risk-reducing performance is achievable even in a high-risk environment. But it’ll take planning, prioritization, and execution.