Metrics here, metrics there, metrics everywhere—it can be daunting. How do you choose a cybersecurity services provider: by performance metrics alone, or are other factors important? According to Section1, a provider of IT management platforms, cybersecurity leaders must consider not only
You need to battle the confusion that can hamstring your efforts at identifying a trusted MSS provider and achieving your optimal security posture. We can make a comparison to sporting statistics: their inherent complexity and the difficulty in making them fit a single performance ideal.
Sabermetrics
If you’re a baseball fan—or you’ve seen ‘Moneyball’—you’ve run smack into sabermetrics: the empirical analysis of the game. You can’t listen to a game without getting an earful. But if you judge your favorite player by swinging power, throwing arm, or even outward appearance, you’re overlooking the universe of performance statistics that stack him up against other players.
And if you’re not a fan of the game, can you still appreciate the requirements behind a masterful pitch? Plus, it’s probably advantageous to keep your organization out of left field, technically speaking.
What is WAR?
Every player is analyzed according to Wins Above Replacement: that player’s contribution to wins that wouldn’t have occurred if they had been replaced by someone else. And every year, MLB votes for the best pitcher in the game – the Cy Young Award winner – based on WAR rankings. But there are three versions of WAR for pitchers and none are standardized, leading to confusion (even among sportscasters) as to who has the best pitching performance in the game.
Here’s a look at the three statistical versions, in general terms:
| RWAR | FWAR | WARP |
| Simplest | More accurate | Most complex |
|
Known as the ‘core’ stat:
|
Considers defense-independent pitching (only the actions the pitcher can control):
|
Combines
|
And here is a Fox Sports sampling of late-season results for four pitchers across the three versions:
Who’s the best?
If 1st is best, then Pitcher B is the winner, but the ranking doesn’t show that he has actually pitched 40 fewer innings than Pitcher C, and has four fewer wins for the season than Pitcher A. Not only that, but Pitcher D ranks 1st by the complex WARP, yet he’s 6th by RWAR. So, what’s your metric? According to one Fox sportscaster, “You can wind up all over the map across these three versions, depending on how each applies – or doesn’t apply that well – to an individual pitcher. Different people have their favorite stat to evaluate.”
Vendor Variables
Voting for your winning security service provider can be equally challenging. Here, the variables can be quite complex – and far more consequential to your organization:
| VENDOR | REQUIREMENTS | COST |
|
|
|
It’s evident that this type of analysis is tough to graph.
Choosing the winner
When it comes to your cybersecurity needs, you don’t want to be “all over the map”. You want clear information from a vendor regarding the solutions they offer to determine:
- Which solution best fits your organization’s current needs
- Whether the vendor can help you plan for future needs
- Whether the vendor can accurately determine your current security posture
RedLegg Services
RedLegg offers all of the above, from consulting and advising to proactive monitoring and battleground testing solutions:
- Managed Security Services
- Advisory Services
- Penetration testing, validation, and application assessments
Need clarification with a custom security roadmap, or assistance with incident detection and response? We can do that. It’s all in a day’s work, even after the Boys of Summer have gone.
