REDLEGG BLOG
kpi-war-pic

Choosing a cybersecurity provider: the WAR analogy

Oct 24, 2018 3:39:02 PM  |  by RedLegg Expert Security Engineer

Metrics here, metrics there, metrics everywhere—it can be daunting. How do you choose a cybersecurity services provider: by performance metrics alone, or are other factors important? According to Section1, a provider of IT management platforms, cybersecurity leaders must consider not only performance, but a host of issues surrounding both their organizational needs and the wide world of managed security services (MSS) vendors.

You need to battle the confusion that can hamstring your efforts at identifying a trusted MSS provider and achieving your optimal security posture. We can make a comparison to sporting statistics: their inherent complexity and the difficulty in making them fit a single performance ideal. 

Sabermetrics

If you’re a baseball fan—or you’ve seen ‘Moneyball’—you’ve run smack into sabermetrics: the empirical analysis of the game. You can’t listen to a game without getting an earful. But if you judge your favorite player by swinging power, throwing arm, or even outward appearance, you’re overlooking the universe of performance statistics that stack him up against other players.

And if you’re not a fan of the game, can you still appreciate the requirements behind a masterful pitch? Plus, it’s probably advantageous to keep your organization out of left field, technically speaking. 

What is WAR?

Every player is analyzed according to Wins Above Replacement: that player’s contribution to wins that wouldn’t have occurred if they had been replaced by someone else. And every year, MLB votes for the best pitcher in the game – the Cy Young Award winner – based on WAR rankings. But there are three versions of WAR for pitchers and none are standardized, leading to confusion (even among sportscasters) as to who has the best pitching performance in the game.

Here’s a look at the three statistical versions, in general terms:

RWAR FWAR WARP
Simplest More accurate  Most complex

Known as the ‘core’ stat:

  • Runs allowed
    (earned + unearned)
  • Innings pitched

Considers defense-independent pitching (only the actions the pitcher can control):

  • Home runs allowed
  • Strikeouts
  • Hit batters
  • Walks
  • Fly ball percentage
  • Ground ball percentage
  • Line drive percentage 

Combines RWAR + FWAR, plus:

  • Ballparks
  • Quality of team offense & defense
  • Quality of opposing team & batting lineup 

 

And here is a Fox Sports sampling of late-season results for four pitchers across the three versions:

war-graph

Who’s the best?

If 1st is best, then Pitcher B is the winner, but the ranking doesn’t show that he has actually pitched 40 fewer innings than Pitcher C, and has four fewer wins for the season than Pitcher A. Not only that, but Pitcher D ranks 1st by the complex WARP, yet he’s 6th by RWAR. So, what’s your metric? According to one Fox sportscaster, “You can wind up all over the map across these three versions, depending on how each applies – or doesn’t apply that well – to an individual pitcher. Different people have their favorite stat to evaluate.”

Vendor Variables

Voting for your winning security service provider can be equally challenging. Here, the variables can be quite complex – and far more consequential to your organization:

VENDOR REQUIREMENTS COST
  • Can the vendor deliver the service or solution I need?
  • Is that solution core to what they do internally, or will they subcontract it out?
  • How do their previous customers rate their experiences?
  • How stable is the company?
  • Do the vendor’s offerings match my specific objectives?
  • Are the vendor’s security partners trustworthy or well-known in the industry?
  • Is enough information available about the service or solution to develop the proper requirements?
  • Is the vendor willing to discuss solution details before proposing costs? What about after the proposal is received?
  • Is it clear in the proposal exactly what the solution is, and which/if security partner solutions are used?
  • How do I ensure that quotes from multiple vendors address the same requirements?
  • What requirements do other customers in my industry have that I should be considering?
  • What should I expect for costs: hourly/monthly/ yearly?
  • How does the cost compare to similar solutions provided to other customers?
  • How do costs compare across vendors for the same solution?
  • Can I understand the costs represented as a cost per user or cost per unit?

 

It’s evident that this type of analysis is tough to graph.

Choosing the winner

When it comes to your cybersecurity needs, you don’t want to be “all over the map”. You want clear information from a vendor regarding the solutions they offer to determine:

  • Which solution best fits your organization’s current needs
  • Whether the vendor can help you plan for future needs
  • Whether the vendor can accurately determine your current security posture 

RedLegg Services

RedLegg offers all of the above, from consulting and advising to proactive monitoring and battleground testing solutions:

  • Managed Security Services
  • Advisory Services
  • Penetration testing, validation, and application assessments 

Need clarification with a custom security roadmap, or assistance with incident detection and response? We can do that. It’s all in a day’s work, even after the Boys of Summer have gone.

Subscribe to Our Blog

Follow everything RedLegg as we provide comprehensive solutions for real-world data protection and security challenges.

Related Articles

6 Steps of Vulnerability Scanning Best Practices blog

6 Steps of Vulnerability Scanning Best Practices

Vulnerability scanning (vuln-scan) is the process of finding exploits, flaws, security holes, insecure access entry ...
How to Maximize your MSS Experience blog

How to Maximize your MSS Experience

Your Managed Security Services provider is a crucial player in your overall security posture, and our number one goal ...