MSSPs offer a wide range of services and are often the most cost-effective method of delivering quality operational security, but are they really delivering?
Testing your MSSP is one of the best things a business can do to improve its cyber security. A managed security service provider’s (MSSP) lapse in security will go unnoticed until either someone goes looking for it or a detectable breach occurs. The comprehensive approach to assessing the MSSP is during the annual full security assessment.
Cyber security is a reality of business, like electric bills or information technology. Done right, cyber security can lead to an increase is revenue because whichever company better protects its data assets will have a competitive advantage. MSSPs are becoming an integral part of many strategies to improve cyber security. Knowing whether an MSSP is providing the service it claims to provide is good business and good security.
The Benefit of Testing Your MSSP
Generally, businesses will notice if their information systems (IT) are having a hard time keeping up, or if the promised IT help desk and engineering support is below expected standards. Security issues only get discovered during assessments and hacks.
If a business wants to know if the contracted MSSP is delivering on the promised security, the most comprehensive and informative approach is a full security assessment, testing technical (ex. penetration testing) and administrative (ex. cyber security program assessment) controls of the security stack and cyber security program.
In general, annual assessments of your environment, with activities and reporting highlighting the MSSP’s capabilities is enough to know if obligations are meet. If a full security assessment is not in the budget (they cost about $30k minimum) there are still ways to test the MSSP that are informative and valuable.
How to Test Your MSSP
There are 4 steps to testing your MSSP
1. Understand the MSSP’s Statement of Work (SOW)
The SOW dictates everyone’s responsibilities and describes limitations and constraints on the capabilities and offerings of the MSSP. You don’t want to find yourself calling the MSSP out for not doing something they aren’t obligated, or able to do.
Take care to understand what they are providing to ensure the assessment covers as much of the SOW as possible.
Common MSSP offerings and capabilities:
- Asset Management
- Patch Management
- Vulnerability Scans
- Phishing Countermeasures
- Log Collection, Management, and Analysis
- Endpoint Detection and Response (EDR)
- 24/7 monitoring and incident handling
Not all MSSPs offer that entire list and some MSSP offer more. The important thing to remember is the security assessment (whatever level) activities performed are sufficient to make judgment calls on the efficiency, reliability, and completeness of the MSSPs commitments.
Side Note: If the business is looking at contracting an MSSP (or a new one), it’s not a bad idea to inform them that a few arbitrary times each year a 3rd-Party is coming in to do a security assessment – and they won’t be informed until after its completion.
2. Hire a 3rd-Pary Vendor
A 3rd-party vendor is the most impartial party to assess the MSSP and determine if they are providing all the services contractually obligated to provide. The 3rd-party vendor specializing in security assessments will have the skills to test everything right and conflict-of-interest issues generally won’t arise.
The cost will vary (see our discussion on budgeting for a penetration test), but a full security assessment including all of the above components will cost a minimum of about $30,000 and can go well into the six-figures.
The price will, and should, depend on a variety of factors like the business’s revenue, the complexity of the infrastructure, and the IT budget (including MSSP monthly fees).
3. Get a Full Security Assessment
A full security assessment is the most compressive approach to achieve such an objective. A full security assessment usually has the following components in some form or another.
- Vulnerability Scan
- Penetration Test
- Phishing Assessment
- Cyber Security Program Assessment (sometimes called vCISO Assessment)
- Cyber Risk Assessment
The full security assessment will test every aspect of the security of your organization and as such will test each area the MSSP’s services will overlap.
The vulnerability scan, penetration test, and phishing assessment will test the technical controls. The cyber security program assessment will test the administrative controls. The risk assessment will identify risks in your environment and then provide insight into where and how much to spend on controls.
The technical controls assessments will require zero input or awareness on the part of the MSSP.
The cyber security program assessment will probably require MSSP’s participation if you have never asked them for any of the policy or procedural documentation governing their capabilities. For example, if they provide EDR they should have some form of incident management plan dictating their response and escalation procedures.
If they don’t have any documentation, or the documentation stack is thin, that is a red flag.
The risk assessment may also require participation on the part of the MSSP, but less so than the cyber security assessment. For example, the risk assessors will (or should) identify the high value, revenue generating assets, but they will also want to know what kind of EDR tools are installed and where.
If the budget doesn’t have the space for a full security assessment, consider a penetration test. A typical penetration test covering the internal and external infrastructure also includes vulnerability scans and may or may not include phishing. At the very least a considerable amount of the services and controls are covered with a penetration test.
4. Review the Assessment Results with the MSSP
When the assessment is complete, take the time to review the results with the MSSP. A good MSSP won’t mind that you have run an assessment and will appreciate straight forward candor on where they can improve.
Give them the opportunity to improve. The result will either be improved security or a new MSSP, which is probably improved security.
If the MSSP is upset an assessment was done without their knowledge, that is a red flag.
What to do if You Have to Test the MSSP Yourself
If you don’t have the money or the skill set to run a vulnerability scan or a low sophistication penetration test, and you are committed to assessing your MSSP, here is a “workable” option: Atomic Red Team Tests.
However, before you try and do this yourself consider the following:
- You may not know which ones to run
- It will take quite some time
- You could affect the production environment, costing you more money – assessment firms have insurance for this.
- You may not be sure how to read the results of the tests and then make judgments on the effectiveness of the MSSP
Recommendation: Quality assessments are hard and take a couple years of dedicated experience to learn. Do not do this yourself.
What do if the MSSP Fails the Test
Remember failure is related to the SOW. Determining if the MSSP fails to deliver its security will require combination of grit and understanding. The grit to stand firm on what you know you are supposed to get and the understanding to know that security is hard.
Keep in mind, 53% of breaches go undetected, and, though the numbers can vary, the ones that are detected take 100 – 200 days to get detected.
If after the final analysis they are failing to uphold their obligations, you have a few options:
- If the failure is “mild”, ask them to resolve the findings related to their obligations under the SOW.
- If the failure is “severe”, ask them to resolve and then pay for a portion of a re-assessment.
- If there are dep levels of negligence, find a new MSSP, and maybe even try and get some of your money back (this is not legal advice).
Conclusion
Testing the MSSP is almost just as important as testing the business’s entire security program and technology stack. It lets the business know if the MSSP is reliable. Don’t hesitate to do it.