“Yet another hacker has hit an MSP.”
That headline comes from ChannelE2E last week, referencing the most recent alleged hacker attack on an MSP. Similar headlines ran in December 2018, April 2019, June 2019, and August 2019 (twice). The Department of Homeland Security first warned of “advanced persistent threat actors actively exploiting trust relationships in IT service provider networks around the world” in October 2018.
Full details of the most recent apparent attack have yet to emerge (and maybe never will) but it allegedly went something like this: Hackers accessed and disabled the managed service provider’s backup and disaster recovery appliances. Then they infected the MSP’s system – and through it, end-customer networks – with theGlobeImposter 2.0 virus.
Why do MSPs keep getting hacked?
The biggest reason why MSPs keep getting hacked is the common misperception of security through obscurity. These are MSPs who think that because they can’t see how they could be attacked, they’re safe. They think hackers would need to know far too much about their internal setup to effectively breach it. Yet history proves quite the opposite.
Because it’s so easy to get into the cloud, security considerations are often an after-thought (or a never-thought). Most MSPs start out small and scrappy, often without the necessary security expertise. Some grow up to be big companies, with big customers – and big targets. And that’s when years of bad security practices catch up to them. Planning to ‘build in security if we get big’ is not a solution. Hackers are patient. They could be monitoring an MSP for years, gathering data, probing, putting things in place to exploit at just the right time.
The second source of vulnerability for many MSPs is complacency around the importance of processes to mitigate the risk of bad things happening when humans do stupid things (as humans do). Rarely is the backup tool or the RMM tool or the cloud platform the problem. It’s almost always the MSP’s processes (or lack thereof).
For example, in the most recent attack, hackers allegedly accessed and disabled the MSP’s Datto BCDR appliances. But the problem wasn’t Datto. The company had previously reminded MSPs to turn on two-factor authentication. At least one, it appears, ignored that good advice. (Probably, the MSP’s automation tool didn’t support 2FA so they didn’t turn it on.)
The third reason why MSPs have proven so vulnerable to attack is the highly competitive market they exist in. Profit margins are thin, so MSPs look for advantage any place they can get it. Most turn to tools like Datto, Ninja, and ConnectWise, among others – but again, it’s not the tools that are the problem. The problem is MSPs selecting a tool before thinking about security. A tool should be implemented with security awareness as the first step in a rigorous due diligence process. And no single tool should have access to the MSP’s entire system. Division of roles and responsibilities is a must.
An attack on your MSP will cost you
The direct costs associated with a cyber attack suck, to be sure. Even if you have insurance, that’s an added expense that will only grow the more you’re attacked. But even worse – and a cost against which there is no guard – is the reputational damage. So think of it this way: When you hire an MSP, you are literally putting your brand in their hands.
What you can do about it: 5 security best practices to mitigate the risk that your MSP’s vulnerability becomes your breach
With attacks on MSPs making headlines with increasing regularity, MSPs are investing in defense strategies. Still, not every MSP’s security posture will be equal. Fortunately, there are questions you can ask, or best practices you can look for, to determine whether your current MSP or the MSP you’re considering hiring is well protected. Here are five.
1. MFA! For Pete’s sake, your MSP (and you) should have multi-factor authentication turned on. (Yes, automation can work even with MFA requirements enabled). After all, 80% of security breaches involve compromised credentials. A single weak password is enough to compromise an entire security practice. While you’re at it, ensure your MSP (and you) follow the principle of least privilege. Not everyone needs admin credentials
2. Standardized processes! Enforce standards, policies, and processes to guard against stupid. Standardization is not just about documenting policies and processes but about enforcing the one approved way of doing things. MSPs that enforce standardized processes don’t hack stuff to make it work (like many of those scrappy young MSPs do).
Consider the example of the security pen tester on a mission to breach a small MSP. For months he looked for vulnerabilities to no avail. Then he came across an employee of the MSP in a forum for stamp collectors. The pen tester sent an email to the employee’s work address offering to sell him the rare collection he had been asking about in the forum. The employee clicked on the link, and the pen tester was in.
The pen tester was successful because the MSP hadn’t patched its internal laptops to prevent such a browser exploit. Now, calling the stamp enthusiast stupid is too harsh (though every company should educate employees to identify phishing). It’s not too harsh to call it stupid that the company didn’t have standardized processes that would have guaranteed internal laptops got patched.
3. Division of roles and responsibilities. One user permission shouldn’t have access to the entire system. As just one example, for our DevOps On AWS clients we’ll have access into the client’s AWS account but not their code. It’s a best practice from their perspective, and from ours. (A principle every MSP should love is called non-repudiation; we couldn’t be responsible for a code vulnerability because we don’t have access.)
4. Endpoint security! The beauty – and the curse – of SaaS is that it has enabled employees to work from just about anywhere. Allowing employees to work from the local coffee shop is fine. Allowing them to work from the local coffee shop’s Wi-Fi is not. People with admin access to the MSP’s systems should never be allowed to access the network from anywhere except through a VPN.
5. Regular testing. Hackers will always be one step ahead of those who seek to thwart them. So regular testing to find vulnerabilities to new exploits is essential. Ask the MSP: Are applications, environments, and systems regularly audited for security vulnerabilities? Are regularly security tests performed to discover weaknesses? Are penetration tests performed regularly?
Such tests would have identified the open WebUI that appears to have been a vulnerability in the most recent alleged attack. (It’s truly frightening to think that an MSP ‘mature’ enough to have actual customers had enabled local WebUI access to its BCDR appliance.)
Basically a running security audit, tests should be created and regularly run to ensure new vulnerabilities or misconfigurations are caught. In the most recent alleged attack, regular data restore testing likely would have revealed indicators of the hack (for example, encryption is CPU intensive) and enabled the MSP to mitigate the damage.
The Department of Homeland Security is absolutely right about what the benefits of an IT service provider can be: “IT service providers enable customers to scale and support network environments at a lower cost than financing these resources internally.” But for those benefits to pay off, MSPs have to be better than their customers.
They’re out there. Proceed with due diligence – and these five best security practices – to find one.
In our toolbox
- Sophos Intercept X is the world’s most comprehensive next-generation endpoint protection solution, built to stop the widest range of attacks
- AlienVault® Unified Security Management® combines key security capabilities with expert threat intelligence that is updated every 30 minutes with data from the Open Threat Exchange®
- FireEye offers a single platform that blends innovative security technologies, nation-state grade threat intelligence, and world-renowned Mandiant consulting