Bare metal clouds are hard

The problem, explains Eclypsium, is that a miscreant could rent a bare-metal server instance from a provider, then exploit a firmware-level vulnerability, such as one in UEFI or BMC code, to gain persistence on the machine, and the ability to covertly monitor every subsequent use of that server. In other words, injecting spyware into the server’s motherboard software, which runs below and out of sight of the host operating system and antivirus, so that future renters of the box will be secretly snooped on.

Indeed, the researchers found they could acquire, in the Softlayer cloud, a bare-metal server, modify the underlying BMC firmware, release the box for someone else to use, and then, by tracking the hardware serial number, wait to re-provision server to see if their firmware change was still intact. And it was. BMC is the Baseband Management Controller, the remote-controllable janitor of a server that has full access to the system.

That’s The Register’s Shaun Nichols, though Wired and Tom’s Hardware1 also covered the story. The hullabaloo is because Eclypsium solidly demonstrated a security risk “everybody knew about” and the company had reported back in 2018.

This may be the the most recent, clearest public demonstration of firmware risks, but the attack vector was a component of the 1998 CIH/Chernobyl virus. Persistent firmware attacks were the subject of this 2007 Blackhat talk. Phlashing, a related vulnerability made news in 2008. Firmware hacking made news again in 2015, then 2017, and Blackhat 2018 featured 14 briefings and training sessions related to firmware vulnerabilities.

I’m an interested observer of this, but I’m sure I’ve missed some important milestones in the history of firmware hacking. Still, it’s surprising that this is shocking news in 2019.

But here’s the point: it’s hard to build a bare metal cloud. Customers expect to have full control over the server, but that also gives customers access to the firmware in in all the devices in that server. AWS’ work on Nitro to secure their hardware is noteworthy here, because they clearly anticipated this problem and invested heavily to mitigate it in their cloud (here’s what I had to say about AWS’ bare metal offering when they introduced it).

  1. The Tom’s Hardware story incorrectly suggests this revelation corroborates a Bloomberg Business Week story that claimed Chinese hackers had embedded nearly undetectable custom hardware on server motherboards in the form of a “tiny microchip, not much bigger than a grain of rice, that wasn’t part of the boards’ original design.” The firmware vulnerabilities reported by Eclypsium are unrelated to the claims in the Bloomberg Business Week story. No such hardware was suggested by Eclypsium’s reports. ↩︎