For both network and block storage, AWS is doing significant work to develop and maintain drivers in a variety of guest OSs. Some of this work improves performance for guest OSs running in any modern hardware virtualized environment, but not everything is directly portable. This discussion about adding ENA support for Netmap is one example. OTOH, Amazon seems to be sponsoring driver development (see FreeBSD) when they’re not doing it themselves (see Linux). Who built the Windows driver is unclear.
A contact at Intel spoke rather openly that AWS was consuming about 50% of all Intel CPUs. Ignoring what this means for Intel’s business prospects, consider that it means that AWS is effectively the dominant server manufacturerdesigner. And, now that they’re building their own components, they’re the biggest developer of drivers for server hardware. I can’t cite the claim, but it seems that network and storage vendors are increasingly reverse engineering AWS’ drivers to identify product requirements. In time, I expect server hardware to converge on the driver specs that AWS defines, even for hardware not running in AWS.
For other hypervisor vendors/developers, as well as other clouds, this probably means that the hypervisors need to conform to AWS’ expectations for KVM in order to get AWS-style performance in virtualized environments. It’s hard to estimate the cost to develop a competing approach that would yield similar performance. Just as vendors will copy AWS’ hardware, the open source KVM (separate from KVM that AWS uses internally) and bHyve teams will need to reverse engineer the interfaces they present from the guest OS drivers (I expect much collaboration between hypervisor communities and vendors).
Distilled:
- AWS is the de facto standards body for NICs and storage (and soon, likely more), but the only spec for those standards is the guest OS drivers they publish for supported platforms.
- Any room that might have existed previously for others to develop competing approaches to the core abstractions used by AWS has now passed. Clouds are built on VMs (in very rare cases, API-provisioned bare metal that behaves identically to the VMS, except it can run hypervisors without doing nested virtualization), and the entire industry is organizing around AWS’ definition of those abstractions.