Thunderbolt security has been in the news recently: researchers presented a set of new vulnerabilities involving Thunderbolt which they named Thunderclap¹. The authors built a "fake" network card² and performed various DMA attacks and were able to temper with memory regions that their network card should have no access to whatsoever. In some way this is not all that surprising because the foundation of Thunderbolt are PCIe tunnels to external hardware and one of the reasons that PCIe is fast is because it can do direct memory access (DMA).

Use boltctl domains to inspect the current security level.

The current primary defense against DMA attacks for Thunderbolt 3 are the security levels: if enabled (the default on most systems) it gives the software the ability to decide on a per device level to allow or deny PCIe tunnels (and with that potentially access to all the memory via DMA)³. While not protecting from DMA attacks per se it protects from some — maybe the most — prominent threat scenarios⁴: 1) somebody plugging that evil device into your computer while you are away or 2) you have to plug in a device into your computer that you don't trust, i.e. a projector at a conference. On GNU/Linux boltd will authorize a plugged-in device only if an admin user is logged in and the screen is unlocked. For untrusted environments the authorization by boltd can be disabled entirely, e.g. when you go to a conference, via the GNOME settings panel. The toggle is called "Direct Access" (see screenshot below).

The panel has a global switch to toggle all authorization performed by bolt.

This is not the full story though, because there is a second way to prevent DMA attacks utilizing the input–output memory management unit (IOMMU). The general idea is to assign a specific memory region to a device which is the only area the device can directly access. Mika Westerberg from Intel has worked on kernel patches (will be in 5.0) to use firmware supported IOMMU virtualization (Intel calls this VT-d) which should make DMA attacks harder. Having said that, Mika was pointing out (thanks!) that there are still two possible issues mentioned in the paper that are not yet addressed (I paraphrase Mika): 1) IOMMU window size granularity is such that it may open a "too big" IOMMU window and 2) IOMMU mappings are not immediately torn down, leaving memory exposed for some time. Intel (Lu Baolu) is working on patches to mitigate those issues. On the bolt side of things, I have recently merged MR!137 which adds the necessary IOMMU plumbing. The big caveat for this is that it needs hardware/firmware support and for most (all?) currently shipping systems we are out of luck.

Security levels and IOMMU based DMA protection are orthogonal and the way support is implemented in Linux 5.0 is that userspace (bolt) is still required to authorize devices (if the security level demands it, i.e. user or secure). This means that you still have the ability to globally disable authorization of any device, e.g. for when you go to a conference.

Thunderbolt 3 Security Levels in the BIOS: Enable at least "User Authorization"

To sum it all up: For now, make sure you have Thunderbolt 3 security enabled in the BIOS and whenever you are in an untrusted environment (e.g. conference) disable device authorization completely (via the Settings panel). In the future, with Linux 5.0 and new hardware Linux will also get IOMMU based DMA protection that should greatly reduce the risk of DMA attacks and work is underway to plug the remaining known issues.

Footnotes:

1. Thunderclap: Exploring Vulnerabilities in Operating System IOMMU Protection via DMA from Untrustworthy Peripherals A. Theodore Markettos, Colin Rothwell, Brett F. Gutstein, Allison Pearce, Peter G. Neumann, Simon W. Moore, Robert N. M. Watson. Proceedings of the Network and Distributed Systems Security Symposium (NDSS), 24-27 February 2019, San Diego, USA.
2. How they did it is pretty cool: "[W]e extracted a software model of an Intel E1000 from the QEMU full-system emulator and ran it on an FPGA". More details in the paper on page 5. The whole platform is also available at github.com/thunderclap-io
3. The paper states that situation for Linux is: "patches for approval of hotplug devices have been produced by Intel and distributions are beginning to implement user interfaces." So this is not super up to date (long paper review process I guess). The current situation is: kernel level support landed in 4.13; bolt was included in Fedora 27, RHEL 7.6 and is now included in many distributions. IOMMU support will land in 5.0 (but see the text for remaining issues).
4. Assuming that you trust manufacturers of hardware (attacks can also happen at the "supply chain").

With the release of bolt 0.6¹ "Make the firmware do it!" boltd gained support for preboot access control list (bootacl). The quick summary of that feature is best given by explaining what problem it solves: You are using a Thunderbolt 3 dock that has the main keyboard and mouse connected to it and you want to use the mouse and keyboard during pre- or early-boot, e.g. to enter a password to unlock the hard-drive. If the "pre-boot ACL" feature is enabled in the BIOS, boltd will now do the right thing and Thunderbolt 3 peripherals will be functioning from early on. It comes with one big security caveat though: no device verification is possible, which means the effective security level is user mode during pre-boot (see below for details). The requirements checklist is: bolt version >= 0.6, a kernel >= 4.17 as well as recent enough thunderbolt firmware. The feature usually needs to be explicitly enabled in the BIOS as well (it will be called something like "pre-boot ACL").

The pre-boot BIOS settings

To explain in more detail how all of this works let me first quickly recap Thunderbolt 3 security levels: Thunderbolt tunnels PCIe to connect peripherals and thus allows for direct memory access (DMA); this makes in fast but can be used as an attack vector. In order to prevent DMA attacks Thunderbolt 3 introduced different security levels: none (SL0), user (SL1)², secure (SL2) and dponly (SL3).³ In both user and secure mode the device needs to authorized before the PCIe tunnels to the peripherals are established. This is done by boltd, a system daemon that manages a list of trusted devices. After Thunderbolt devices are enrolled, i.e. stored in boltd trust device database, the daemon will automatically authorize those devices as soon as they appear. Devices are identified via a unique identifier (UUID). But since the device itself is reporting the UUID to the system, a malicious devices could pretend to have the UUID of a already trusted device. To prevent this secure (SL2) mode uses a per-device key that is generated by the host and stored in the device so it can be used to verify the device identity.

As mentioned above, boltd is the user space component that will authorize the devices. As a consequence if boltd is not running, nobody is authorizing the device and the device will not be able to function. Two of important instances where boltd is NOT running but you might want use the mouse and keyboard in BIOS setup and probably way more important during early boot, to enter the LUKS password.

Recent BIOS versions enabled support for storing a limited list of UUIDs directly in the thunderbolt controller. This is called the pre-boot access control list (or preboot ACL), in bolt simply called "bootacl". The devices corresponding to the devices in the bootacl will be authorized during pre-boot (and only then) by the firmware. One big caveat about this feature should be become obvious now: No device verification can happen because only the UUIDs are stored but not the key, so if you are using SECURE mode but enable preboot ACL in the BIOS you effectively will get USER mode during boot.

The kernel exposes the bootacl via a per-domain sysfs attribute boot_acl. Every time a device is enrolled, boltd will automatically add it to the bootacl as well. Conversely if the device is forgotten and it is in the bootacl, boltd will automatically remove it from the bootacl. There are is small complication to these seemingly straight forward operations: in BIOS assist mode, the thunderbolt controller is powered down by the firmware if no device is connected to it. Therefore when devices are forgotten boltd might not be able to directly write to the boot_acl sysfs attribute. In a dual boot scenario this is complicated by the fact that another operating system might also modify the bootacl and thus we might be out of sync. As the solution to this boltd will write individual changes to a journal file if the thunderbolt controller is powered down and re-apply these changes (as good as possible) the next time the controller is powered up.

boltctl showing the list of domains including the bootcl

An easy way to check the current bootacl entries is via boltctl domains. Currently this is read only but issue #139 is open for write support. This would be a perfect issue to start if anyone would like to hack a bit bolt.

Footnotes:

1. Meanwhile I have released bolt 0.7 'The Known Unknowns', which contains fixes for a few bugs that slipped into 0.6.
2. This security mode is the default on most systems.
3. A option called usbonly (SL4) was added added to the new TitanRidge Thunderbolt 3 controller (and support for it was added to kernel 4.17).