Thunderbolt allows for peer-to-peer network connections by connecting two computers directly via a thunderbolt cable. Mika from Intel added support for this to the 4.15 kernel. Recently,Β Thomas Haller from NetworkManager and I worked together to figure out what needs to be done in userspace to make it work. As it turns out, it was not that hard and the pull-request was merged swiftly.


GNOME Settings' Network Panel

So now if you connect your GNU/Linux box to another machine, NetworkManager will configure the connection to be a Link-Local one.

pinging macOS

ping Linux from macOS

I have successfully tested Fedora to Fedora, Fedora to macOS and Fedora to Windows 10. The other operating systems also use a Link-Local address by default.


Copying from Linux to Windows

The technical details

When another host is connected via thunderbolt, we get a new device in sysfs that resembles a normal thunderbolt device but is of DEVTYPE thunderbolt_xdomain. Like a normal thunderbolt device, it has a unique_id and a device_name, but is missing the authorized and key attributes. A new udev rule in NetworkManager was added to load the thunderbolt-net kernel module. As a result, we get a new network device (thunderbolt0).

nmcli showing the thunderbolt networking device

Another udev rule will set a new udev property (NM_AUTO_DEFAULT_LINK_LOCAL_ONLY) to all network devices where the driver is thunderbolt-net to instruct NetworkManager to use the Link-Local configuration on these interfaces.

The dev snapshot release 1.11.3-dev has the thunderbolt networking support. Grab it, build it, and start playing around with it! Feel free to post your record transfer speeds in the comments ⚑ πŸ†.

Fedora 28 is around the corner and I wanted to highlight what we did to make the Thunderbolt 3 experience as smooth as possible. Although this post focuses on Fedora 28 for what is currently packaged and shipping, all changes are of course available upstream and should hit other distributions in the future.

The new thunderbolt control center panel

The new Control Center Thunderbolt Panel

bolt

First of all, the home of bolt has moved from Github to the new Gitlab instance at freedesktop.org1. File issues and spread your love in the form of stars β˜… there now ;)

I released version 0.3 of bolt about a week ago, with a lot of internal changes, small D-Bus API changes and support for future kernel changes2. For example it already supports the new USBONLY security level of the upcoming Titan Ridge controller, and the boot attribute; kernel level support for both landed in 4.17.

GNOME Shell

The GNOME shell bits to enable Thunderbolt 3 support were merged some time ago and made it into 3.28. This means that the GNOME shell will act as authorization agent and will automatically authorize new thunderbolt 3 devices β€” if the user has administrator rights, is currently logged in and the session is unlocked. It will also show a little indicator (called the cable_snake by some) in the status indicator section while devices are being authorized because the whole authorization dance can take quite a while.

Activity indicator, aka The Cable Snake

GNOME Control Center

We have a new panel to manage Thunderbolt 3 devices, woohoo πŸŽ‰! It landed in master not too long ago and will be in 3.30, but we backported it to Fedora 28. It lets you remove currently enrolled devices and shows if devices need authorization (they will be listed under "Pending Devices"). The latter might be happening, for example, because the devices were connected while the session was locked. The device detail dialog can then be used to authorize them.

Authorizing a device

Another important feature is the possibility to temporarily disable authorization of Thunderbolt devices. In the panel this is called "Direct Access", because disabling it will stop the authorization of Thunderbolt 3 peripherals, but the DisplayPort and "pure" USB functionality of the thunderbolt port will be preserved. In technical terms, it means that without authorization no PCIe tunnels are allowed to be created, but due to a feature called alternate modes, the thunderbolt port will still allow USB devices and DisplayPort streams to connect.

Switching "Direct Access" on and off

This feature might come in handy if you are traveling, say to defcon: while you don't want malicious devices (like specially prepared thunderbolt cables) to read your main memory, you still want attached projectors to work correctly.

Firmware updates

Support for updating the firmware of the Thunderbolt host controller and devices landed in fwupd quite some time ago and is of course available in F28 as well. On supported platforms, like Dell, it is therefore only one click away in GNOME Software. Mario also added support to force power the host controller during firmware updates, so updating the host controller is possible without the need to have something plugged into the Thunderbolt port.

Outlook

All in all I am quite happy with the current state of Thunderbolt on GNU/Linux and what we will be shipping in Fedora 28. For most people, connecting Thunderbolt 3 devices should just work. In the next few weeks I will focus on adding support for authorizing devices during pre- and early boot.

Footnotes

  1. The bolt Github repository will continue to exist and be a mirror of the main Gitlab repo. Release tarballs will also be available in both locations at least for the next few releases, but not forever, so please update your links.
  2. If you are upgrading from bolt 0.1: the database default location has changed to /var/lib/boltd so you might have to re-authorize your devices. Also, devices names are now cleaned-up and stored in the clean form in the DB. If you were running earlier versions of bolt, you might see "Vendor Vendor Device-Name" in the Control Center (see #89). The easiest way to fix this is to forget and re-authorize the device.

Thunderbolt icon by jimmac

 

Thunderbolt 3 security levels

Thunderbolt is an I/O technology that can be used to connect external peripherals to a computer β€” similar to USB and FireWire. It works by bridging PCIe between the controllers on each end of the connection, which in turn means that devices connected via Thunderbolt are ultimately connected via PCIe. Therefore thunderbolt can achieve very high connection speeds, fast enough to even drive external graphics cards. The downside is that it also makes certain attacks possible (e.g. Thunderstrike, DMA attack).
To mitigate these security problems, the latest version β€” known as Thunderbolt 3 β€” supports different security levels:

  • none: No security. The behavior is identical to previous Thunderbolt versions.
  • dponly: No PCIe tunnels are created at all, but DisplayPort tunnels are allowed and will work.
  • user: Connected devices must be authorized by the user. Only then will the PCIe tunnels be activated.
  • secure: Basically the same as user mode, but additionally a key will be written to the device the first time the device is connected. This key will then be used to verify the identity of the connected device.

The active security level can normally be selected prior boot via a BIOS option, but it is interesting to note that in the future the none option is likely to go away. This of course means connected thunderbolt devices wont work at all unless they are authorized by the user from with the running operating system.

Intel has added support for the different security levels to the kernel and starting with Linux 4.13. The interface to interact with the devices is via files in sysfs. Since July we have been working on the userspace bits to make Thunderbolt 3 support "just work" πŸ˜‰. The UX design design was drafted by Jimmac. The solution that we came up with to implement it consists of two parts: a generic system daemon and for GNOME a (new) component in gnome-shell. The latter will use the daemon to automatically authorize new devices. This will happen if and only if the currently active user is an administrator and the session is not locked.

bolt 0.1 Accidentally Working

Today I released the first version 0.1 (aka "Accidentally Working") of bolt, a system daemon that manages Thunderbolt 3 devices. It provides a D-Bus API to list devices, enroll them (authorize and store them in the local database) and forget them again (remove previously enrolled devices). It also emits signals if new devices are connected (or removed). During enrollment devices can be set to be automatically authorized as soon as they are connected. A command line tool, called boltctl, can be used to control the daemon and perform all the above mentioned tasks (see the man page of boltctl(1) for details).

I hope that other desktop environments will also find that daemon useful and use it. As this is not a stable release yet, we still have room for API changes, so feedback is welcome.

botlctl example output

New software needs testers, so everybody who has a computer with Thunderbolt 3 and feels courageous enough is welcome to give it a try. I created a copr with builds for Fedora 27 & rawhide and Jaroslav created a PKGBUILD file so Arch users can find it already in the AUR. As this is very fresh software it will contain bugs and those can be filed at the issue tracker of the github repo.

what's next: gnome-shell integration

I am locally running a Proof-of-Concept gnome-shell extension that implements the user session bits to complete the aforementioned : It uses bolt's D-Bus interface and listens for new Thunderbolt devices and then enrolls them, if the user is logged in. Since it can take a while until all the devices that are attached via thunderbolt are properly connected the daemon has a Probing property that is used to display a little icon as a way to inform the user that something is happening on the thunderbolt bus. All of this is already working quite well here on my test machine. In the next few days (weeks?) I will be working on integrating that code into gnome-shell. There are a few open UX questions that need to be addressed, but all in all things looking good.


thunderbolt activity indicator (aka cable snake)

Special thanks to Alberto Ruiz, Benjamin Berg, Hans de Goede, Harald Hoyer, Javier Martinez Canillas, Jaroslav Lichtblau, Jakub Steiner, Richard Hughes who all helped and supported this project during the last few months! ❀️


Ever since we (Red Hat's Desktop Hardware Enablement Team) received the 2017 models from Lenovo for testing (e.g. the T470s), we experienced an issue (rhbz#1480844) where sometimes the fan would run at 100% after resuming from suspend. A warm reboot alone would not make the fan go back to normal and a hard reboot was required. It seems the behavior is the result of firmware bug and kernel ACPI changes. Patches for 4.13 reduced the likelihood for the appearance of the noisy issue. Additionally, we have been working together with Lenovo to fix the firmware side and I am happy to report that for the T470s Lenovo recently released a new firmware that should completely fix the issue. Since Lenovo is not yet(!) part of Linux Vendor Firmware Service updating the BIOS is currently not super straight-forward. Thankfully, Jeff has provided detailed instructions how to do this from GNU/Linux only.

FOSDEM 2018 is approaching fast. There will be a Hardware Enablement Devroom,Β among many other very interesting ones. We invite everybody to come and participate:

Important dates

  • Conference date: 3 & 4 February 2018 in Brussels, Belgium
  • Devroom date: Sunday 4 February 2018
  • Submission deadline: Sunday 26 November 2017
  • Speaker notified: Sunday 10 December 2017

About

In this devroom we want to discuss topics surrounding hardware enablement. Subjects can range from the firmware running on the bare metal machine, drivers and plumbing all the way to the user interface.
We welcome a board range of presentations, including but not limited to technical talks, state of union summaries as well as discussions that facilitate the collaboration between community members, software vendors and OEMs. A particular emphasis will be given to talks covering a significant part of the software stack involved in hardware enablement, with an obvious focus on using open source throughout the whole stack.

Visit https://fosdem.org for general information about FOSDEM.

Talk Format

  • To cover the wide range of topics we will prefer short talks (about 15-25 minutes). Please include at least 5 minutes for discussions and questions.
  • Presentations will be recorded and streamed. Sending your proposal implies giving permission to be recorded. Exceptions may be possible.
  • Proposals need to be submitted via pentabarfΒ  (see "Submission" below for details)

Topics & Examples

  • UX design to enable users to use their HW effectively
  • Firmware:
    • coreboot
    • flashrom
    • UEFI EDK2 (Tianocore)
    • Security
    • Lockdown of platform using firmware
    • Updating
  • Secure Boot
  • Hardware testing / certification
  • Thunderbolt 3 security modes
  • Gaming input devices (keyboards, mice, piper)
  • Biometric authentication
  • Miracast or controlling remote devices
  • Why vendors should facilitate upstream development

Submission

The FOSDEM Pentabarf is used for submission and scheduling:

  • Please reuse and existing account otherwise register a new one.
  • Please provide your full name and email address. If you provide bio then this will be visible publically.
  • Create a new event:
    • Select "Hardware Enablement devroom" as the track
    • Provide a descriptive title
    • Provide a public abstract for your talk
    • Add any further information for paper review into the submission notes (e.g. outline, why this devroom)

I joined Red Hat's Desktop Hardware Enablement team almost a year ago. One of the things that I have been looking into recently is Thunderbolt 3. With kernel 4.13 we got a completely new kernel interface for interacting with it from userspace (the work was done by Intel). One of the two big things this interface provides is updating the firmware (the non-volatile memory, or NVM in short) of the host controller and attached thunderbolt devices. With help from Dell's Mario Limonciello, Intel's Yehezkel Bernat, and of course our own Richard Hughes I created a thunderbolt 3 plugin for fwupd, which device and host firmware updates should show up in GNOME Software (or any other fwupd userspace clients) and updating them should be a breeze. The code landed already in fwupd 0.9.7.

On the technical side this is done all done via sysfs. Thunderbolt devices (including the host) are exposed via /sys/bus/thunderbolt/devices and for the NVM we get two additional subdevices for each devices, i.e. nvm_activeX and nvm_non_activeX (where X is a global nvm device counter). The latter one contains a file called nvmem where we can write new firmware binaries to. The upgrade process is triggered by authenticating the NVM via a write to the nvm_authenticate file of the device, like # echo 1 > /sys/bus/thunderbolt/devices/0-0/nvm_authenticate. That in brief is what the thunderbolt 3 fwupd does to apply firmware updates. It uses udev to monitor for changes of attached devices. Discovery of updates is all done by the existing fwupd code, and the fwupd plugin API was actually nice to work with and made writing the plugin straight-forward and, yes even fun.

Once vendors start uploading firmware to the LVFS it will be easy, safe and reliable to update the NVM. If you're interested in helping out with fwupd or the Thunderbolt stuff then please join the mailing list where we discuss this kind of thing.

Just a very quick announcement. I created two new packages for Fedora:

  • rtags - A source code indexer for the C language family. I use it together with Emacs, to get IDE-like functionality. The package contains systemd (user) unit files that should socket-activate the rtags daemon on demand. Check via systemctl status --user rtags.socket.
  • renderdoc - a stand-alone graphics debugger; it is a wonderful tool to help developers when doing anything 3D (OpenGL, Vulkan) related.
    Thanks goes to @baldurk for a patch to install renderdoc's library into a private location.

Install on Fedora 26, 27 and rawhide via dnf install rtags renderdoc.

I also created my first flatpak:

  • NixView - as the name suggests a viewer for the free and open NIX scientific data format. It is available via flathub. Open data formats are the necessary basis for data sharing, which is one of the most important problems in science that is still not solved properly.

All this was done during the workshop & hackathon on open data formats and data sharing in neuroscience (in Japan!). Thanks goes to the German Neuroinformatics Node and Red Hat (my employer), that made it possible for me to attend it; and also to the Japanese Node for organizing it. I had a great and productive time in Japan.

I have been working on a little rust project lately (more on that some time later). For this I have been using Emacs with rust-mode, flycheck-rust, racer-mode and cargo-mode (see my config). This setup already works quite nicely with all the nice IDE-like features you would expect. But, I also really like the idea of Microsoft's language server protocol , where the various editors out there can use a single implementation of a common "language" server that does the indexing, auto-completion, docs, etc.pp. for a given programming language. The common language server for rust (rls) can be installed via rustup (assuming one is already using rustup and rust nightly, if not go to the rls README.md and follow the instructions there):


rustup component add rls --toolchain nightly
rustup component add rust-analysis --toolchain nightly
rustup component add rust-src --toolchain nightly

As for Emacs, there is the common lsp-mode and the rust specific lsp-rust. Btw, RMS also seems to like lsp; who knows maybe we get out of the box support for it someday. For now the installation via use-package is also fairly trivial:


(use-package rust-mode
    :mode "\\.rs\\'"
    :init
    (setq rust-format-on-save t))
(use-package lsp-mode
    :init
    (add-hook 'prog-mode-hook 'lsp-mode)
    :config
    (use-package lsp-flycheck
        :ensure f ; comes with lsp-mode
        :after flycheck))
(use-package lsp-rust
    :after lsp-mode)

This already provides all the goodies: auto-completion, eldoc, goto definition, symbol references, flycheck.
I have used it for the last couple of days and it mostly works. On the initial opening of a project it will hog the CPU and spin up the fans but that doesn't last too long. Sometimes it completely hangs Emacs. Nothing that killall rls followed by a M-x revert-buffer can't fix. (It seems to correlate with my having written some bad rust code full of errors, so ultimately I am to be blamed I guess). Long story short, quite usable already with a few glitches, but very promising!

Although Builder clearly is The Future as GNOME IDE, I still all my coding in Emacs, mostly because I have been using it for such a long time that my brain is to all the shortcuts and workflows. But Emacs can be a good IDE too. The most obvious everyday features that I want from an IDE are good source code navigation and active assistance while editing. In the first category are tasks like jumping to symbol's definition, find all callers of a function and such things. For editing, auto-completion, immediate warnings and error reporting, semantic-aware re-factoring are a must. Specifically for GNOME related development, I need all this to also work with JHBuild.

Emacs autocompletion

Auto-completion via irony-mode

Emacs can do all these with a combination of various packages: RTags indexes C/C++ source code and provides all sorts of functionality on top of that, like follow-symbol, find-references, rename-symbol and even fix obvious errors for you (via clang's "Fix-It Hints"). Auto-completion and "online" diagnostics are currently handled via Irony-mode (via company-irony and company-flycheck); RTags could do both too, I use irony-mode mostly for historic reasons and it works quite well. Additionally, irony-mode also has integration with eldoc-mode, that will show function signatures in the mode-line.

Eldoc integration

Eldoc integration

compile database compiler wrapper

Both of these packages use (lib)clang internally, which in turn needs to know the right compile flags for all source files. CMake and other modern build system can generate a compile_commands.json file that contains that information. For automake/autoconf based projects a wrapper around gcc can in theory be used to generate the file. But for JHBuild that approach is not working, because the build directory is not the source directory, so the wrapper generates the files in the wrong place. To work around this I wrote a special gcc wrapper and small set of tools, called cdcc (for compile database cc). The cdcc-gcc wrapper will store the compile flags in a sqlite database and the cdcc-gen command can then be called with a path to the source directory to generate the corresponding compile_commands.json. The easy way to use cdcc with JHBuild is to put the following in your ~/.config/jhbuildrc (this includes a fallback if it is not installed):

if spawn.find_executable('cdcc-gcc') is not None:
    os.environ['CC'] = 'cdcc-gcc'
    os.environ['CXX'] = 'cdcc-g++'

After the build is done, the compile_commands.json for each module can then be generated with cdcc-gen * in the JHBuild checkout root. The Emacs package cmake-ide is used to automatically set up RTags and friends with the correct settings, if it detects the compile_commands.json file.

Installation

RPMs for Fedora of RTags and cdcc can be found in my copr. The rdm server for RTags can easily be started via systemd socket activation, instructions are here. I install the irony-server binary locally in my ~/.emacs.d folder via a custom script.
I load all Emacs packages with use-package, you can look at my init.el for details.

Last Thursday I merged the Apple Filing Protocol (AFP) backend for GVfs; so we finally have support for Apple shares too now. It has been written by Carl-Anton Ingmarsson and it was his Summer of Code 2011 project. It is on the master branch and thus will be in the next unstable release. Please test it and report bugs against the "afp backend" component.

Carl-Anton did quite an impressive job - probably best depicted by the diffstat of the merge:

 client/Makefile.am            |    1 
 client/afpuri.c               |  269 ++
 client/gdaemonvfs.c           |    2 
 configure.ac                  |   31 
 daemon/Makefile.am            |   45 
 daemon/afp-browse.mount.in    |    8 
 daemon/afp.mount.in           |    5 
 daemon/gvfsafpconnection.c    | 1651 ++++++++++++++++
 daemon/gvfsafpconnection.h    |  420 ++++
 daemon/gvfsafpserver.c        | 1033 ++++++++++
 daemon/gvfsafpserver.h        |   85 
 daemon/gvfsbackendafp.c       | 4292 +++++++++++++++++++++++++++++++++++++++++-
 daemon/gvfsbackendafp.h       |   23 
 daemon/gvfsbackendafpbrowse.c |  608 +++++
 daemon/gvfsbackendafpbrowse.h |   47 
 daemon/gvfsbackenddnssd.c     |    6 
 daemon/gvfsjobsetattribute.h  |    1 
 17 files changed, 8491 insertions(+), 36 deletions(-)
Made by ThemesKult