Resolve the biggest blockers to Linux building on stable Rust
| Metadata | |
|---|---|
| Short title | Rust-for-Linux |
| Owner(s) | Niko Matsakis, Josh Triplett |
| Teams | lang, libs-api, compiler |
| Status | Flagship |
| Tracking issue | rust-lang/rust-project-goals#116 |
Summary
Stabilize unstable features required by Rust for Linux project including
- Stable support for RFL's customized ARC type
- Labeled goto in inline assembler and extended
offset_of!support - RFL on Rust CI
- Pointers to statics in constants
Motivation
The experimental support for Rust development in the Linux kernel is a watershed moment for Rust, demonstrating to the world that Rust is indeed capable of targeting all manner of low-level systems applications. And yet today that support rests on a number of unstable features, blocking the effort from ever going beyond experimental status. For 2024H2 we will work to close the largest gaps that block support.
The status quo
The Rust For Linux (RFL) project has been accepted into the Linux kernel in experimental status. The project's goal, as described in the Kernel RFC introducing it, is to add support for authoring kernel components (modules, subsystems) using Rust. Rust would join C as the only two languages permitted in the linux kernel. This is a very exciting milestone for Rust, but it's also a big challenge.
Integrating Rust into the Linux kernel means that Rust must be able to interoperate with the kernel's low-level C primitives for things like locking, linked lists, allocation, and so forth. This interop requires Rust to expose low-level capabilities that don't currently have stable interfaces.
The dependency on unstable features is the biggest blocker to Rust exiting "experimental" status. Because unstable features have no kind of reliability guarantee, this in turn means that RFL can only be built with a specific, pinned version of the Rust compiler. This is a challenge for distributions which wish to be able to build a range of kernel sources with the same compiler, rather than having to select a particular toolchain for a particular kernel version.
Longer term, having Rust in the Linux kernel is an opportunity to expose more C developers to the benefits of using Rust. But that exposure can go both ways. If Rust is constantly causing pain related to toolchain instability, or if Rust isn't able to interact gracefully with the kernel's data structures, kernel developers may have a bad first impression that causes them to write off Rust altogether. We wish to avoid that outcome. And besides, the Linux kernel is exactly the sort of low-level systems application we want Rust to be great for!
For deeper background, please refer to these materials:
- The article on the latest Maintainer Summit: Committing to Rust for kernel code
- The LWN index on articles related to Rust in the kernel
- The latest status update at LPC.
- Linus talking about Rust.
- Rust in the linux kernel, by Alice Ryhl
- Using Rust in the binder driver, by Alice Ryhl
The next six months
The RFL project has a tracking issue listing the unstable features that they rely upon. After discussion with the RFL team, we identified the following subgoals as the ones most urgent to address in 2024. Closing these issues gets us within striking distance of being able to build the RFL codebase on stable Rust.
- Stable support for RFL's customized ARC type
- Labeled goto in inline assembler and extended
offset_of!support - RFL on Rust CI ([done now!])
- Pointers to statics in constants
Stable support for RFL's customized ARC type
One of Rust's great features is that it doesn't "bake in" the set of pointer types.
The common types users use every day, such as Box, Rc, and Arc, are all (in principle) library defined.
But in reality those types enjoy access to some unstable features that let them be used more widely and ergonomically.
Since few users wish to define their own smart pointer types, this is rarely an issue and there has been relative little pressure to stabilize those mechanisms.
The RFL project needs to integrate with the Kernel's existing reference counting types and intrusive linked lists.
To achieve these goals they've created their own variant of Arc (hereafter denoted as rfl::Arc),
but this type cannot be used as idiomatically as the Arc type found in libstd without two features:
- The ability to be used in methods (e.g.,
self: rfl::Arc<Self>), aka "arbitrary self types", specified in RFC #3519. - The ability to be coerce to dyn types like
rfl::Arc<dyn Trait>and then support invoking methods onTraitthrough dynamic dispatch.- This requires the use of two unstable traits,
CoerceUnsizedandDynDispatch, neither of which are close to stabilization. - However, RFC #3621 provides for a "shortcut" -- a stable interface using
derivethat expands to those traits, leaving room to evolve the underlying details.
- This requires the use of two unstable traits,
Our goal for 2024 is to close those gaps, most likely by implementing and stabilizing RFC #3519 and RFC #3621.
Labeled goto in inline assembler and extended offset_of! support
These are two smaller extensions required by the Rust-for-Linux kernel support. Both have been implemented but more experience and/or development may be needed before stabilization is accepted.
RFL on Rust CI
Update: Basic work was completed in PR #125209 by Jakub Beránek during the planning process! We are however still including a team ask of T-compiler to make sure we have agreed around the policy regarding breakage due to unstable features.
Rust sometimes integrates external projects of particular importance or interest into its CI. This gives us early notice when changes to the compiler or stdlib impact that project. Some of that breakage is accidental, and CI integration ensures we can fix it without the project ever being impacted. Otherwise the breakage is intentional, and this gives us an early way to notify the project so they can get ahead of it.
Because of the potential to slow velocity and incur extra work, the bar for being integrated into CI is high, but we believe that Rust For Linux meets that bar. Given that RFL would not be the first such project to be integrated into CI, part of pursuing this goal should be establishing clearer policies on when and how we integrate external projects into our CI, as we now have enough examples to generalize somewhat.
Pointers to statics in constants
The RFL project has a need to create vtables in read-only memory (unique address not required). The current implementation relies on the const_mut_refs and const_refs_to_static features (representative example). Discussion has identified some questions that need to be resolved but no major blockers.
The "shiny future" we are working towards
The ultimate goal is to enable smooth and ergonomic interop between Rust and the Linux kernel's idiomatic data structures.
In addition to the work listed above, there are a few other obvious items that the Rust For Linux project needs. If we can find owners for these this year, we could even get them done as a "stretch goal":
Stable sanitizer support
Support for building and using sanitizers, in particular KASAN.
Custom builds of core/alloc with specialized configuration options
The RFL project builds the stdlib with a number of configuration options to eliminate undesired aspects of libcore (listed in RFL#2). They need a standard way to build a custom version of core as well as agreement on the options that the kernel will continue using.
Code-generation features and compiler options
The RFL project requires various code-generation options. Some of these are related to custom features of the kernel, such as X18 support (rust-lang/compiler-team#748) but others are codegen options like sanitizers and the like. Some subset of the options listed on RFL#2 will need to be stabilized to support being built with all required configurations, but working out the precise set will require more effort.
Ergonomic improvements
Looking further afield, possible future work includes more ergonomic versions of the special patterns for safe pinned initialization or a solution to custom field projection for pinned types or other smart pointers.
Design axioms
- First, do no harm. If we want to make a good first impression on kernel developers, the minimum we can do is fit comfortably within their existing workflows so that people not using Rust don't have to do extra work to support it. So long as Linux relies on unstable features, users will have to ensure they have the correct version of Rust installed, which means imposing labor on all Kernel developers.
- Don't let perfect be the enemy of good. The primary goal is to offer stable support for the particular use cases that the Linux kernel requires. Wherever possible we aim to stabilize features completely, but if necessary, we can try to stabilize a subset of functionality that meets the kernel developers' needs while leaving other aspects unstable.
Ownership and team asks
Here is a detailed list of the work to be done and who is expected to do it. This table includes the work to be done by owners and the work to be done by Rust teams (subject to approval by the team in an RFC/FCP).
- The
badge indicates a requirement where Team support is needed.
| Subgoal | Owner(s) or team(s) | Notes |
|---|---|---|
| Overall program management | Niko Matsakis, Josh Triplett | |
| Arbitrary self types v2 | Adrian Taylor | |
| ↳ |  RFC #3519 | |
| ↳ | | |
| ↳ Implementation | ||
| ↳ Standard reviews | compiler | |
| ↳ Stabilization decision | lang | |
| Derive smart pointer | Alice Ryhl | |
| ↳ | RFC #3621 | |
| ↳ RFC decision | lang | |
| ↳ Implementation | Ding Xiang Fei | |
| ↳ Author stabilization report | Ding Xiang Fei | |
| ↳ Stabilization decision | lang | |
asm_goto | Gary Guo | |
| ↳ | | |
| ↳ Real-world usage in Linux kernel | Alice Ryhl | |
| ↳ Extend to cover full RFC | ||
| ↳ Author stabilization report | ||
| ↳ Stabilization decision | lang | |
Extended offset_of syntax | Ding Xiang Fei | |
| ↳ Stabilization report | ||
| ↳ Stabilization decision | libs-api | |
| Jakub Beránek | ||
| ↳ | #125209 | |
| ↳ Policy draft | ||
| ↳ Policy decision | compiler | |
| Pointers to static in constants | Niko Matsakis | |
| ↳ Stabilization report | ||
| ↳ Stabilization decision | lang |
Support needed from the project
- Lang team:
- Prioritize RFC and any related design questions (e.g., the unresolved questions)
Outputs and milestones
Outputs
Final outputs that will be produced
Milestones
Milestones you will reach along the way
Frequently asked questions
None yet.