Stable MIR Librarification Project Group
Welcome to the repository for the Stable MIR Librarification Project Group! Our goal is to provide a stable API based on the rust compiler mid-level intermediate representation (MIR) that can be used as the foundation for development of tools that want to perform sophisticated analyses and make stronger guarantees about the behavior of Rust programs.
This is the repository we use to organise and document our work.
If you are wondering how to use Stable MIR in your project, please check out the Getting Started chapter.
How Can I Get Involved?
You can find a list of the current members available
on rust-lang/team
.
If you’d like to participate be sure to check out any of our open issues on this repository.
We also participate on Zulip, feel free to introduce yourself over there and ask us any questions you have.
Getting Started
The StableMIR APIs are currently exposed as a crate in the compiler named stable_mir
1.
This crate includes the definition of structures and methods to be stabilized,
which are expected to become the stable APIs in the compiler.
These APIs were designed to provide information about a Rust crate, including the body of functions, as well as type and layout information.
This chapter has two sections directed at different use cases.
- If you already have a crate that uses some of the Rust compiler libraries, and you are interested in migrating them to the StableMIR APIs, you can find more details about your use case at the Migrating to StableMIR section.
- If you are starting your integration with the Rust compiler via StableMIR, we recommend reading through the Initial Integration chapter.
We also include a Tips and Tricks section that is related to a few common obstacles tool writers
encounter,
that is not directly related to the stable_mir
crate and APIs.
Our repository also includes a little demo crate that
demonstrate how stable_mir
crate can be used to analyze the main function of a crate.
The latest crate documentation can be found in the nightly documentation here
We are planning to release the stable_mir
crate into crates.io in the near future.
See issue #0030 for the current release status.
Initial Integration
In order to use stable_mir
in your crate, you will need to do the following:
- Use a nightly toolchain that includes the
stable_mir
crate. - Install at least the following rustup components: “rustc-dev” and “llvm-tools”
- Declare
stable_mir
as an external crate at the top of your crate:
extern crate stable_mir;
For 1 and 2, we highly recommend adding a “rust-toolchain.toml” file to your project.
We also recommend to pin down a specific nightly version, to ensure all users and tests are using the same compiler
version.
Therefore, the same stable_mir
crate version. E.g.:
# Example of a rust-toolchain.toml
[toolchain]
# Update the date to change the toolchain version.
channel = "nightly-2024-06-17"
components = ["llvm-tools", "rustc-dev", "rust-src"]
Initializing StableMIR
There’s currently no stable way to initialize the Rust compiler and StableMIR. See #0069 for more details.
Instead, StableMIR includes two unstable workarounds to give you a quick start.
The run
and run_with_tcx
macros, both from present in the rustc_smir
crate.
In order to use the run
macro, you first need to declare the following external crates:
extern crate rustc_driver;
extern crate rustc_interface;
#[macro_use]
extern crate rustc_smir;
// This one you should know already. =)
extern crate stable_mir;
Then start the driver using the run!()
macro:
let result = run!(rustc_args, callback_fn);
This macro takes two arguments:
- A vector with the command arguments to the compiler.
- A callback function, which can either be a closure expression or a function ident.
- This callback function shouldn’t take any argument, and it should return a
ControlFlow<B,C>
.
- This callback function shouldn’t take any argument, and it should return a
It will trigger the compilation in the current process, with the provided arguments, and invoke the callback after the compiler ran all its analyses, but before code generation.
The macro will return a Result<C, CompilerError<B>>
representing the compilation result and the callback return value.
The second option is the run_with_tcx!()
macro, which is very similar to the run
macro.
The main difference is that this macro passes a copy of the Rust compiler context (TyCtxt
) to the callback,
which allows the user to also make calls to internal compiler APIs.
Note that this option also requires the declaration of the rustc_middle
external crate, i.e., you should now have the
following declarations:
extern crate rustc_driver;
extern crate rustc_interface;
extern crate rustc_middle; // This one is new!
#[macro_use]
extern crate rustc_smir;
extern crate stable_mir;
Scope of StableMIR objects
StableMIR objects should not be used outside the scope of the callback function. Any usage outside this scope can panic or return an incorrect value.
For example, the following code is valid, since the logic we are storing is only used while the callback function is running:
fn print_items(rustc_args: Vec<String>) {
let _result = run!(rustc_args, || {
for item in stable_mir::all_local_items() {
// Using items inside the callback!
println!(" - {}", item.name())
}
});
}
However, the following usage isn’t valid, and stable_mir
will panic when we invoke the name()
function.
fn broken_print_items(rustc_args: Vec<String>) {
let result = run!(rustc_args, || { ControlFlow::Continue(stable_mir::all_local_items())});
if let ControlFlow::Continue(items) = result {
for item in items {
// Using item outside the callback function is wrong!
println!(" - {}", item.name())
}
}
}
StableMIR objects should also not be shared across different threads.
Analyzing crate definitions
TODO
Analyzing monomorphic instances
TODO
Migrating to StableMIR
For now, we recommend looking at the Kani migration documentation.
Tricks and tips
The goal of this project is to provide an interface to the Rust compiler that can be used to empower users to build their own analysis tools. Most of these tools, however, have similar requirements that goes beyond analyzing with the Rust compiler IR. For example, most tools want to be able to analyze cargo crates, including their dependencies.
In this section, we document a few tricks that we found useful while developing different Rust analysis tools.
Storing MIR for dependencies
There is a compiler flag, -Z always-encode-mir
, that can be used for storing the MIR of all functions in the crate
metadata.
Handling the Std Library
Either use Xargo
or cargo -Z build-std
to build a new version of the std library that includes the MIR body of
all functions.
You can then use the compiler --sysroot
argument to point to the version you compiled.
Enabling Rust Analyzer for compiler crates
- Ensure that any crate that use rustc data structures have the following configuration in their
Cargo.toml
[package.metadata.rust-analyzer]
rustc_private = true
- Set the
rust-analyzer.rustc.source
to “discover”. See Rust Analyzer manual for more advanced options.
Developer documentation
This folder contains some documentation useful for the development of Stable MIR
Tools
We currently have two different tools that is used to help with testing StableMIR
TestDrive
This is a small driver that we build on the top of the rust compiler.
By default, this driver behaves exactly like rustc
, and it can be used to build a crate
or multiple packages using cargo.
This driver also contains multiple checks that will inspect the stable MIR of the crate that was compiled.
In order to run those tests, you need to pass an extra argument --smir-check
.
Let’s say we have a crate input.rs
, in order to run all checks, one should invoke:
cargo run -p test-drive -- --smir-check test.rs
# or run the test-drive directly where ${BIN_PATH} is the binary location
${BIN_PATH}/test-drive --smir-check test.rs
In order to run this as part of a cargo build, you can run from a workspace:
# Only check SMIR for the target crate
RUSTC=${BIN_PATH} cargo rustc -p ${TARGET_PACKAGE} -- --smir-check
# Check SMIR for all crates being compiled, including dependencies
RUSTC=${BIN_PATH} RUSTFLAGS=--smir-check cargo build
This driver accepts a few other options that are all preceded with --smir
prefix1. For example
--smir-fixme
: Will run checks that currently trigger a known bug in StableMIR.--smir-verbose
: Print status and failure messages.
Since these are test tools, this documentation may be outdated.
Compiletest
This is a little utility built on the top of ui_test
crate.
It scans our test folders and run tests according to the selected mode.
For more details on how to run this utility, check its help option:
cargo run -p compiletest -- --help
Test Suites
We have a few different test suites for Stable MIR:
- Rust compiler [
ui-fulldeps/stable-mir
](https://github. com/rust-lang/rust/tree/master/tests/ui-fulldeps/stable-mir): These are very simple sanity checks that live inside the main rust repository. These tests should cover very basic functionality related to the translation of internal structures to stable ones. - Rust compiler suites: We are enabling the execution of rust compiler test suites.
These suites are run with the rust respository compiletest.
To run them, I recommend using our script
scripts/run_rustc_tests.sh
since there are a lot of arguments. - Local suites: These are suites hosted in this repository inside
tests/
. These are run using our localcompilest
andscripts/run_own_tests.sh
is a script to run all local suitesfixme
: Single file crates that are currently failing. These tests are run using--smir-fixme
.sanity-checks
: Small examples that exercise some specific construct. These tests succeed if compilation and all smir checks succeed.
Toolchain versions
Our CI currently run a nightly job that build and run our test suites against the latest nightly toolchain.
This repository also contains a .toolchain file that specifies to use nightly
channel.
However, if you already have the nightly toolchain installed, this will not update the toolchain to the latest nightly. You need to explicitly do that.
If you see some errors while compiling our tools, please ensure you have the latest nightly. You might also want to check [our nightly runs](https://github. com/rust-lang/project-stable-mir/actions/workflows/nightly.yml?query=branch%2Amain) to ensure they are not currently broken. If so, you can check what was the last successful nightly run, and use its nightly version.
Custom toolchain
In order to run the tools and test suites using a local copy of rustc
, do the following:
- Build stage 2 of the compiler.
See
rustc
build guide for more details. E.g.:
git clone https://github.com/rust-lang/rust.git
cd rust
git checkout ${COMMIT_ID:?"Missing rustc commit id"}
./configure --enable-extended --tools=src,rustfmt,cargo --enable-debug --set=llvm.download-ci-llvm=true
./x.py build -i --stage 2
- Create a custom toolchain:
# This will create a toolchain named "local"
# Set the TARGET variable, e.g.: x86_64-unknown-linux-gnu
rustup toolchain link local build/${TARGET}/stage2
cp build/${TARGET}/stage2-tools-bin/* build/${TARGET}/stage2/bin/
- Override the current toolchain in your
project-stable-mir
repository.
cd ${SMIR_FOLDER}
rustup override set local
cargo clean
cargo build
By default, the build scripts will use the active toolchain for the project folder. If you run step 3, the scripts should already pick up the local toolchain. Additionally, you can also set the rust toolchain by setting the TOOLCHAIN environment variable. E.g.:
# Clean old build
cargo clean
# Run test with local toolchain
TOOLCHAIN=local ./.github/scripts/run_own_tests.sh
Note: To remove the override created on step 3, run rustup override unset
in the same folder.