Stable MIR Librarification Project Group

project group status: active project group documentation Run compiler tests

Welcome to the repository for the Stable MIR Librarification Project Group! Our goal is to provide a stable API based on the rust compiler mid-level intermediate representation (MIR) that can be used as the foundation for development of tools that want to perform sophisticated analyses and make stronger guarantees about the behavior of Rust programs.

This is the repository we use to organise and document our work.

If you are wondering how to use Stable MIR in your project, please check out the Getting Started chapter.

How Can I Get Involved?

You can find a list of the current members available on rust-lang/team.

If you’d like to participate be sure to check out any of our open issues on this repository.

We also participate on Zulip, feel free to introduce yourself over there and ask us any questions you have.

Getting Started

The StableMIR APIs are currently exposed as a crate in the compiler named stable_mir1. This crate includes the definition of structures and methods to be stabilized, which are expected to become the stable APIs in the compiler.

These APIs were designed to provide information about a Rust crate, including the body of functions, as well as type and layout information.

This chapter has two sections directed at different use cases.

  1. If you already have a crate that uses some of the Rust compiler libraries, and you are interested in migrating them to the StableMIR APIs, you can find more details about your use case at the Migrating to StableMIR section.
  2. If you are starting your integration with the Rust compiler via StableMIR, we recommend reading through the Initial Integration chapter.

We also include a Tips and Tricks section that is related to a few common obstacles tool writers encounter, that is not directly related to the stable_mir crate and APIs.

Our repository also includes a little demo crate that demonstrate how stable_mir crate can be used to analyze the main function of a crate.

The latest crate documentation can be found in the nightly documentation here

1

We are planning to release the stable_mir crate into crates.io in the near future. See issue #0030 for the current release status.

Initial Integration

In order to use stable_mir in your crate, you will need to do the following:

  1. Use a nightly toolchain that includes the stable_mir crate.
  2. Install at least the following rustup components: “rustc-dev” and “llvm-tools”
  3. Declare stable_mir as an external crate at the top of your crate:
extern crate stable_mir;

For 1 and 2, we highly recommend adding a “rust-toolchain.toml” file to your project. We also recommend to pin down a specific nightly version, to ensure all users and tests are using the same compiler version. Therefore, the same stable_mir crate version. E.g.:

# Example of a rust-toolchain.toml
[toolchain]
# Update the date to change the toolchain version.
channel = "nightly-2024-06-17"
components = ["llvm-tools", "rustc-dev", "rust-src"]

Initializing StableMIR

There’s currently no stable way to initialize the Rust compiler and StableMIR. See #0069 for more details.

Instead, StableMIR includes two unstable workarounds to give you a quick start. The run and run_with_tcx macros, both from present in the rustc_smir crate.

In order to use the run macro, you first need to declare the following external crates:

extern crate rustc_driver;
extern crate rustc_interface;
#[macro_use]
extern crate rustc_smir;
// This one you should know already. =)
extern crate stable_mir;

Then start the driver using the run!() macro:

 let result = run!(rustc_args, callback_fn);

This macro takes two arguments:

  1. A vector with the command arguments to the compiler.
  2. A callback function, which can either be a closure expression or a function ident.
    • This callback function shouldn’t take any argument, and it should return a ControlFlow<B,C>.

It will trigger the compilation in the current process, with the provided arguments, and invoke the callback after the compiler ran all its analyses, but before code generation.

The macro will return a Result<C, CompilerError<B>> representing the compilation result and the callback return value.

The second option is the run_with_tcx!() macro, which is very similar to the run macro. The main difference is that this macro passes a copy of the Rust compiler context (TyCtxt) to the callback, which allows the user to also make calls to internal compiler APIs.

Note that this option also requires the declaration of the rustc_middle external crate, i.e., you should now have the following declarations:

extern crate rustc_driver;
extern crate rustc_interface;
extern crate rustc_middle; // This one is new!
#[macro_use]
extern crate rustc_smir;
extern crate stable_mir;

Scope of StableMIR objects

StableMIR objects should not be used outside the scope of the callback function. Any usage outside this scope can panic or return an incorrect value.

For example, the following code is valid, since the logic we are storing is only used while the callback function is running:

fn print_items(rustc_args: Vec<String>) {
    let _result = run!(rustc_args, || {
       for item in stable_mir::all_local_items() {
           // Using items inside the callback!
           println!(" - {}", item.name())
       }
    });
}

However, the following usage isn’t valid, and stable_mir will panic when we invoke the name() function.

fn broken_print_items(rustc_args: Vec<String>) {
    let result = run!(rustc_args, || { ControlFlow::Continue(stable_mir::all_local_items())});
    if let ControlFlow::Continue(items) = result {
        for item in items {
            // Using item outside the callback function is wrong!
            println!(" - {}", item.name())
        }
    }
}

StableMIR objects should also not be shared across different threads.

Analyzing crate definitions

TODO

Analyzing monomorphic instances

TODO

Migrating to StableMIR

For now, we recommend looking at the Kani migration documentation.

Tricks and tips

The goal of this project is to provide an interface to the Rust compiler that can be used to empower users to build their own analysis tools. Most of these tools, however, have similar requirements that goes beyond analyzing with the Rust compiler IR. For example, most tools want to be able to analyze cargo crates, including their dependencies.

In this section, we document a few tricks that we found useful while developing different Rust analysis tools.

Storing MIR for dependencies

There is a compiler flag, -Z always-encode-mir, that can be used for storing the MIR of all functions in the crate metadata.

Handling the Std Library

Either use Xargo or cargo -Z build-std to build a new version of the std library that includes the MIR body of all functions.

You can then use the compiler --sysroot argument to point to the version you compiled.

Enabling Rust Analyzer for compiler crates

  1. Ensure that any crate that use rustc data structures have the following configuration in their Cargo.toml
[package.metadata.rust-analyzer]
rustc_private = true
  1. Set the rust-analyzer.rustc.source to “discover”. See Rust Analyzer manual for more advanced options.

Developer documentation

This folder contains some documentation useful for the development of Stable MIR

Tools

We currently have two different tools that is used to help with testing StableMIR

TestDrive

This is a small driver that we build on the top of the rust compiler. By default, this driver behaves exactly like rustc, and it can be used to build a crate or multiple packages using cargo.

This driver also contains multiple checks that will inspect the stable MIR of the crate that was compiled. In order to run those tests, you need to pass an extra argument --smir-check.

Let’s say we have a crate input.rs, in order to run all checks, one should invoke:

cargo run -p test-drive -- --smir-check test.rs
# or run the test-drive directly where ${BIN_PATH} is the binary location
${BIN_PATH}/test-drive --smir-check test.rs

In order to run this as part of a cargo build, you can run from a workspace:

# Only check SMIR for the target crate
RUSTC=${BIN_PATH} cargo rustc -p ${TARGET_PACKAGE} -- --smir-check

# Check SMIR for all crates being compiled, including dependencies
RUSTC=${BIN_PATH} RUSTFLAGS=--smir-check cargo build

This driver accepts a few other options that are all preceded with --smir prefix1. For example

  • --smir-fixme: Will run checks that currently trigger a known bug in StableMIR.
  • --smir-verbose: Print status and failure messages.
1

Since these are test tools, this documentation may be outdated.

Compiletest

This is a little utility built on the top of ui_test crate. It scans our test folders and run tests according to the selected mode. For more details on how to run this utility, check its help option:

cargo run -p compiletest -- --help

Test Suites

We have a few different test suites for Stable MIR:

  • Rust compiler [ui-fulldeps/stable-mir](https://github. com/rust-lang/rust/tree/master/tests/ui-fulldeps/stable-mir): These are very simple sanity checks that live inside the main rust repository. These tests should cover very basic functionality related to the translation of internal structures to stable ones.
  • Rust compiler suites: We are enabling the execution of rust compiler test suites. These suites are run with the rust respository compiletest. To run them, I recommend using our script scripts/run_rustc_tests.sh since there are a lot of arguments.
  • Local suites: These are suites hosted in this repository inside tests/. These are run using our local compilest and scripts/run_own_tests.sh is a script to run all local suites
    • fixme: Single file crates that are currently failing. These tests are run using --smir-fixme.
    • sanity-checks: Small examples that exercise some specific construct. These tests succeed if compilation and all smir checks succeed.

Toolchain versions

Our CI currently run a nightly job that build and run our test suites against the latest nightly toolchain. This repository also contains a .toolchain file that specifies to use nightly channel.

However, if you already have the nightly toolchain installed, this will not update the toolchain to the latest nightly. You need to explicitly do that.

If you see some errors while compiling our tools, please ensure you have the latest nightly. You might also want to check [our nightly runs](https://github. com/rust-lang/project-stable-mir/actions/workflows/nightly.yml?query=branch%2Amain) to ensure they are not currently broken. If so, you can check what was the last successful nightly run, and use its nightly version.

Custom toolchain

In order to run the tools and test suites using a local copy of rustc, do the following:

  1. Build stage 2 of the compiler. See rustc build guide for more details. E.g.:
git clone https://github.com/rust-lang/rust.git
cd rust
git checkout ${COMMIT_ID:?"Missing rustc commit id"}
./configure --enable-extended --tools=src,rustfmt,cargo --enable-debug --set=llvm.download-ci-llvm=true
./x.py build -i --stage 2
  1. Create a custom toolchain:
# This will create a toolchain named "local"
# Set the TARGET variable, e.g.: x86_64-unknown-linux-gnu
rustup toolchain link local build/${TARGET}/stage2
cp build/${TARGET}/stage2-tools-bin/* build/${TARGET}/stage2/bin/
  1. Override the current toolchain in your project-stable-mir repository.
cd ${SMIR_FOLDER}
rustup override set local
cargo clean
cargo build

By default, the build scripts will use the active toolchain for the project folder. If you run step 3, the scripts should already pick up the local toolchain. Additionally, you can also set the rust toolchain by setting the TOOLCHAIN environment variable. E.g.:

# Clean old build
cargo clean

# Run test with local toolchain
TOOLCHAIN=local ./.github/scripts/run_own_tests.sh

Note: To remove the override created on step 3, run rustup override unset in the same folder.