rustc-perf improvements

| compiler champion | David Wood | | infra champion | Jakub Beránek | [channel]: https://rust-lang.zulipchat.com/#narrow/channel/478771-project-goals.2F2025h1.2Frustc-perf-improvements

This goal will be primarily worked on by James, but David Wood or Jakub Beránek can always be contacted for updates.

Summary

Continue our efforts from 2025H1 to add support to rustc-perf for distributed benchmarking across multiple platforms and configuration.

Motivation

Improving the performance of the Rust compiler is a long-standing objective of the Rust project and compiler team, which has led to the development of the project's performance tracking infrastructure. While the performance tracking infrastructure has seen many improvements in recent years, it cannot scale to support multiple benchmarking machines simultaneously.

There are increasingly demands on the performance infrastructure which require a more scalable benchmarking infrastructure - benchmarking the parallel compiler with different thread counts, different codegen backends, or different architectures.

The status quo

rustc-perf does not yet support scheduling or accepting benchmarks from multiple machines.

To ensure ongoing changes don't break existing functionality, tests have been added to guard the current system. Notable progress includes the introduction of database tests and incremental improvements to error handling in the new system.

A new system architecture has been outlined in this design document of which progress has commenced in implementing it. The new system will operate in parallel with the existing one until we are confident it can fully replace it. rustc-perf does not currently support scheduling and accepting benchmarks from multiple machines, requiring a non-trivial rearchitecting to do so. None of our policies around performance triage and handling regressions currently consider what to do in case of conflicting benchmarking results.

The next 6 months

This period is a continuation of the previous six months, during which meaningful progress was made in beginning the implementation of a new system for rustc-perf.

• Switch the default collector to the new AX-42, initially running on the legacy non-parallel system.
• Enable benchmarking to run in parallel across multiple collectors.
• Continue expanding test coverage for the new distributed rustc-perf infrastructure to reduce the risk of future regressions.
• Build a status page to display the health of each collector.
• Add support for benchmarking on AArch64 as a separate architecture.
• Enhance perf.rust-lang.org to display performance data from multiple collectors, with the ability to compare results within a given configuration (not across different configurations).

The "shiny future" we are working towards

Following the completion of this goal, it is anticipated that new platforms and configurations will be added to rustc-perf, but this is unlikely to warrant further goals.

Ownership and team asks

Task	Owner(s) or team(s)	Notes
Discussion and moral support	infra
Improve rustc-perf implementation work	James, Jakub Beránek
Standard reviews	infra
Deploy to production	infra	rustc-perf improvements, testing infrastructure
Draft performance regression policy	David Wood
Policy decision	compiler	Update performance regression policy
Inside Rust blog post announcing improvements	Jakub Beránek, David Wood

Definitions

Definitions for terms used above:

• Discussion and moral support is the lowest level offering, basically committing the team to nothing but good vibes and general support for this endeavor.
• Author RFC and Implementation means actually writing the code, document, whatever.
• Design meeting means holding a synchronous meeting to review a proposal and provide feedback (no decision expected).
• RFC decisions means reviewing an RFC and deciding whether to accept.
• Org decisions means reaching a decision on an organizational or policy matter.
• Secondary review of an RFC means that the team is "tangentially" involved in the RFC and should be expected to briefly review.
• Stabilizations means reviewing a stabilization and report and deciding whether to stabilize.
• Standard reviews refers to reviews for PRs against the repository; these PRs are not expected to be unduly large or complicated.
• Prioritized nominations refers to prioritized lang-team response to nominated issues, with the expectation that there will be some response from the next weekly triage meeting.
• Dedicated review means identifying an individual (or group of individuals) who will review the changes, as they're expected to require significant context.
• Other kinds of decisions:
  • Lang team experiments are used to add nightly features that do not yet have an RFC. They are limited to trusted contributors and are used to resolve design details such that an RFC can be written.
  • Compiler Major Change Proposal (MCP) is used to propose a 'larger than average' change and get feedback from the compiler team.
  • Library API Change Proposal (ACP) describes a change to the standard library.

Frequently asked questions

None yet.