2020-05-29 Roadmap 2020-2021

Links

Key notes and points from the meeting itself

  • Parallel compilation:
    • There was a general consensus to deprioritize parallel compilation in favor of pursuing better improvements to incremental compilation.
    • One interesting use case is CI turnaround times. Parallel compilation doesn’t do much there since most CI systems are not very beefy. Incremental doesn’t either, but it might if you enabled incremental support (and could trust it well enough).
    • We can still land some small PRs etc without it being on the roadmap necessarily.
  • Incremental compilation:
    • pnkfelix would like to get an incremental working group up and going.
    • simulacrum is interested in working on improving profiling.
  • impl Trait: we discussed a bit what it would take to finish up impl Trait.
  • How to drum up excitement:
  • We discussed “saying no” to some of the things on the list, notably a lot of them are assigned to nikomatsakis.

Initial proposal

This section contains the proposal that was made up at the start of the meeting.

General areas and themes

These are not meant to be specific projects, but more categories that justify why we are pursuing specific projects. The categories include a bit of “what” that clarifies what the category is as well as “why” indicating why this is a category of work worth pursuing.

  • Improving the IDE experience
    • Why? In survey, a strong IDE experience was one of the top reasons given for “why I don’t use Rust” or “what would make me use Rust more often”, and the largest one that is directly under compiler team’s control.
  • Improving build times for CI
    • What? From-scratch batch compilation performance
    • What? Integration with
    • Why? Anecdotal feedback that “interactive execution is ok, but CI is painful”
    • Why? Persistent complaint when it comes to large codebases especially
  • Polish and quality assurance
    • What? Fixing bugs, improving diagnostics
  • Deliver on long-standing features, finish things up
    • Why? Rust 2020 roadmap goal
    • Why? Anecdotally, a sense of “everything I want to use is always half-finished” has come up from numerous surveys (e.g., compiler team survey, a mozilla internal surveys) as well as frequent conversations.
  • Async polish and performance
    • What? Improving diagnostics, performance of async-await
    • Why? In survey, servers are the number one application for Rust.
  • Improving the way we organize ourselves
    • What? Structural changes to help us engage newcomers and be clearer in our priorities.
    • Why? Acting on the feedback from compiler-team survey that we need more prioritization and ability to communicate what we’re doing.
    • Why? Help address the need
  • Improving incremental compilation
    • What? Making more source inputs incremental
    • Why? Improve local edit-compile-debug cycle
    • What? Increase test surface (e.g. discover new cases to test)
    • Why? Most incr-comp bugs reported do not have enough info for replication.
      • (An alternative here: Figure out ways to capture needed info automatically)

Top goals for 2021 (Edition)

These are actual specific projects. The idea here is to be thinking about “things that would be complete by mid 2021” – but really I have in mind “what can we get done by 2020”, per Hofstadter’s Law. The list is perhaps too ambitious. It’s not meant to be exhaustive necessarily, but it should I think try to identify the major things we think we can realistic accomplish and is meant to help us in driving our efforts. In particular, places where a single person is driving multiple projects are somewhat suspect and we should look carefully at which is the most important thing. It’s better to do one thing completely than two things half-way, in other words.

For each goal, we list the following:

  • Motivation: one of the motivations above
  • Driver(s): people who are driving this work, if known (typically leads of groups)
  • Group(s): groups driving this work, if any
  • Goals for 2021: try to enumerate the thing we think we can accomplish in more detail

Goals

  • Triage work
    • Motivation: Polish and quality assurance, Improving the way we organize ourselves
    • Driver(s): pnkfelix, spastorino
    • Group(s): wg-prioritization
    • Goals for 2021:
      • keep on keepin' on?
      • Keep the weekly triage meeting focused and on-time?
  • Rust-analyzer transition
    • Motivation: Improving the IDE experience
    • Driver(s): matklad, flodiebold
    • Groups(s): wg-rls
    • Goals for 2021:
      • moving rust-analyzer to be default IDE solution
  • rustc-dev-guide
    • Motivation: Improving the way we organize ourselves
    • Driver(s): spastorino
    • Group(s): wg-rustc-dev-guide
    • Goals for 2021:
      • Reorganized table of contents
      • Complete walk through?
      • Slowly fill out gaps in useful knowledge
  • Progress towards chalk integration
    • Motivation: Improving the IDE experience
    • Driver(s): nikomatsakis, flodiebold, jackh726
    • Group(s): wg-rls, wg-traits
    • Goal for 2021:
      • rust-analyzer is using chalk, enable it to type-check and manage most projects
      • extract a shared library for types usable by rustc, chalk, and rust-analyzer
      • integrate experimentally with rustc and get to the point where -Zchalk allows things to largely work
  • MIR Optimizations
    • Driver(s): oli, wesleywiser
    • Group(s): wg-mir-opt
    • Goal for 2021:
      • Speed up compile-time by removing work llvm has to do
      • Speed up runtime by doing optimizations that llvm can’t do or doesn’t have the (type?) information in order to do.
        • Basically make idiomatic code as fast as handcrafted
  • Generic associated types
    • Driver(s): nikomatsakis, matthewjasper perhaps
    • Group(s): wg-traits?
    • Goal for 2021:
      • Have some form of GATs, including type-generic associated types, usable in rustc under a feature flag
      • Fix known soundness bugs
  • Const generics
    • Motivation: Deliver on long-standing features, finish things up
    • Driver(s): varkor? who else?
    • Group(s): XXX no project group at present
    • Why this feature?
      • Commonly cited in the Rust survey as one of the things people are most excited about
      • Blocks a number of language improvements, such as SIMD, let x: [u32; _] = ..., etc
    • Goal for 2021:
      • Correct the bug around the “generic parent” (blocked on lazy normalization)
      • Stabilize? Is that too ambitious? Can we clarify other goals?
  • Const evaluation
    • Motivation: Deliver on long-standing features, finish things up
    • Driver(s): oli, ralf, ecstatic-morse
    • Group(s): wg-const-eval, miri
    • Why this feature?
      • Commonly requested feature for embedded (very important there) and for replacing lazy_static things with compile-time things
      • Loads of finished features that need a bit of carrying over the stabilization line.
  • Improve the performance of the trait system
    • Driver(s): eddyb, to a lesser extent nikomatsakis
    • Goal:
      • Complete work described in w3f#261
  • Introduce major change proposals (MCP)
    • Motivation: Improving the way we work
    • Driver(s): nikomatsakis
    • Group(s): wg-meta
    • Goal for 2021:
      • See widespread use of MCPs for major changes
  • Introduce groups for major areas in the compiler as well as targets
    • Motivation: Improving the way we work
    • Driver(s): nikomatsakis, spastorino
    • Groups(s): wg-meta
    • Goal:
      • Create clearer folks that
  • RFC 2229 Implementation
    • Motivation: Deliver on long-standing features, finish things up
    • Driver(s): nikomatsakis, matthewjasper
    • Group(s): wg-rfc-2229
    • Goal:
      • Land an initial version of the feature
  • Create a wg-incr-comp to focus on improvements to incremental compilation
    • Motivation:
    • Driver(s): pnkfelix, XXX?
    • Group(s) wg-incr-comp
    • Goal: *
  • Improving profiling infrastructure and self-profile
    • Motivation: Enabling other groups to establish metric-driven goals
    • Driver(s): simulacrum
    • Goals for 2021:
      • Be able to track disk space utilization of incremental, normal compilation
      • Track codegen unit splitting
  • impl Trait plan
    • Motivation: Deliver on long-standing features, finish things up
    • Driver(s): nikomatsakis, matthewjasper, oli-obk
    • Group(s): wg-impl-trait?
    • Goal:
      • Have to define the timeline, but basically clean up and deliver on impl Trait in various positions
      • Coordinate with lang team to see how much consensus we can get on other niggling questions
  • async-await polish
    • Motivation: Async polish and performance
    • Driver(s): tmandry, …?
    • Group(s): wg-async-await
    • Goals for 2021:
      • Hard to define concretely, maybe we can?

Awesome accomplishments

I’d like to also note some of the awesome accomplishments we’ve had since Rust 2018. Let’s list them out here:

  • self-profile and its integration into the perf page
  • Cross-lang ThinLTO and PGO (when did this ship?)
  • rustc-dev-guide has grown enormously
  • Const evaluation has made a ton of progress
  • MCP process introduced :tada:

Projects not currently planned to move forward and why

This is based on the discussion in the meeting.

  • Parallelization
    • Motivation: improving build times, but it’s not clear which project above this helps (maybe CI)
    • Would need a good driver and plan
    • But the code does exist and there are real wins to be had here for people’s experience, so quite possibly worth pursuing
    • Status for 2021:
      • In stasis
      • Might be willing to pursue, but it would require a real assessment of the plan and a set of committed people