The Rust project is currently working towards a slate of 17 project goals, with 7 of them designated as Flagship Goals. This post provides selected updates on our progress towards these goals (or, in some cases, lack thereof). The full details for any particular goal are available in its associated tracking issue on the rust-project-goals repository.
Flagship goals
"Beyond the `&`"
| Progress | |
| Point of contact | |
| Champions | compiler (Oliver Scherer), lang (TC) |
| Task owners |
2 detailed updates available.
Status update:
- [x] pattern matching support of
&pin const|mut Ttypes, merged. - [x]
&pinpattern andref pin mutbinding mode, merged. - [ ]
Drop::pin_drop, waiting for review (new updates since the last review).- Unresolved question: the current implementation requires changing the
src/docs/booksubmodule, but the top repo and the sub repo must be changed together to pass the CI tests in both repos. It's because a default body is added toDrop::dropand it becomes a provided method instead of a required method in rustdoc. Is there any way to avoid that? (Possibly keep rustdoc treatingDrop::dropas a required method?)
- Unresolved question: the current implementation requires changing the
- [ ] coercion between
&pin const|mut Tand&{mut} T, waiting for review (fresh).
Status update:
Regarding the TODO list in the next 6 months, here is the current status:
Introduce &pin mut|const place borrowing syntax
- [x] parsing: #135731(https://github.com/rust-lang/rust/pull/135731), merged.
- [ ] lowering and borrowck: not started yet.
I've got some primitive ideas about borrowck, and I probably need to confirm with someone who is familiar with MIR/borrowck before starting to implement.
A pinned borrow consists two MIR statements:
- a borrow statement that creates the mutable reference,
- and an adt aggregate statement that put the mutable reference into the
Pinstruct.
I may have to add a new borrow kind so that pinned borrows can be recognized. Then traverse the dataflow graph to make sure that pinned places cannot been moved.
Pattern matching of &pin mut|const T types
In the past few months, I have struggled with the !Unpin stuffs (the original design sketch Alternative A), trying implementing it, refactoring, discussing on zulips, and was constantly confused; luckily, we have finally reached a new agreement of the Alternative B version.
- [ ] #139751(https://github.com/rust-lang/rust/pull/139751) under review (reimplemented regarding Alternative B).
Support drop(&pin mut self) for structually pinned types
- [ ] adding a new
Drop::pin_drop(&pin mut self)method: draft PR #144537(https://github.com/rust-lang/rust/pull/144537)
Supporting both Drop::drop(&mut self) and Drop::drop(&pin mut self) seems to introduce method-overloading to Rust, which I think might need some more general ways to handle (maybe by a rustc attribute?). So instead, I'd like to implemenent this via a new method Drop::pin_drop(&pin mut self) first.
Introduce &pin pat pattern syntax
Not started yet (I'd prefer doing that when pattern matching of &pin mut|const T types is ready).
Support &pin mut|const T -> &|&mut T coercion (requires T: Unpin of &pin mut T -> &mut T)
Not started yet. (It's quite independent, probably someone else can help with it)
Support auto borrowing of &pin mut|const place in method calls with &pin mut|const self receivers
Seems to be handled by Autoreborrow traits?
| Progress | |
| Point of contact | |
| Champions | |
| Task owners |
There have been lots of internal developments since the last update:
- field representing types and chained projections have received a fundamental overhaul: disallowing field paths and requiring projections to decompose. Additionally we explored how const generics could emulate FRTs.
- we discussed a potential solution to having only a single project operator & trait through a decay operation with special borrow checker treatement.
- we were able to further simplify the project trait moving the generic argument of the represented field to the project function. We also discovered the possibility that FRTs are not fundamentally necessary for field projections -- however, they are still very useful in other applications and my gut feeling is that they are also right for field projections. So we will continue our experiment with them.
- we talked about making
Project::projectbe a safe function by introducing a new kind of type.
Next Steps:
- we're still planning to merge https://github.com/rust-lang/rust/pull/146307, after I have updated it with the new FRT logic and it has been reviewed
- once that PR lands, I plan to update the library experiment to use the experimental FRTs
- then the testing using that library can begin in the Linux kernel and other projects (this is where anyone interested in trying field projections can help out!)
8 detailed updates available.
A new Perspective: Projections via Places
[@Nadrieril][] opened this zulip thread with the idea that "The normal rust way to reborrow a field uses places". He then proceeded to brainstorm a similar design for field projections with a crucial difference: making places the fundamental building block. We had a very long discussion in that thread (exchanging the existing ideas about field projection and the novel place-involving ones) that culminated in this awesome writeup by [@Nadrieril][]: https://hackmd.io/[[@Nadrieril][]][]/HJ0tuCO1-e. It is a very thorough document, so I will only be able to summarize it partially here:
- instead of the
Project*traits, we have thePlace*traits which govern what kind of place operations are possible on*xgivenx: MySmartPtr, those are reading, writing and borrowing. - we can allow custom smart pointer reborrowing possibly using the syntax
[[@MySmartPtr][]][] <place-expr> - we need multi-projections to allow simultaneous existence of
&mut x.field.aand&mut x.field.b
We still have many things to flesh out in this proposal (some of these pointed out by [@Nadrieril][]):
- how do FRTs still fit into the equation? And what are the types implementing the
Projectiontrait? - What do we do about non-indirected place containers like
MaybeUninit<T>,UnsafeCell<T>andManuallyDrop<T>? - does
BorrowKindwork as a model for the borrow checker? - how do we make
matchergonomics work nicely? - how do we get around the orphan rule limitations?
- several smaller issues/questions...
This is a very interesting viewpoint and I'm inclined to make this the main proposal idea. The traits are not too different from the current field projection design and the special borrow checker behavior was also intended at least for the first level of fields. So this is a natural evolution of the field projection proposal. Thanks a lot to [@Nadrieril][] for the stellar writeup!
Having a single Project trait
The definition of the now 3 Project* traits are 100% verbatim the same (modulo renaming of course), so we spent some time trying to unify them into a single trait. While we cannot get rid of having to have three traits, we can merge them into a single one by adding a generic:
#![allow(unused)] fn main() { #[sealed] pub trait ProjectKind { type Ptr<T: ?Sized>; }pub enum Shared {} pub enum Exclusive {}
impl ProjectKind for Shared { type Ptr<T: ?Sized> = *const T; }
impl ProjectKind for Exclusive { type Ptr<T: ?Sized> = *mut T; }
pub trait Projectable { type Target; }
pub unsafe trait Project<Kind: ProjectKind>: Projectable { type Output<'a, F: Field<Base = Self::Target>>;
unsafe fn project<'a, F: Field<Base = Self::Target>>( this: Kind::Ptr<Self>, ) -> Self::Output<'a, F>; }}
We would need some more compiler magic to ensure that nobody implements this trait generically, so impl<K> Project<K> for MyType, to keep our approach extendible (this could be an attribute if it is also useful in other cases #[rustc_deny_generic_impls]).
The benefit of merging the definitions is that we only have one single trait that we need to document and we could also add documentation on the ProjectKind types. There are also ergonomic downsides, for example all output types are now called Output and thus need to be fully qualified if multiple projection impls exist (<MyType as Project<Exclusive>>::Output<'_, F> vs MyType::OutputExclusive<'_, F>).
To make this proposal compatible with moving projections, we also either need more compiler magic to ensure that if Kind = Move we require Self: DropHusk. Or we could use associated traits and add one to ProjectKind that's then used in Project (Kind = Shared would then set this to Pointee).
This approach also makes me think a bit more about the syntax, if we discover more projections in the future, it might make sense to go for an extensible approach, like [[@keyword][]][] expr{->,.@,.,~}ident (so for example [[@move][]][] x->y or [[@mut][]][] x.y).
Moving Projections and &own
Moving projections are a third kind of projection that already exists in Rust today for Box as well as any local variable holding a struct. While we won't be including it in an MVP, we still want to make sure that we can extend the language with moving projections. Here is an example with Box:
#![allow(unused)] fn main() { fn destructure_box(mut b: Box<Struct>) -> Box<Struct> { let f1 = b.f1; b.f1 = F1::new(); b } }
This projection moves the field out of the box, invalidating it in the process. To make it valid again, a new value has to be moved in for that field. Alternatively, the partially valid box can be dropped, this will drop all other fields of Struct and then deallocate the Box. Note that this last property is implemented by compiler magic today and moving projections would allow this special behavior for Box to be a library implementation instead.
To make this kind of projection available for all types, we can make it a proper operation by adding this trait:
#![allow(unused)] fn main() { pub unsafe trait ProjectMove: Projectable { type OutputMove<'a, F: Field<Base = Self::Target>>;unsafe fn project_move<'a, F: Field<Base = Self::Target>>( this: *mut Self, ) -> Self::OutputMove<'a, F>; unsafe fn drop_husk(husk: *mut Self); }}
Importantly, we also need a drop_husk function which is responsible for cleaning up the "husk" that remains when all fields have been move-projected. In the case of Box, it deallocates the memory. So for Box we could implement this trait like this:
#![allow(unused)] fn main() { impl<T> ProjectMove for Box<T> { type OutputMove<'a, F: Field<Base = T>> = F::Type;unsafe fn project_move<'a, F: Field<Base = T>>( this: *mut Self, ) -> F::Type { let ptr = unsafe { (*this).0.pointer.as_ptr() }; ptr::read(unsafe { <*const T as Project>::project::<'a, F>(&raw const ptr) }) } unsafe fn drop_husk(husk: *mut Self) { // this is exactly the code run by `Box::drop` today, as the compiler // drops the `T` before `Box::drop` is run. let ptr = (*husk).0; unsafe { let layout = Layout::for_value_raw(ptr.as_ptr()); if layout.size() != 0 { (*husk).1.deallocate(From::from(ptr.cast()), layout); } } } }}
To support moving back into a value we have two options:
- Add a
ProjectMoveBacktrait that declares an operation which accepts a value that is moved back into the projected one, or - Add
&ownreferences.
Until now, we have explored the second option, because there are lot's of other applications for &own.
&own References
A small interlude on &own references.
An &'a own T is a special kind of exclusive reference that owns the value it points to. This means that if you drop an &own T, you also drop the pointee. You can obtain an &own T by constructing it directly to local variable &own my_local or by deriving it from an existing &own via field projections. Smart pointers generally also allow creating &own T from &own SmartPtr<T>.
One important difference to &mut T is that &own is not only temporally unique (i.e. there are no other references to that value not derived from it) but also unique for that value. In other words, one can create at most one &own T to a local variable.
#![allow(unused)] fn main() { let mut val = Struct { ... }; let x = &own val; //~ HELP: ownership transferred here drop(x); let y = &own val; //~ ERROR: cannot own `val` twice }
Since the drop(x) statement drops val, the borrow checker must disallow any future access. However, we are allowed to move a value back into the memory of val:
#![allow(unused)] fn main() { let mut val = Struct { ... }; let x = &own val; drop(x); val = Struct { ... }; let y = &own val; }
The lifetime 'a in &'a own T is that of the backing memory. It means that when 'a expires, the memory also is no longer valid (or rather it cannot be proven that it is valid after 'a). For this reason an &'a own T has to be dropped (or forgotten) before 'a expires (since after that it cannot be dropped any more).
&own T itself supports moving projections (another indicator that having them is a good idea). However only for types that don't implement Drop (similar to normal struct destructuring -- there are also talks about lifting this requirement, but no new issues arise from projecting &own).
&own and pinning
To make &pin own T with !(T: Unpin) sound in the face of panics, we have to add drop flags or have unforgettable types. We explored a design using drop flags below; there are separate efforts to experimenting with a Leak/Forget trait ongoing, I think it might be a better solution than drop flags at least for &own.
We need drop flags to ensure the drop guarantee of pinned values. The drop flag will be stored when the original &own is created and it will live on the stack of the function that created it. They are needed for the following scenario:
#![allow(unused)] fn main() { fn foo() { let x = Struct { ... }; bar(&pin own x); }fn bar(x: &pin own Struct) { if random() { std::mem::forget(x); } if random() { panic!() } } }
Since x is pinned on the stack, it needs to be dropped before foo returns (even if it unwinds). When bar forgets the owned reference, the destructor is not run, if it now panics, the destructor needs to be run in foo. But since it gave away ownership of x to bar, it is possible that bar already dropped x (this is the case when the first random() call returns false). To keep track of this, we need a drop flag in the stack frame of foo that gets set to true when x is dropped.
There are several issues with drop flags:
- we can't have
&'static own Tpointing to non-static values (for example coming from aBox::leak_ownedfunction). - field projections complicate things: if we project to a field, then we could possibly forget one field, but drop another
- solution: just store drop flags not only for the whole struct, but also all transitive fields that implement
Drop
- solution: just store drop flags not only for the whole struct, but also all transitive fields that implement
- there is different behavior between
&own Tand&pin own T, the former can be forgotten and the destructor will not run, the latter can also be forgotten, but the destructor runs regardless.
This last point convinces me that we actually want &pin own T: !Leak when T: !Leak; but IIUC, that wouldn't prevent the following code from working:
fn main() { let x = Struct { ... }; let x = &pin own x; Box::leak(Box::new(x)); }
DerefMove
The DerefMove operation & trait is something that has been discussed in the past (I haven't dug up any discussions on it though). It is the analogous operation of &own to Deref. We need to figure out the hierarchy wrt. Deref and DerefMut, but ignoring that issue for the moment, here is how DerefMove would look like:
#![allow(unused)] fn main() { trait DerefMove: DropHusk { trait Target: ?Sized;fn deref_move(&own self) -> &own Self::Target; }}
Note the super trait requirement DropHusk -- it provides a special drop operation for Self when the &own Self::Target reference has been dropped. Box<T> for example would deallocate the backing memory via DropHusk. Its definition looks like this:
#![allow(unused)] fn main() { pub unsafe trait DropHusk { unsafe fn drop_husk(husk: *mut Self); } }
We would of course also use this trait for ProjectMove. Implementing DropHusk on its own does nothing; implementing DerefMove or ProjectMove will make the compiler call drop_husk instead of Drop::drop when the value goes out of scope after it has been projected or DerefMove::deref_move has been called.
We observed that DerefMove is a lot more restrictive in its usability than Deref--- and we need projections to make it actually useful in the common case. The reason for this is that &own can only be created once, but one would like to be able to create it once per field (which is exactly what moving projections allow). Consider this example:
#![allow(unused)] fn main() { let b = Box::new(Struct { ... }); let field1 = &own b.field1; // desugars to `DerefMove::deref_move` let field2 = &own b.field2; //~ ERROR: cannot own `b` twice }
The "cannot own `b` twice error comes from the way the deref desugaring works:
#![allow(unused)] fn main() { let b = Box::new(Struct { ... }); let field1 = &own DerefMove::deref_move(&own b).f1; let field2 = &own DerefMove::deref_move(&own b).f2; // ^^^ ERROR: cannot own `b` twice }
Now it's clear that we're trying to create two &own to the same value and that can't work (the issue also arises for &mut, but that already is covered by ProjectExclusive).
We can write this instead:
#![allow(unused)] fn main() { let b = Box::new(Struct { ... }); let b = &own b; let field1 = &own b.field1; let field2 = &own b.field2; }
But that's cumbersome.
We also note that ProjectMove is the correct projection for ArcRef, as it avoids any additional refcount updates. We can rely on the ergonomic refcounting proposal to provide ergonomic ways to clone the value & perform more projections.
Making Project::project safe
In the current proposal the Project::project function is unsafe, because it takes a raw pointer as an argument. This is pretty unusual for an operator trait (it would be the first). Tyler Mandry thought about a way of making it safe by introducing "partial struct types". This new type is spelled Struct.F where F is an FRT of that struct. It's like Struct, but with the restriction that only the field represented by F can be accessed. So for example &Struct.F would point to Struct, but only allow one to read that single field. This way we could design the Project trait in a safe manner:
#![allow(unused)] fn main() { // governs conversion of `Self` to `Narrowed<F>` & replaces Projectable pub unsafe trait NarrowPointee { type Target;type Narrowed<F: Field<Base = Self::Target>>; }}
pub trait Project: NarrowPointee { type Output<F: Field<Base = Self::Type>>;
fn project(narrowed: Self::Narrowed<F>) -> Self::Output<F>;}
The NarrowPointee trait allows a type to declare that it supports conversions of its Target type to Target.F. For example, we would implement it for RefMut like this:
#![allow(unused)] fn main() { unsafe impl<'a, T> NarrowPointee for RefMut<'a, T> { type Target = T; type Narrowed<F: Field<Base = T>> = RefMut<'a, T.F>; } }
Then we can make the narrowing a builtin operation in the compiler that gets prepended on the actual coercion operation.
However, this "partial struct type" has a fatal flaw that Oliver Scherer found (edit by oli: it was actually boxy who found it): it conflicts with mem::swap, if Struct.F has the same layout as Struct, then writing to such a variable will overwrite all bytes, thus also overwriting field that aren't F. Even if we make an exception for these types and moves/copies, this wouldn't work, as a user today can rely on the fact that they write size_of::<T>() bytes to a *mut T and thus have a valid value of that type at that location. Tyler Mandry suggested we make it !Sized and even !MetaSized to prevent overwriting values of that type (maybe the Overwrite trait could come in handy here as well). But this might make "partial struct types" too weak to be truly useful. Additionally this poses many more questions that we haven't yet tackled.
Simplifying the Project trait
There have been some developments in pin ergonomics https://github.com/rust-lang/rust/issues/130494: "alternative B" is now the main approach which means that Pin<&mut T> has linear projections, which means that it doesn't change its output type depending on the concrete field (really depending on the field, not only its type). So it falls into the general projection pattern Pin<&mut Struct> -> Pin<&mut Field> which means that Pin doesn't need any where clauses when implementing Project.
Additionally we have found out that RCU also doesn't need where clauses, as we can also make its projections linear by introducing a MutexRef<'_, T> smart pointer that always allows projections and only has special behavior for T = Rcu<U>. Discussed on zulip after this message.
For this reason we can get rid of the generic argument to Project and mandate that all types that support projections support them for all fields. So the new Project trait looks like this:
#![allow(unused)] fn main() { // still need a common super trait for `Project` & `ProjectMut` pub trait Projectable { type Target: ?Sized; }pub unsafe trait Project: Projectable { type Output<F: Field<Base = Self::Target>>;
unsafe fn project<F: Field<Base = Self::Target>>( this: *const Self, ) -> Self::Output<F>; }}
Are FRTs even necessary?
With this change we can also think about getting rid of FRTs entirely. For example we could have the following Project trait:
#![allow(unused)] fn main() { pub unsafe trait Project: Projectable { type Output<F>;unsafe fn project<const OFFSET: usize, F>( this: *const Self, ) -> Self::Output<F>; }}
There are other applications for FRTs that are very useful for Rust-for-Linux. For example, storing field information for intrusive data structures directly in that structure as a generic.
More concretely, in the kernel there are workqueues that allow you to run code in parallel to the currently running thread. In order to insert an item into a workqueue, an intrusive linked list is used. However, we need to be able to insert the same item into multiple lists. This is done by storing multiple instances of the Work struct. Its definition is:
#![allow(unused)] fn main() { pub struct Work<T, const ID: u64> { ... } }
Where the ID generic must be unique inside of the struct.
trait safely. // It asserts that there is a field of type#![allow(unused)] fn main() { struct MyDriver { data: Arc<MyData>, main_work: Work<Self, 0>, aux_work: Work<Self, 1>, // more fields ... }// Then you call a macro to implement the unsafe
HasWork }Work<MyDriver, 0>at the given field // (and also exposes its offset). impl_has_work!(impl HasWork<MyDriver, 0> for MyDriver { self.main_work }); impl_has_work!(impl HasWork<MyDriver, 1> for MyDriver { self.aux_work });// Then you implement
WorkItemtwice:impl WorkItem<0> for MyDriver { type Pointer = Arc<Self>;
fn run(this: Self::Pointer) { println!("doing the main work here"); }}
impl WorkItem<1> for MyDriver { type Pointer = Arc<Self>;
fn run(this: Self::Pointer) { println!("doing the aux work here"); }}
// And finally you can call
enqueueon aQueue:let my_driver = Arc::new(MyDriver::new()); let queue: &'static Queue = kernel::workqueue::system_highpri(); queue.enqueue::<_, 0>(my_driver.clone()).expect("my_driver is not yet enqueued for id 0");
// there are different queues let queue = kernel::workqueue::system_long(); queue.enqueue::<_, 1>(my_driver.clone()).expect("my_driver is not yet enqueued for id 1");
// cannot insert multiple times: assert!(queue.enqueue::<_, 1>(my_driver.clone()).is_err());
FRTs could be used instead of this id, making the definition be Work<F: Field> (also merging the T parameter).
#![allow(unused)] fn main() { struct MyDriver { data: Arc<MyData>, main_work: Work<field_of!(Self, main_work)>, aux_work: Work<field_of!(Self, aux_work)>, // more fields ... }impl WorkItem<field_of!(MyDriver, main_work)> for MyDriver { type Pointer = Arc<Self>;
fn run(this: Self::Pointer) { println!("doing the main work here"); } }}
impl WorkItem<field_of!(MyDriver, aux_work)> for MyDriver { type Pointer = Arc<Self>;
fn run(this: Self::Pointer) { println!("doing the aux work here"); }}
let my_driver = Arc::new(MyDriver::new()); let queue: &'static Queue = kernel::workqueue::system_highpri(); queue .enqueue(my_driver.clone(), field_of!(MyDriver, main_work)) // ^ using Gary's idea to avoid turbofish .expect("my_driver is not yet enqueued for main_work");
let queue = kernel::workqueue::system_long(); queue .enqueue(my_driver.clone(), field_of!(MyDriver, aux_work)) .expect("my_driver is not yet enqueued for aux_work");
assert!(queue.enqueue(my_driver.clone(), field_of!(MyDriver, aux_work)).is_err());
This makes it overall a lot more readable (by providing sensible names instead of magic numbers), and maintainable (we can add a new variant without worrying about which IDs are unused). It also avoids the unsafe HasWork trait and the need to write the impl_has_work! macro for each Work field.
I still think that having FRTs is going to be the right call for field projections as well, so I'm going to keep their experiment going. However, we should fully explore their necessity and rationale for a future RFC.
Single Project Operator & Trait via Exclusive Decay
It would be great if we only had to add a single operator and trait and could obtain the same features as we have with two. The current reason for having two operators is to allow both shared and exclusive projections. If we could have another operation that decays an exclusive reference (or custom, exclusive smart-pointer type) into a shared reference (or the custom, shared version of the smart pointer). This decay operation would need borrow checker support in order to have simultaneous projections of one field exclusively and another field shared (and possibly multiple times).
This goes into a similar direction as the reborrowing project goal https://github.com/rust-lang/rust-project-goals/issues/399, however, it needs extra borrow checker support.
#![allow(unused)] fn main() { fn add(x: cell::RefMut<'_, i32>, step: i32) { *x = *x + step; }struct Point { x: i32, y: i32, }
fn example(p: cell::RefMut<', Point>) { let y: cell::Ref<', i32> = coerce_shared!(p.[@y][]); let y2 = coerce_shared!(p.[@y][]); // can project twice if both are coerced add(p.[@x][], *y); add(p.[@x][], *y2); assert_eq!(*y, *y2); // can still use them afterwards } }
Problems:
- explicit syntax is annoying for these "coercions", but
- we cannot make this implicit:
- if this were an implicit operation, only the borrow checker would know when one had to coerce,
- this operation is allowed to change the type,
- this results in borrow check backfeeding into typecheck, which is not possible or at least extremely difficult
Syntax
Not much movement here, it depends on the question discussed in the previous section, since if we only have one operator, we could choose .@, -> or ~; if we have to have two, then we need additional syntax to differentiate them.
Decomposing Projections
A chained projection operation should naturally decompose, so foo.[[@bar][]][].[[@baz][]][] should be the same as writing (foo.[[@bar][]][]).[[@baz][]][]. Until now, the different parenthesizing would have allowed different outcomes. This behavior is confusing and also makes many implementation details more complicated than they need to be.
Field Representing Types
Since projections now decompose, we have no need from a design perspective for multi-level FRTs. So field_of!(Foo, bar.baz) is no longer required to work. Thus we have decided to restrict FRTs to only a single field and get rid of the path. This simplifies the implementation in the compiler and also avoids certain difficult questions such as the locality of FRTs (if we had a path, we would have to walk the path and it is local, if all structs included in the path are local). Now with only a single field, the FRT is local if the struct is.
We also discovered that it is a good idea to make FRTs inhabited (they still are ZSTs), since then it allows the following pattern to work:
#![allow(unused)] fn main() { fn project_free_standing<F: Field>(_: Field, r: &F::Base) -> &F::Type { ... }// can now call the function without turbofish: let my_field = project_free_standing(field_of!(MyStruct, my_field), &my_struct); }
FRTs via const Generics
We also spent some time thinking about const generics and FRTs on zulip:
- https://rust-lang.zulipchat.com/#narrow/channel/144729-t-types/topic/const.20generics.3A.20implementing.20field.20representing.20types/with/544617587
- https://rust-lang.zulipchat.com/#narrow/channel/144729-t-types/topic/field.20representing.20values.20.26.20.60Field.3Cconst.20F.3A.20.3F.3F.3F.3E.60.20trait/with/542855620
In short, this won't be happening any time soon. However, it could be a future implementation of the field_of! macro depending on how reflection through const generics evolves (but also only in the far-ish future).
Key Developments
- coordinating with
# to ensure compatibility between the two features (allow custom pin projections to be the same as the ones for&pin mut T) - identified connection to auto reborrowing
- https://github.com/rust-lang/rust-project-goals/issues/399
- https://github.com/rust-lang/rust/issues/145612
- held a design meeting
- very positive feedback from the language team
- approved lang experiment
- got a vibe check on design axioms
- created a new Zulip channel #t-lang/custom-refs for all new features needed to make custom references more similar to
&T/&mut Tsuch as field projections, auto reborrowing and more - created the tracking issue for
#![feature(field_projections)] - opened https://github.com/rust-lang/rust/pull/146307 to implement field representing types (FRTs) in the compiler
Next Steps
- Get https://github.com/rust-lang/rust/pull/146307 reviewed & merged
Help Wanted
- When the PR for FRTs lands, try out the feature & provide feedback on FRTs
- if possible using the field-projection crate and provide feedback on projections
Internal Design Updates
Shared & Exclusive Projections
We want users to be able to have two different types of projections analogous to &T and &mut T. Each field can be projected independently and a single field can only be projected multiple times in a shared way. The current design uses two different traits to model this. The two traits are almost identical, except for their safety documentation.
We were thinking if it is possible to unify them into a single trait and have coercions similar to autoreborrowing that would allow the borrow checker to change the behavior depending on which type is projected.
Syntax
There are lots of different possibilities for which syntax we can choose, here are a couple options: [[@x][]][]->f/[[@mut][]][] x->f, [[@x][]][].f/[[@mut][]][] x.f, x.[[@f][]][]/x.mut[[@f][]][], x.ref.[[@f][]][]/x.[[@f][]][]. Also many alternatives for the sigils used: x[[@f][]][], x~f, x.@.f.
We have yet to decide on a direction we want to go in. If we are able to merge the two project traits, we can also settle on a single syntax which would be great.
Splitting Projections into Containers & Pointers
There are two categories of projections: Containers and Pointers:
- Containers are types like
MaybeUninit<T>,Cell<T>,UnsafeCell<T>,ManuallyDrop<T>. They arerepr(transparent)and apply themselves to each field, soMaybeUninit<MyStruct>has a field of typeMaybeUninit<MyField>(ifMyStructhas a field of typeMyField). - Pointers are types like
&T,&mut T,cell::Ref[Mut]<'_, T>,*const T/*mut T,NonNull<T>. They support projectingPointer<'_, Struct>toPointer<'_, Field>.
In the current design, these two classes of projections are unified by just implementing Pointer<'_, Container<Struct>> -> Pointer<'_, Container<Field>> manually for the common use-cases (for example &mut MaybeUninit<Struct> -> &mut MaybeUninit<Field>). However this means that things like &Cell<MaybeUninit<Struct>> doesn't have native projections unless we explicitly implement them.
We could try to go for a design that has two different ways to implement projections -- one for containers and one for pointers. But this has the following issues:
- there are two ways to implement projections, which means that some people will get confused which one they should use.
- making projections through multiple container types work out of the box is great, however this means that when defining a new container type and making it available for projections, one needs to consider all other container types and swear coherence with them. If we instead have an explicit way to opt in to projections through multiple container types, the implementer of that trait only has to reason about the types involved in that operation.
- so to rephrase, the current design allows more container types that users actually use to be projected whereas the split design allows arbitrary nestings of container types to be projected while disallowing certain types to be considered container types.
- The same problem exists for allowing all container types to be projected by pointer types, if I define a new pointer type I again need to reason about all container types and if it's sound to project them.
We might be able to come up with a sensible definition of "container type" which then resolves these issues, but further investigation is required.
Projections for &Custom<U>
We want to be able to have both a blanket impl<T, F: Field<Base = T>> Project<F> for &T as well as allow people to have custom projections on &Custom<U>. The motivating example for custom projections is the Rust-for-Linux Mutex that wants these projections for safe RCU abstractions.
During the design meeting, it was suggested we could add a generic to Project that only the compiler is allowed to insert, this would allow disambiguation between the two impls. We have now found an alternative approach that requires less specific compiler magic:
- Add a new marker trait
ProjectableBasethat's implemented for all types by default. - People can opt out of implementing it by writing
impl !ProjectableBase for MyStruct;(needs negative impls for marker traits). - We add
where T: ProjectableBaseto theimpl Project for &T. - The compiler needs to consider the negative impls in the overlap check for users to be able to write their own
impl<U, F> Project<F> for &Custom<U> where ...(needs negative impl overlap reasoning)
We probably want negative impls for marker traits as well as improved overlap reasoning for different reasons too, so it is probably fine to depend on them here.
enum support
enum and union shouldn't be available for projections by default, take for example &Cell<Enum>, if we project to a variant, someone else could overwrite the value with a different variant, invalidating our &Cell<Field>. This also needs a new trait, probably AlwaysActiveField (needs more name bikeshedding, but too early for that) that marks fields in structs and tuples.
To properly project an enum, we need:
- a new
CanProjectEnum(TBB) trait that provides a way to read the discriminant that's currently inhabiting the value.- it also needs to guarantee that the discriminant doesn't change while fields are being projected (this rules out implementing it for
&Cell)
- it also needs to guarantee that the discriminant doesn't change while fields are being projected (this rules out implementing it for
- a new
matchoperator that will project all mentioned fields (for&Enumthis already is the behavior formatch)
Field Representing Types (FRTs)
While implementing https://github.com/rust-lang/rust/pull/146307 we identified the following problems/design decisions:
- a FRT is considered local to the orphan check when each container base type involved in the field path is local or a tuple (see the top comment on the PR for more infos)
- FRTs cannot implement
Drop - the
Fieldtrait is not user-implementable - types with fields that are dynamically sized don't have a statically known offset, which complicates the
UnalignedFieldtrait,
I decided to simplify the first implementation of FRTs and restrict them to sized structs and tuples. It also doesn't support packed structs. Future PRs will add support for enums, unions and packed structs as well as dynamically sized types.
| Progress | |
| Point of contact | |
| Champions | |
| Task owners |
2 detailed updates available.
We've worked towards coherence checking of the CoerceShared trait, and have come to a conclusion that (at least as a first step) only one lifetime, the first one, shall participate in reborrowing. Problems abound with how to store the field mappings for CoerceShared.
Initial implementation of a Reborrow trait for types with only lifetimes with exclusive reference semantics is working but not yet upstreamed not in review. CoerceShared implementation is not yet started.
Proper composable implementation will likely require a different tactic than the current one. Safety and validity checks are currently absent as well and will require more work.
"Higher-level Rust"
| Progress | |
| Point of contact | |
| Champions | |
| Task owners |
5 detailed updates available.
New blog post:
- https://smallcultfollowing.com/babysteps/blog/2025/11/10/just-call-clone/
Exploring one way to make things more ergonomic while remaining explicit, which is to make .clone() and .alias() (1) understood by move closure desugaring and (2) optimized away when redundant.
Three new blog posts:
- Explicit capture clauses
- Bikeshedding
Handleand other follow-up thoughts - But then again...maybe alias?
The most important conclusions from those posts are
- Explicit capture clauses would be useful, I proposed one specific syntax but bikeshedding will be required. To be "ergonomic" we need the ability to refer to full places, e.g.,
move(cx.foo.clone()) || use(cx.foo). - We should consider
AliasorShareas the name forHandletrait; I am currently leaning towardsAliasbecause it can be used as both a noun and a verb and is a bit more comparable to clone -- i.e., you can say "analiasoffoo" just like you'd say "acloneoffoo". - We should look for solutions that apply well to
cloneandaliasso that higher-level Rust gets the ergonomic benefits even when cloning "heavier-weight" types to whichAliasdoes not apply.
Update:
There has been more discussion about the Handle trait on Zulip and elsewhere. Some of the notable comments:
- Downsides of the current name: it's a noun, which doesn't follow Rust naming convention, and the verb
handleis very generic and could mean many things. - Alternative names proposed:
Entangle/entangleorentangled,Share/share,Alias/alias, orRetain/retain. if we want to seriously hardcore on the science names --Mitose/mitoseorFission/fission. - There has been some criticism pointing out that focusing on handles means that other types which might be "cheaply cloneable" don't qualify.
For now I will go on using the term Handle, but I agree with the critique that it should be a verb, and currently prefer Alias/alias as an alternative.
I'm continuing to work my way through the backlog of blog posts about the conversations from Rustconf. The purposes of these blog posts is not just to socialize the ideas more broadly but also to help myself think through them. Here is the latest post:
https://smallcultfollowing.com/babysteps/blog/2025/10/13/ergonomic-explicit-handles/
The point of this post is to argue that, whatever else we do, Rust should have a way to create handles/clones (and closures that work with them) which is at once explicit and ergonomic.
To give a preview of my current thinking, I am working now on the next post which will discuss how we should add an explicit capture clause syntax. This is somewhat orthogonal but not really, in that an explicit syntax would make closures that clone more ergonomic (but only mildly). I don't have a proposal I fully like for this syntax though and there are a lot of interesting questions to work out. As a strawperson, though, you might imagine [this older proposal I wrote up](https://hackmd.io/Niko Matsakis/SyI0eMFXO?type=view), which would mean something like this:
#![allow(unused)] fn main() { let actor1 = async move(reply_tx.handle()) { reply_tx.send(...); }; let actor2 = async move(reply_tx.handle()) { reply_tx.send(...); }; }
This is an improvement on
#![allow(unused)] fn main() { let actor1 = { let reply_tx = reply_tx.handle(); async move(reply_tx.handle()) { reply_tx.send(...); } }; }
but only mildly.
The next post I intend to write would be a variant on "use, use everywhere" that recommends method call syntax and permitting the compiler to elide handle/clone calls, so that the example becomes
#![allow(unused)] fn main() { let actor1 = async move { reply_tx.handle().send(...); // -------- due to optimizations, this would capture the handle creation to happen only when future is *created* }; }
This would mean that cloning of strings and things might benefit from the same behavior:
#![allow(unused)] fn main() { let actor1 = async move { reply_tx.handle().send(some_id.clone()); // -------- the `some_id.clone()` would occur at future creation time }; }
The rationable that got me here is (a) minimizing perceived complexity and focusing on muscle memory (just add .clone() or .handle() to fix use-after-move errors, no matter when/where they occur). The cost of course is that (a) Handle/Clone become very special; and (b) it blurs the lines on when code execution occurs. Despite the .handle() occurring inside the future (resp. closure) body, it actually executes when the future (resp. closure) is created in this case (in other cases, such as a closure that implements Fn or FnMut and hence executes more than once, it might occur during each execution as well).
I wrote up a brief summary of my current thoughts on Zulip; I plan to move this content into a series of blog posts, but I figured it was worth laying it out here too for those watching this space:
09:11 (1) I don't think clones/handles are categorically different when it comes to how much you want to see them made explicit; some applications want them both to be explicit, some want them automatic, some will want a mix -- and possibly other kinds of categorizations.
09:11 (2) But I do think that if you are making everything explicit, it's useful to see the difference between a general purpose clone and a handle.
09:12 (3) I also think there are many classes of software where there is value in having everything explicit -- and that those classes are often the ones most in Rust's "sweet spot". So we should make sure that it's possible to have everything be explicit ergonomically.
09:12 (4) This does not imply that we can't make automatic clones/handles possible too -- it is just that we should treat both use cases (explicit and automatic) as first-class in importance.
09:13 (5) Right now I'm focused on the explicit case. I think this is what the use-use-everywhere was about, though I prefer a different proposal now -- basically just making handle and clone methods understood and specially handled by the compiler for optimization and desugaring purposes. There are pros and cons to that, obviously, and that's what I plan to write-up in more detail.
09:14 (6) On a related note, I think we also need explicit closure captures, which is a whole interesting design space. I don't personally find it "sufficient" for the "fully explicit" case but I could understand why others might think it is, and it's probably a good step to take.
09:15 (7) I go back and forth on profiles -- basically a fancy name for lint-groups based on application domain -- and whether I think we should go that direction, but I think that if we were going to go automatic, that's the way I would do it: i.e., the compiler will automatically insert calls to clone and handle, but it will lint when it does so; the lint can by deny-by-default at first but applications could opt into allow for either or both.
I previously wanted allow-by-default but I've decided this is a silly hill to die on, and it's probably better to move in smaller increments.
I posted this blog post that proposes that we ought to name the trait Handle and define it as a trait where clone produces an "entangled" value -- i.e., a second handle to the same underlying value.
Before that, there's been a LOT of conversation that hasn't made its way onto this tracking issue. Trying to fix that! Here is a brief summary, in any case:
- It began with the first Rust Project Goals program in 2024H2, where [@jkelleyrtp][] from Dioxus wrote a thoughtful blog post about a path to high-level Rust that eventually became a 2024H2 project goal towards ergonomic ref-counting.
- I wrote a series of blog posts about a trait I called
Claim. - Josh Triplett and I talked and Josh Triplett opened RFC #3680[], which proposed a
usekeyword anduse ||closures. Reception, I would say, was mixed; yes, this is tackling a real problem, but there were lots of concerns on the approach. I summarized the key points here. - Santiago Pastorino implemented experimental support for (a variant of) RFC #3680[] as part of the 2025H1 project goal.
- I authored a 2025H2 project goal proposing that we create an alternative RFC focused on higher-level use-cases which prompted Josh Triplett and I have to have a long and fruitful conversation in which he convinced me that this was not the right approach.
- We had a lang-team design meeting on 2025-08-27 in which I presented this survey and summary of the work done thus far.
- And then at the RustConf 2025 Unconf we had a big group discussion on the topic that I found very fruitful, as well as various follow-up conversations with smaller groups. The name
Handlearose from this and I plan to be posting further thoughts as a result.
RFC #3680: https://github.com/rust-lang/rfcs/pull/3680
| Progress | |
| Point of contact | |
| Champions | cargo (Ed Page), lang (Josh Triplett), lang-docs (Josh Triplett) |
| Task owners |
2 detailed updates available.
Key developments
- rust-lang/rust#148051
Blockers:
- rustdoc deciding on and implementing how they want frontmatter handled in doctests
Key developments:
- Overall polish
- https://github.com/rust-lang/rust/pull/145751
- https://github.com/rust-lang/rust/pull/145754
- https://github.com/rust-lang/rust/pull/146106
- https://github.com/rust-lang/rust/pull/146137
- https://github.com/rust-lang/rust/pull/146211
- https://github.com/rust-lang/rust/pull/146340
- https://github.com/rust-lang/rust/pull/145568
- https://github.com/rust-lang/cargo/pull/15878
- https://github.com/rust-lang/cargo/pull/15886
- https://github.com/rust-lang/cargo/pull/15899
- https://github.com/rust-lang/cargo/pull/15914
- https://github.com/rust-lang/cargo/pull/15927
- https://github.com/rust-lang/cargo/pull/15939
- https://github.com/rust-lang/cargo/pull/15952
- https://github.com/rust-lang/cargo/pull/15972
- https://github.com/rust-lang/cargo/pull/15975
- rustfmt work
- https://github.com/rust-lang/rust/pull/145617
- https://github.com/rust-lang/rust/pull/145766
- Reference work
- https://github.com/rust-lang/reference/pull/1974
"Unblocking dormant traits"
| Progress | |
| Point of contact | |
| Champions | |
| Task owners | Taylor Cramer, Taylor Cramer & others |
1 detailed update available.
Current status: there is an RFC for auto impl supertraits that has received some discussion and updates (thank you, Ding Xiang Fei!).
The major open questions currently are:
Syntax
The current RFC proposes:
#![allow(unused)] fn main() { trait Subtrait: Supertrait { auto impl Supertrait { // Supertrait items defined in terms of Subtrait items, if any } } }
Additionally, there is an open question around the syntax of auto impl for unsafe supertraits. The current proposal is to require unsafe auto impl Supertrait.
Whether to require impls to opt-out of auto impls
The current RFC proposes that
for MyType. extern impl Supertrait; }#![allow(unused)] fn main() { impl Supertrait for MyType {}impl Subtrait for MyType { // Required in order to manually write
Supertrait }
This makes it explicit via opt-out whether an auto impl is being applied. However, this is in conflict with the goal of allowing auto impls to be added to existing trait hierarchies. The RFC proposes to resolve this via a temporary attribute which triggers a warning. See my comment here.
Note that properly resolving whether or not to apply an auto impl requires coherence-like analysis.
| Progress | |
| Point of contact | |
| Champions | |
| Task owners | Benno Lossin, Alice Ryhl, Michael Goulet, Taylor Cramer, Josh Triplett, Gary Guo, Yoshua Wuyts |
2 detailed updates available.
On Nov 12th, there was a mini-design meeting organized by Xiangfei Ding on inplace initialization. The attendees were Xiangfei Ding, Alice Ryhl, Benno Lossin, Tyler Mandry, and Taylor Cramer.
We discussed this document: https://hackmd.io/[@rust-for-linux-][]/H11r2RXpgl
This is our first update we’re posting for the in-place init work. Overall things are progressing well, with lively discussion happening on the newly minted t-lang/in-place-init Zulip channel. Here are the highlights since the lang team design meeting at the end of July:
- Zulip: we now have a dedicated zulip channel that includes all topics surrounding in-place initialization: #t-lang/in-place-init.
- Guaranteed value emplacement: Olivier FAURE shared a new version of C++ inspired emplacement in #t-lang/in-place-init > RFC Draft: Guaranteed Value Emplacement inspired by C++’s emplacement system.
- Rosetta code sample: to help guide the comparison of the various proposals, we’ve started collecting examples to compare against each other. The first one was contributed by Alice Ryhl and is: “How can we construct a
Box<Mutex<MyType>>in-place inside theBox”. For more see #t-lang/in-place-init > Shared example: emplacing into `Box. - Evolution of the outptr proposal: Taylor Cramer’s original outptr-based emplacement proposal used concrete types as part of her proposal. Since then there has been significant discussion about alternative ways to represent out-pointers, including: #t-lang/in-place-init > out-pointer type and MIR semantics consideration.
- Placing functions as a high-level notation: Yoshua Wuyts has begun reworking the “placing functions” proposal as a high-level sugar on top of one of the other proposals, instead of directly desugaring to
MaybeUninit. For more see: #t-lang/in-place-init > Placing functions as sugar for low-level emplacement. - Generic fallibility for the
Initproposal: following feedback from the lang team meeting, Alice Ryhl posted an update showing how theInittrait could be made generic over allTrytypes instead of being limited to justResult. For more see: #t-lang/in-place-init > Makingimpl Initgeneric overResult/Option/infallible. - Interactions between emplacement and effects: Yoshua Wuyts has begun documenting the expected interactions between placing functions and other function-transforming effects (e.g.
async,try,gen). For more see: #t-lang/in-place-init > placing functions and interactions with effects.
Goals looking for help
Other goal updates
| Progress | |
| Point of contact | |
| Champions | compiler (Oliver Scherer), lang (Tyler Mandry), libs (David Tolnay) |
| Task owners |
No detailed updates available.
| Progress | |
| Point of contact | |
| Champions | |
| Task owners |
2 detailed updates available.
Since the lang meeting most progress on this project goal has been unrelated to adt_const_params.
There's been a large amount of work on min_generic_const_args, specifically Noah Lev's PR (#139558) which once landed the core of the impl work for the feature will be done. I've reviewed it together with Oliver Scherer and it's pretty much ready to go other than some small reviews.
Once this PR lands I'm hoping that there should be a fair amount of "smallish" PRs that can be made which could be a good set of PRs to mentor new-ish contributors on.
We had a design meeting on 2025-09-10, minutes available here, aiming at these questions:
There are a few concrete things I would like to get out of this meeting, listed sequentially in order of most to least important:
- Would you be comfortable stabilizing the initial ADTs-only extensions?
- This would be properly RFC'd before stabilization, this ask is just a "vibe check".
- Are you interested in seeing Per-Value Rejection for enums with undesirable variants?
- How do you feel about the idea of Lossy Conversion as an approach in general, what about specifically for the References and Raw Pointers extensions?
- How do you feel about the idea of dropping the One Equality ideal in general, what about specifically for
-0.0vs+0.0, what about specifically forNaNvalues?
The vibe checks on the first one were as follows:
Vibe check
The main ask:
Would you be comfortable stabilizing the initial ADTs-only extensions?
(plus the other ones)
nikomatsakis
I am +1 on working incrementally and focusing first on ADTs. I am supportive of stabilization overall but I don't feel like we've "nailed" the way to talk or think about these things. So I guess my "vibe" is +1 but if this doc were turned into an RFC kind of "as is" I would probably wind up -1 on the RFC, I think more work is needed (in some sense, the question is, "what is the name of the opt-in trait and why is it named that"). This space is complex and I think we have to do better at helping people understand the fine-grained distinctions between runtime values, const-eval values, and type-safe values.
Niko: if we add some sort of derive of a trait name, how much value are we getting from the derive, what should the trait be named?
tmandry
I think we'll learn the most by stabilizing ADTs in a forward compatible way (including an opt-in) now. So +1 from me on the proposed design.
It's worth noting that this is a feature that interacts with many other features, and we will be considering extensions to the MVP for the foreseeable future. To some extent the lang team has committed to this already but we should know what we're signing ourselves up for.
scottmcm
scottmcm: concern over the private fields restriction (see question below), but otherwise for the top ask, yes happy to just do "simple" types (no floats, no cells, no references, etc).
TC
As Niko said, +1 on working incrementally, and I too am supportive overall.
As a vibe, per-value rejection seems fairly OK to me in that we decided to do value-based reasoning for other const checks. It occurs to me there's some parallel with that.
https://github.com/rust-lang/rust/pull/119044
As for the opt-in on types, I see the logic. I do have reservations about adding too many opt-ins to the language, and so I'm curious about whether this can be safely removed.
Regarding floats, I see the question on these as related to our decision about how to handle padding in structs. If it makes sense to normalize or otherwise treat
-0.0and+0.0as the same, then it'd also make sense in my view to normalize or otherwise treat two structs with the same values but different padding (or where only one has initialized padding) as the same.
| Progress | |
| Point of contact | |
| Champions | bootstrap (Jakub Beránek), lang (Niko Matsakis), spec (Pete LeVasseur) |
| Task owners | Pete LeVasseur, Contributors from Ferrous Systems and others TBD, |
4 detailed updates available.
Meeting notes here: 2025-11-14 - t-fls Meeting
Key developments: PR merged for 1.90 update of the FLS. We're preparing now to work on the 1.91 update of the FLS. Blockers: None currently Help wanted: Anyone that's familiar with the Rust Reference is more than encouraged to read through the FLS to get a sense of it and where further alignment may be possible. Feel free to open issues on the FLS repo as you find things.
Meeting minutes from meeting held on 2025-10-31 (thank you to Tomas Sedovic 🥰)
Top-level:
- Keep high quality bar, merge small, well-vetted changes when possible
- Need concentrated effort to get the 1.90 FLS updates merged
- Hristian Kirtchev and Tshepang Mbambo are navigating this currently with TC
- Once 1.90 merged, we attempt first go as a team at 1.91
Discussion:
- Suggest that everyone read the Glossary as a starting point
- How to best triage / handle incoming issues?
- TC and Pete LeVasseur moved labels onto FLS repo that were needed
- Pete LeVasseur created issue template, that's in review, to help focus triage
PR with charters:
https://github.com/rust-lang/team/pull/2028
After much discussion, we have decided to charter this team as a t-spec subteam. Pete LeVasseur and I are working to make that happen now.
| Progress | |
| Point of contact | |
| Champions | |
| Task owners |
2 detailed updates available.
Here's our first status update!
-
We've been experimenting with a few different ways of emitting retags in codegen, as well as a few different forms that retags should take at this level. We think we've settled on a set of changes that's worth sending out to the community for feedback, likely as a pre-RFC. You can expect more engagement from us on this level in the next couple of weeks.
-
We've used these changes to create an initial working prototype for BorrowSanitizer that supports finding Tree Borrows violations in tiny, single-threaded Rust programs. We're working on getting Miri's test suite ported over to confirm that everything is working correctly and that we've quashed any false positives or false negatives.
-
This coming Monday, I'll be presenting on BorrowSanitizer and this project goal at the Workshop on Supporting Memory Safety in LLVM. Please reach out if you're attending and would like to chat more in person!
| Progress | |
| Point of contact | |
| Champions | compiler (Manuel Drehwald), lang (TC) |
| Task owners | Manuel Drehwald, LLVM offload/GPU contributors |
2 detailed updates available.
Automatic Differentiation
Time for the next update. By now, we've had std::autodiff for around a year in upstream rustc, but not in nightly. In order to get some more test users, I asked the infra team to re-evaluate just shipping autodiff as-is. This means that for the moment, we will increase the binary size of rustc by ~5%, even for nightly users who don't use this feature. We still have an open issue to avoid this overhead by using dlopen, please reach out if you have time to help. Thankfully, my request was accepted, so I spent most of my time lately preparing that release.
- As part of my cleanup I went through old issues, and realized we now partly support rlib's! That's a huge improvement, because it means you can use autodiff not only in your
main.rsfile, but also in dependencies (eitherlib.rs, or even rely on crates that use autodiff). With the help of Ben Kimock I figured out how to get the remaining cases covered, hopefully the PR will land soon. - I started documentation improvements in https://github.com/rust-lang/rust/pull/149082 and https://github.com/rust-lang/rust/pull/148201, which should be visible on the website from tomorrow onwards. They are likely still not perfect, so please keep opening issues if you have questions.
- We now provide a helpful error message if a user forgets enabling
lto=fat: https://github.com/rust-lang/rust/pull/148855 - After two months of work, [@sgasho][] managed to add Rust CI to enzyme! Unfortunately, Enzyme devs broke and disabled it directly, so we'll need to talk about maintaining it as part of shipping Enzyme in nightly.
I have the following elements on my TODO list as part shipping AD on nightly
- Re-enable macOS build (probably easy)
- Talk with Enzyme Devs about maintenance
- Merge rlib support (under review)
- upstream ADbenchmarks from r-l/enzyme to r-l/r as codegen tests (easy)
- Write a block post/article for https://blog.rust-lang.org/inside-rust/
GPU offload
- The llvm dev talk about GPU programming went great, I got to talk to a lot of other developers in the area of llvm offload. I hope to use some of the gained knowledge soon. Concrete steps planned are the integration of libc-gpu for IO from kernels, as well as moving over my code from the OpenMP API to the slightly lower level liboffload API.
- We confirmed that our gpu offload prototype works on more hardware. By now we have the latest AMD APU generation covered, as well as an MI 250X and an RTX 4050. My own Laptop with a slightly older
AMD Ryzen 7 PRO 7840Uunfortunately turned out to be not supported by AMD drivers. - The offload intrinsic PR by Marcelo Domínguez is now marked as ready, and I left my second round of review. Hopefully, we can land it soon!
- I spend some time trying to build and potentially ship the needed offload changes in nightly, unfortunately I still fail to build it in CI: https://github.com/rust-lang/rust/pull/148671.
All in all, I think we made great progress over the last month, and it's motivating that we finally have no blockers left for flipping the llvm.enzyme config on our nightly builds.
A longer update of the changes over the fall. We had two gsoc contributors and a lot of smaller improvements for std::autodiff. The first two improvements were already mentioned as draft PRs in the previous update, but got merged since. I also upstreamed more std::offload changes.
- Marcelo Domínguez refactored the autodiff frontend to be a proper rustc intrinsic, rather than just hackend into the frontend like I first implemented it. This already solved multiple open issues, reduced the code size, and made it generally easier to maintain going forward.
- Karan Janthe upstreamed a first implementation of "TypeTrees", which lowers rust type and layout information to Enzyme, our autodiff backend. This makes it more likely that you won't see compilation failures with the error message "Can not deduce type of ". We might refine in the future what information exactly we lower.
- Karan Janthe made sure that std::autodiff has support for f16 and and f128 types.
- One more of my offload PRs landed. I also figured out why the LLVM-IR generated by the std::offload code needed some manual adjustments in the past. We were inconsistent when communicating with LLVM's offload module, about whether we'd want a magic, extra, dyn_ptr argument, that enables kernels to use some extra features. We don't use these features yet, but for consistency we now always generate and expect the extra pointer. The bugfix is currently under review, once it lands upstream, rustc is able to run code on GPUs (still with a little help of clang).
- Marcelo Domínguez refactored my offload frontend, again introducing a proper rustc intrinsic. That code will still need to go through review, but once it lands it will get us a lot closer to a usable frontend. He also started to generate type information for our offload backend to know how many bytes to copy to and from the devices. This is a very simplified version of our autodiff typetrees.
- At RustChinaConf, I was lucky to run into the wild linker author David Lattimore, which helped me to create a draft PR that can dlopen Enzyme at runtime. This means we could ship it via rustup for people interested in std::autodiff, and don't have to link it in at build time, which would increase binary size even for those users that are not interested in it. There are some open issues, so please reach out if you have time to get the PR ready!
- [@sgasho][] spend a lot of time trying to get Rust into the Enzyme CI. Unfortunately that is a tricky process due to Enzyme's CI requirements, so it's not merged yet.
- I tried to simplify building std::autodiff by marking it as compatible with download-llvm-ci. Building LLVM from source was previously the by far slowest part of building rustc with autodiff, so this has a large potential. Unfortunately the CI experiments revealed some issues around this setting. We think we know why Enzyme's Cmake causes issues here and are working on a fix to make it more reliable.
- [@osamakader][] and bjorn3 looked into automatically enabling fat-lto when autodiff is enabled. In the past, forgetting to enable fat-lto resulted in incorrect (zero) derivatives. The first approach unfortunately wasn't able to cover all cases, so we need to see whether we can handle it nicely. If that turns out to be too complicated, we will revert it and instead "just" provide a nice error message, rather than returning incorrect derivatives.
All-in-all I spend a lot more time on infra (dlopen, cmake, download-llvm-ci, ...) then I'd like, but on the happy side there are only so many features left that I want to support here so there is an end in sight. I am also about to give a tech-talk at the upcoming LLVM dev meeting about safe GPU programming in Rust.
| Progress | |
| Point of contact | |
| Champions | lang (Josh Triplett), lang-docs (TC) |
| Task owners |
5 detailed updates available.
Update from the 2025-11-05 meeting.
Deref/Receiver
Ding Xiang Fei posted his reasoning for the trait split in the Zulip thread and suggested adding a second RFC to explain.
TC recommended writing a Reference PR. The style forces one to explain the model clearly which should then make writing the RFC easier.
The lang experiment PR for arbitrary self types have feature gates for the two options we're exploring.
Arbitrary Self Types and derive(CoercePointee) / tracking issue #44874
theemathas opened an issue derive(CoercePointee) accepts ?Sized + Sized #148399. This isn't a critical issue, just an error that arguably should be a lint.
Boxy opened a fix for a derive(CoercePointee) blocker: Forbid freely casting lifetime bounds of dyn-types
.
RFC #3851: Supertrait Auto-impl
Ding Xiang Fei is working on the implementation (the parser and HIR interface for it). Ding's also working on a more complete section dedicated to questions raised by obi1kenobi
Field projections
Benno Lossin has been posting super detailed updates on the tracking issue
We've discussed the idea of virtual places (see Zulip thread where they were proposed).
Inlining C code into Rust code
Matt Mauer had an idea to compile C code into LLVM bytecode (instead of object file) and then the llvm-link tool to merge them together and treat everything in the second bytecode file as a static inlined function. Matt suggested we could integrate this into the rustc passes.
This would make it easy to inline certain functions into Rust code without full LTO.
Relevant Zulip thread.
This sounds like a good candidate for the next Project Goals period.
Layout of core::any::TypeId
Danilo asked about the layout of TypeId -- specifically its size and whether they can rely on it because they want to store it in a C struct. The struct's size is currently 16 bytes, but that's an implementation detail.
As a vibe check, Josh Triplett and Tyler Mandry were open to guaranteeing that it's going to be at most 16 bytes, but they wanted to reserve the option to reduce the size at some point. The next step is to have the full Lang and Libs teams discuss the proposal.
Danilo will open a PR to get that discussion started.
rustfmt
Miguel brought up the "trailing empty comment" workaround for the formatting issue that made the rounds on the Linux kernel a few weeks ago. The kernel style places each import on a single line:
#![allow(unused)] fn main() { use crate::{ fmt, page::AsPageIter, }; }
rustfmt compresses this to:
#![allow(unused)] fn main() { use crate::{fmt, page::AsPageIter}; }
The workaround is to put an empty trailing comment at the end
#![allow(unused)] fn main() { use crate::{ fmt, page::AsPageIter, // }; }
This was deemed acceptable (for the time being) and merged into the mainline kernel: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4a9cb2eecc78fa9d388481762dd798fa770e1971
Miguel is in contact with rustfmt to support this behaviour without a workaround.
// PANIC: ... comments / clippy#15895
This is a proposal to add a lint that would require a PANIC comment (modeled after the SAFETY comment) to explain the circumstances during which the code will or won't panic.
Alejandra González was open to the suggestion and Henry Barker stepped up to implement it.
Deref/Receiver
During the experimentation work, Ding ran into an issue with overlapping impls (that was present even with #[unstable_feature_bound(..)]). We ran out of time but we'll discuss this offline and return to it at the next meeting.
Deref/Receiver
- Ding Xiang Fei keeps updating the PR: https://github.com/rust-lang/rust/pull/146095
- They're also working on a document to explain the consequences of this split
Arbitrary Self Types
- https://github.com/rust-lang/rust/issues/44874
- Waiting on the
Deref/Receiverwork, no updates
derive(CoercePointee)
- https://github.com/rust-lang/rust/pull/133820
- Waiting on Arbitrary self types
Pass pointers to const in asm! blocks
- RFC: https://github.com/rust-lang/rfcs/pull/3848
- The Lang team went through the RFC with Alice Ryhl on 2025-10-08 and it's in FCP now
Field projections
- Benno Lossin opened a PR here: https://github.com/rust-lang/rust/pull/146307
- Being reviewed by the compiler folks
Providing \0 terminated file names with #[track_caller]
- The feature has been implemented and stabilized with
file_as_c_stras the method name: https://github.com/rust-lang/rust/pull/145664
Supertrait auto impl RFC
- Ding Xiang Fei opened the RFC and works with the reviewers: https://github.com/rust-lang/rfcs/pull/3851
Other
- Miguel Ojeda spoke to Linus about rustfmt and they came to agreement.
I've updated the top-level description to show everything we're tracking here (please let me know if anything's missing or incorrect!).
Compiler flags
- Add -Zindirect-branch-cs-prefix PR#140740 -- landed
- Fix -Zregparm for LLVM builtins PR#145309 -- also landed
- -Zharden-sls PR#136597 -- ready for reviews
- Sanitizers target modificators PR#138736 -- close to landing
- Add assembly test for -Zreg-struct-return option PR#145382 -- landed
arbitrary_self_types and derive(CoercePointee)
Ding Xiang Fei is working on splitting the Deref and Receiver traits (the original plan was to make Deref a subtrait of Receiver). Ding has investigated and written down the method probing algorithm and documented the changes he's proposing. He'll be work on a PR that could show this in action next.
Support for pointers with asm_const
We've scheduled a design meeting with the Lang team to read through [Alice Ryhl's RFC](https://github.com/rust-lang/rfcs/pull/3848). It is scheduled for 2025-10-08.
Field projections
Benno Lossin and Ding Xiang Fei put together an experimental implementation of field projection: https://github.com/BennoLossin/rust/tree/field-projections.
They plan to polish this up, cover the missing cases and then post a PR. Tyler Mandry's been championing this from the Lang side and trying to burn down some of the open questions.
In-place initialization
Experimental PR: https://github.com/rust-lang/rust/pull/142518
Progress has slowed somewhat, it's not clear which of the proposed in-place init options to go forward with. Alice, Ding and Benno will talk and see if they can get this unblocked.
Reducing codegen size
A discussion on the status / next steps towards stabilizing cfg(no_fp_fmt_parse) PR#86048 triggered a broader discussion of Libs team's concern for littering the standard libraries with endless cfg flags and other ways of approaching this. For example, getting to a more minimal core library that could be produced using build-std and then implement the missing capabilities in the kernel.
This discussion suggested multiple avenues that I plan to follow: checking back with Libs on their current stance of cfg, appetite for building a smaller core, specifying behaviours through custom targets, etc.
NUL-terminated file names with #[track_caller] aka file_with_nul
The Libs-API team discussed this on Tuesday 2025-08-26 and agreed to the feature, but under a different name: the function will be called: file_as_c_str instead of file_with_nul:
https://github.com/rust-lang/rust/issues/141727
The tracking issue now has a new FCP proposed with that name change.
In the meantime, Alice Ryhl opened a PR renaming the function in nightly: https://github.com/rust-lang/rust/pull/145928.
| Progress | |
| Point of contact | |
| Champions | |
| Task owners |
1 detailed update available.
An RFC draft covering the MIR changes necessary to support this optimization has been written and is currently being reviewed by T-opsem. It has already received one round of review and the feedback has been incorporated in the draft.
| Progress | |
| Point of contact | |
| Champions | compiler (Oliver Scherer), lang (Scott McMurray), libs (Josh Triplett) |
| Task owners | oli-obk |
2 detailed updates available.
Another related PR:
https://github.com/rust-lang/rust/pull/148820
I implemented an initial MVP supporting only tuples and primitives (tho those are just opaque things you can't interact with further), and getting offsets for the tuple fields as well as the size of the tuple: https://github.com/rust-lang/rust/pull/146923
There are two designs of how to expose this from a libs perspective, but after a sync meeting with scottmcm yesterday we came to the conclusion that neither is objectively better at this stage so we're just going to go with the nice end-user UX version for now. For details see the PR description.
Once the MVP lands, I will mentor various interested contributors who will keep adding fields to the Type struct and variants the TypeKind enum.
The next major step is restricting what information you can get from structs outside of the current module or crate. We want to honor visibility, so an initial step would be to just never show private fields, but we want to explore allowing private fields to be shown either just within the current module or via some opt-in marker trait
| Progress | |
| Point of contact | |
| Champions | compiler (David Wood), lang (Niko Matsakis), libs (Amanieu d'Antras) |
| Task owners |
2 detailed updates available.
Notes from our meeting today:
Syntax proposal: only keyword
We are exploring the use of a new only keyword to identify "special" bounds that will affect the default bounds applied to the type parameter. Under this proposal, T: SizeOfVal is a regular bound, but T: only SizeOfVal indicates that the T: const Sized default is suppressed.
For the initial proposal, only can only be applied to a known set of traits; one possible extension would be to permit traits with only supertraits to also have only applied to them:
#![allow(unused)] fn main() { trait MyDeref: only SizeOfVal { } fn foo<T: only MyDeref>() { }// equivalent to
trait MyDeref: only SizeOfVal { } fn foo<T: MyDeref + only SizeOfVal>() { } }
We discussed a few other syntactic options:
- A
^SizeOfValsigil was appealing due to the semver analogy but rejected on the basis of it being cryptic and hard to google. - The idea of applying the keyword to the type parameter
only T: SizeOfValsort of made sense, but it would not compose well if we add additional families of "opt-out" traits likeDestructandForget, and it's not clear how it applies to supertraits.
Transitioning target
After testing, we confirmed that relaxing Target bound will result in significant breakage without some kind of transitionary measures.
We discussed the options for addressing this. One option would be to leverage "Implementable trait aliases" RFC but that would require a new trait (Deref20XX) that has a weaker bound an alias trait Deref = Deref20XX<Target: only SizeOfVal>. That seems very disruptive.
Instead, we are considering an edition-based approach where (in Rust 2024) a T: Target bound is defaulted to T: Deref<Target: only SizeOfVal> and (in Rust 20XX) T: Target is defaulted to T: Deref<Target: only Pointee>. The edition transition would therefore convert bounds to one of those two forms to be fully explicit.
One caveat here is that this edition transition, if implemented naively, would result in stronger bounds than are needed much of the time. Therefore, we will explore the option of using bottom-up analysis to determine when transitioning whether the 20XX bound can be used instead of the more conservative 2024 bound.
Supertrait bounds
We explored the implications of weakening supertrait bounds a bit, looking at this example
and#![allow(unused)] fn main() { trait FooTr<T: ?Sized> {}struct Foo<T: ?Sized>(std::marker::PhantomData<T>);
fn bar<T: ?Sized>() {}
trait Bar: FooTr<Self> /*: no longer MetaSized */ { // ^^^^^^^^^^^ error! // real examples are
Pin }TypeOf::of: fn foo(&self, x: Foo<Self>) { // ^^^^^^^^^^^^ error! bar::<Self>(); // ^^^^^^^^^^ error!// real examples are in core::fmt and core::iter: trait DoThing { fn do_thing() {} } impl<T: ?Sized> DoThing for T { default fn do_thing() {} } impl<T: Sized> DoThing for T { fn do_thing() {} } self.do_thing(); // ^^^^^^^^^^^^^ error! // specialisation case is not an issue because that feature isn't stable, we can adjust core, but is a hazard with expanding trait hierarchies in future if stabilisation is ever stabilised }}
The experimental_default_bounds work originally added Self: Trait bounds to default methods but moved away from that because it could cause region errors (source 1 / source 2). We expect the same would apply to us but we are not sure.
We decided not to do much on this, the focus remains on the Deref::Target transition as it has more uncertainty.
Sized hierarchy
The focus right now is on the "non-const" parts of the proposal, as the "const" parts are blocked on the new trait solver (https://github.com/rust-lang/rust-project-goals/issues/113). Now that the types team FCP https://github.com/rust-lang/rust/pull/144064 has completed, work can proceed to land the implementation PRs. David Wood plans to split the RFC to separate out the "non-const" parts of the proposal so it can move independently, which will enable extern types.
To that end, there are three interesting T-lang design questions to be considered.
Naming of the traits
The RFC currently proposes the following names
SizedMetaSizedPointeeSized
However, these names do not follow the "best practice" of naming the trait after the capability that it provides. As champion Niko is recommending we shift to the following names:
Sized-- should righly be calledSizeOf, but oh well, not worth changing.SizeOfVal-- named after the methodsize_of_valthat you get access to.Pointee-- the only thing you can do is point at it.
The last trait name is already used by the (unstable) std::ptr::Pointee trait. We do not want to have these literally be the same trait because that trait adds a Metadata associated type which would be backwards incompatible; if existing code uses T::Metadata to mean <T as SomeOtherTrait>::Metadata, it could introduce ambiguity if now T: Pointee due to defaults. My proposal is to rename std::ptr::Pointee to std::ptr::PointeeMetadata for now, since that trait is unstable and the design remains under some discussion. The two traits could either be merged eventually or remain separate.
Note that PointeeMetadata would be implemented automatically by the compiler for anything that implements Pointee.
Syntax opt-in
The RFC proposes that an explicit bound like T: MetaSized disabled the default T: Sized bound. However, this gives no signal that this trait bound is "special" or different than any other trait bound. Naming conventions can help here, signalling to users that these are special traits, but that leads to constraints on naming and may not scale as we consider using this mechanism to relax other defaults as proposed in my recent blog post. One idea is to use some form of syntax, so that T: MetaSized is just a regular bound, but (for example) T: =MetaSized indicates that this bound "disables" the default Sized bound. This gives users some signal that something special is going on. This = syntax is borrowing from semver constraints, although it's not a precise match (it does not mean that T: Sized doesn't hold, after all). Other proposals would be some other sigil (T: ?MetaSized, but it means "opt out from the traits above you"; T: #MetaSized, ...) or a keyword (no idea).
To help us get a feel for it, I'll use T: =Foo throughout this post.
Implicit trait supertrait bounds, edition interaction
In Rust 2024, a trait is implicitly ?Sized which gets mapped to =SizeOfVal:
#![allow(unused)] fn main() { trait Marker {} // cannot be implemented by extern types }
This is not desirable but changing it would be backwards incompatible if traits have default methods that take advantage of this bound:
#![allow(unused)] fn main() { trait NotQuiteMarker { fn dummy(&self) { let s = size_of_val(self); } } }
We need to decide how to handle this. Options are
- Just change it, breakage will be small (have to test that).
- Default to
=SizeOfValbut let users explicitly write=Pointeeif they want that. Bad because all traits will be incompatible with extern types. - Default to
=SizeOfValonly if defaulted methods are present. Bad because it's a backwards incompatible change to add a defaulted method now. - Default to
=Pointeebut addwhere Self: =SizeOfValimplicitly to defaulted methods. Now it's not backwards incompatible to add a new defaulted method, but it is backwards incompatible to change an existing method to have a default.
If we go with one of the latter options, Niko proposes that we should relax this in the next Edition (Rust 2026?) so that the default becomes Pointee (or maybe not even that, if we can).
Relaxing associated type bounds
Under the RFC, existing ?Sized bounds would be equivalent to =SizeOfVal. This is mostly fine but will cause problems in (at least) two specific cases: closure bounds and the Deref trait. For closures, we can adjust the bound since the associated type is unstable and due to the peculiarities of our Fn() -> T syntax. Failure to adjust the Deref bound in particular would prohibit the use of Rc<E> where E is an extern type, etc.
For deref bounds, David Wood is preparing a PR that simply changes the bound in a backwards incompatible way to assess breakage on crater. There is some chance the breakage will be small.
If the breakage proves problematic, or if we find other traits that need to be relaxed in a similar fashion, we do have the option of:
- In Rust 2024,
T: Derefbecomes equivalent toT: Deref<Target: SizeOfVal>unless written likeT: Deref<Target: =Pointee>. We add that annotation throughout stdlib. - In Rust 202X, we change the default, so that
T: Derefdoes not add any special bounds, and existing Rust 2024T: Derefis rewritten toT: Deref<Target: SizeOfVal>as needed.
Other notes
One topic that came up in discussion is that we may eventually wish to add a level "below" Pointee, perhaps Value, that signifies webassembly external values which cannot be pointed at. That is not currently under consideration but should be backwards compatible.
| Progress | |
| Point of contact | |
| Champions | |
| Task owners |