Dyn upcasting coercion initiative

initiative status: active

What is this?

This page tracks the work of the dyn upcasting coercion initiative! To learn more about what we are trying to do, and to find out the people who are doing it, take a look at the charter.

Current status

The following table lists of the stages of an initiative, along with links to the artifacts that will be produced during that stage.

StageStateArtifact(s)
ProposalProposal issue
Charter
Tracking issue
Development🦀Explainer
Feature complete💤Stabilization report
Stabilized💤

Key:

  • ✅ -- phase complete
  • 🦀 -- phase in progress
  • 💤 -- phase not started yet

How Can I Get Involved?

  • Check for 'help wanted' issues on this repository!
  • If you would like to help with development, please contact the owner to find out if there are things that need doing.
  • If you would like to help with the design, check the list of active design discussions first.
  • If you have questions about the design, you can file an issue, but be sure to check the FAQ or the design-discussions first to see if there is already something that covers your topic.
  • If you are using the feature and would like to provide feedback about your experiences, please [open a "experience report" issue].
  • If you are using the feature and would like to report a bug, please open a regular issue.

We also participate on Zulip, feel free to introduce yourself over there and ask us any questions you have.

Building Documentation

This repository is also an mdbook project. You can view and build it using the following command.

mdbook serve

✏️ Updates

Lang-team initiatives give monthly updates. This section collects the updates from this initiative for posterity.

2021-Oct: Lang team update

Summary

  • Impl is largely ready
  • Blocked on deciding the question of upcast safety

Goals for this month

  • Hold design meeting about upcast safety and reach a conclusion

Deref coercion

It was discovered in #89190 that extending "unsizing coercions" to include upcasts can cause deref coercions not to trigger, if those deref coercions "deref" to a dyn type. Current plan is to issue future compatibility warnings in #89461, since the only known use of this is emulating the upcasting coercion (and hence the code will continue to work afterwards, but the Deref impl is not invoked).

Dyn upcast Charter

Goals

Summary and problem statement

  • Add the trait_upcasting feature to the language.

Motivation, use-cases, and solution sketches

  • The trait_upcasting feature adds support for trait upcasting coercion. This allows a trait object of type dyn Bar to be cast to a trait object of type dyn Foo so long as Bar: Foo.

#![allow(unused)]
#![feature(trait_upcasting)]

fn main() {
trait Foo {}

trait Bar: Foo {}

impl Foo for i32 {}

impl<T: Foo + ?Sized> Bar for T {}

let bar: &dyn Bar = &123;
let foo: &dyn Foo = bar;
}

Non-goals

  • To support dyn Trait1 + Trait2 for general traits (this syntax naturally works, and should continue to work, with auto traits).

Membership

RoleGithub
[Owner]crlf0710
[Liaison]nikomatsakis

📚 Explainer

The "explainer" is "end-user readable" documentation that explains how to use the feature being deveoped by this initiative. If you want to experiment with the feature, you've come to the right place. Until the feature enters "feature complete" form, the explainer should be considered a work-in-progress.

Design questions

Catalogs various interesting design questions that have arisen.

What qualitifies as a dyn upcasting coercion

The condition comes from the existing infrastructure within the compiler.

Currently the unsizing coercion on a trait object type allows it to:

  1. Removing one or more auto traits. (i.e. dyn Foo + Send: Unsize<dyn Foo>)

  2. Changing lifetime bounds according to subtyping rules. (dyn Bar<'static>: Unsize<dyn Bar<'a>>)

  3. Changing the principal trait to one of its supertraits. (dyn Goo: Unsize<dyn Foo> where Goo is trait Goo: Foo {})

When the third rule is involved, this unsizing coercion is a dyn upcasting coercion.

Upcast safety

Scenario

  • Casting *dyn Foo to *dyn Bar requires adjusting the vtable from a vtable for Foo to one for Bar
  • For raw pointers, the metadata can be supplied by the user with from_raw_parts
    • If that metadata is incorrect, this could cause UB:
      • our current vtable format requires loads, and the pointers may not be valid
      • a flat vtable layout could give rise to out-of-bounds loads
  • Unsafety is needed, but where?

Options

Unsafe to create a *dyn Foo

Announce that every fat pointer needs to have valid metadata part. Needs to switch the std::ptr::from_raw_parts{,_mut} APIs to be unsafe. And updates other documentations.

Implication: Not able to create a "null pointer" version of *dyn Foo unless:

  • You use Option<*dyn Foo>, of course
  • We create some kind of "dummy" vtable that is structurally correct but has no actual functions within it; we would need a function for creating "valid-but-default" metadata as part of custom DST

Update:

  • It was pointed out by steffahn that std::ptr::from_raw_parts won't create fat pointer with invalid metadata.
  • One of the remaining ways to create a pointer with invalid metadata is by using transmute.

Question:

  • Does the language UB happen at transmute site or coercion site?

Alter vtable layout to be flat

Make vtables "flat", by removing all pointer indirections in vtables and appending all the data to the tail. This makes upcasting coercion codegen become adding an offset to the metadata scalar, so won't cause real UB. Will waste some more static bytes in multiple inheritance cases than before, might make embedded-dev people unhappy.

Question from Niko:

  • Is this sufficient? Does it imply we can't use InBoundsGep? Is that already true?

Make raw pointer unsizing unsafe

Announce that raw pointer unsizing coercion must happen in unsafe blocks, while other unsizing coercions can happen outside an unsafe block. This is actually a small breaking change. So need a future compat lint to migrate existing users dealing with raw pointers and some more changes to std(POC PR at #88239 explains the details but it's a bit long). A few other MIR-level details become observable by user: whether the compiler thinks it's a unsizing coercion or not.

nikomatsakis: (cc @RalfJ, if you happen to be around)

Conversation log

Vtable format to support dyn upcasting coercion

This current design was proposed by Mario Carneiro based on previous proposals on Zulip discussion. It's a hybrid approach taking the benefits of both a "flat" design, and a "pointer"-based design.

This is implemented in #86461.

The vtable is generated by this algorithm in principle for a type T and a trait Tr:

  1. First emit the header part, including MetadataDropInPlace, MetadataSize, MetadataAlign items.
  2. Create a tree of all the supertraits of this TraitRef, by filtering out all of duplicates.
  3. Collect a set of TraitRefs consisting the trait and its first supertrait and its first supertrait's super trait,... and so on. Call this set PrefixSet
  4. Traverse the tree in post-order, for each TraitRef emit all its associated functions as either Method or Vacant entries. If this TraitRef is not in PrefixSet, emit a TraitVPtr containing a constant pointer to the vtable generated for the type T and this TraitRef.

Example


#![allow(unused)]
fn main() {
trait A {
    fn foo_a(&self) {}
}

trait B: A {
    fn foo_b(&self) {}
}

trait C: A {
    fn foo_c(&self) {}
}

trait D: B + C {
    fn foo_d(&self) {}
}
}
Vtable entries for `<S as D>`: [
    MetadataDropInPlace,
    MetadataSize,
    MetadataAlign,
    Method(<S as A>::foo_a),
    Method(<S as B>::foo_b),
    Method(<S as C>::foo_c),
    TraitVPtr(<S as C>),
    Method(<S as D>::foo_d),
]

Vtable entries for `<S as C>`: [
    MetadataDropInPlace,
    MetadataSize,
    MetadataAlign,
    Method(<S as A>::foo_a),
    Method(<S as C>::foo_c),
]

Runtime behavior for dyn upcasting coercion

At the codegen time, the same algorithm above is performed for the source principal trait and the source trait object type. If the target principal trait is in the PrefixSet set, this coercion is a no-op.

If the target principal trait is not in the PrefixSet, generate code that read the data pointer from the corresponding TraitVPtr slot.

Alternatives

Flat design

The vtable is generated by this algorithm in principle for a type T and a trait Tr:

  1. Create a tree of all the supertraits of this TraitRef, duplicate for the cyclic cases.
  2. Traverse the tree in post-order, for each TraitRef,
    1. if it has no supertrait, emit a header part, including MetadataDropInPlace, MetadataSize, MetadataAlign items.
    2. emit all its associated functions as either Method or Vacant entries.

Cons

If there is a lot of diamond inheritance that could result in exponential blowup of the vtable for example, trait A(n+1): Bn + Cn {}, trait Bn: An { fn bn(&self); }, trait Cn: An { fn cn(&self); }

the vtable for An will contain 2^n DSAs

Pointer-based design

All traits have their own vtables, and embedded all links to supertraits in the vtables

Cons

Codegen regression for single-inheritance cases, which is very widely used.

😕 FAQ