Async Stakeholders Sep 2022: Notes

Date: Sep 13, 2022

Participants:

Tyler Mandry

Niko Matsakis

Marie Janssen

Alice Ryhl

Russell Cohen

Dario Nieuwenhuis

Eric Holk

Fabien Gaud

Zelda Hessler

Hippolyte Barraud

Yoshua Wuyts

Agenda

Our discussion will be based on this doc: Async fn in trait: A user's guide from the future

Factual questions about the doc will be answered first.

Function return syntax

#![allow(unused)]
fn main() {
async fn fetch_and_process_in_task<F>(
    f: F,
)
where
    F: Fetch,
    F: Send + 'static, // 👈 Added so `F` can be sent to new task
    F::fetch(..): Send // 👈 Added, this syntax is new!
}

#![allow(unused)]
fn main() {
async fn fetch_and_process_in_task(
    f: impl Fetch<fetch(..): Send> + Send + 'static,
)
}

Is it confusing?
Is it easy to explain to new Rust devs?
Is it ergonomic enough to use?

`Boxing::new`

#![allow(unused)]
fn main() {
let mut f = HttpFetch { .. };
fetch_and_process_dyn(Boxing::new(&mut f));
}

Is it confusing?
Is it easy to explain to new Rust devs?
Is it ergonomic enough to use?
Would you rather have implicit boxing? How strong is your preference for or against that?
How likely do you think it is that you will use an adapter besides Boxing to use async traits via dyn?
- If so, what strategy would you want?

Work stealing

Lots of the new syntax introduced is to support work stealing executors that require their tasks to be Send.

For executors that use work stealing, why?
Do we have benchmarks or data on this?

Open ended

What about async Rust has been giving you the most trouble lately?

Notes

Function return syntax

Confusing? Easy to explain? Thoughts on ergonomics?

Confusion about what F::fetch(..) refers to -- does it refer to the type before/after the await? Expectation was that it would be the type after the await, not before!

Is this one new syntax, or is it two? Can you write F::fetch by itself, or is F::fetch(..) the only accepted syntax.

Would have to rule out cases where associated types had the same name as the method, or else limit it to async functions.

We did support that, and could go forward, modulo shadowing. Explaining the idea of a syntax where you can name the zero-sized type for functions did require explaining the zero-sized function types, which was just hard to talk about.

In this we are saying that: F is a function that returns something which is send. Could you also specify that it returns T and then say that T is Send?

#![allow(unused)]
fn main() {
fn foo<T>()
where
    F: Fetch<for<'a> fetch(&'a mut Self) = T>,
    T: Send,


fn foo<T>()
where
    for<'a> {
        exists<T> {
            F: Fetch<fetch(&'a mut Self) = T>,
            T: Send
        }
    }
}

How common is thread-per-core setups? How often will people interact with Send?

Some customers for AWS SDK are trying to build TPC and don't want to have the need for send bounds. Unclear how much this is a theoretical concern.

In Embedded, Send is pretty much never needed, because you rarely have more than one core. It's quite annoying because some libraries require Send. You structure your program in a way that things are not Send because you don't need it (e.g., using RefCell). I think it'd be better if the ecosystem didn't assume Send by default.

Question ultimately is are we getting a lot of value out of work stealing.

The fact that Send and Sync is required in many places is a problem and it doesn't seem like people necessarily benefit from it.

In async-std we didn't really consider the option of not having work-stealing, but now that glommio came along people are talking about it.

In Fuchsia it's very common to have a future that is not send and not sync. Most components are run single threaded. Common answer to "oh this doesn't work because it's asking you to use Send" is "don't do that".

Browser environments are another place where Send/Sync mostly don't make sense, since you usually run on the main thread.

We definitely shouldn't have implement where Send bound on all the futures -- but it's important to consider the ergonomics of where Send bounds, especially as many people are using runtimes that require Send. For them it's important that they're ergonomic, even if maybe they didn't need it. We could talk about whether runtimes should stop requiring send and propose some kind of non-work-stealing alternative, but we do have to make sure they're ergonomic to use.

It's hard to tell if the proposed where Send bound will be an ergonomics problem or not -- if there were a lot of functions, probably.

The syntax was intended also to solve -> impl Trait for existing things.

This issue is tracking our customer's desire to use alternate async runtimes, including those that run on a single thread: https://github.com/awslabs/aws-sdk-rust/issues/52 We currently use tokio by default so our futures are almost all defined as send and sync

Lifetimes of manual desugaring (yosh)

#![allow(unused)]
fn main() {
impl Fetch for HttpFetch {
    #[refine]
    fn fetch(
        &mut self,
        request: ResourceId
    ) -> Box<dyn Future<Output = Result<ResourceData>> {
        Box::pin(async move {
            // `&mut self` would need to be live here somehow, is that supported?
        })
    }
}
}

you'd need Box<dyn Future<> + '_>, but apart from that?

But the '_ is not required. If you can write it without self, for example.

#![allow(unused)]
fn main() {
fn foo<T>()
where
    F: Fetch<for<'a> fetch(&'a mut Self) = T>,
    T: Send,
}

This generic code says that T is independent from 'a, albeit in a subtle way.

Would allow you to have implementations that are independent

Mentioning the return type of fn (alice)

Can we do this now?

#![allow(unused)]
fn main() {
async fn my_async_fn() -> i32 { .. }

type MyFut = my_async_fn(..);
}

All other left-hand-sides of where bounds are usable in all places where types go.

nikomatsakis: Yes, that's my intent, anyway. I'd like the return type syntax to just work for everything.

If there were generic arguments and the type were dependent on them, you couldn't do it, so you'd need something like this...

#![allow(unused)]
fn main() {
async fn my_async_fn(x: impl Display) -> i32 { .. }

type MyFut<T: Display> = my_async_fn(T);
}

Boxing

Summary is: in order to use dyn if there are async functions (or RPITIT), you have to wrap your object in a Boxing::new, which will cause it to allocate the future returned by Box.

#![allow(unused)]
fn main() {
fn foo() {
    let mut x = something;
    bar(Boxing::new(&x))
}

fn bar(x: &dyn AsyncIterator)
}

This is one of the things we changed.

Would be good to show some of the alternatives. Like, would Boxing work for other things? Can you plugin other things here? I like that this keeps future allocation explicit, but seems like it will be used commonly, might get heavy if every time you create a dyn you need the Boxing. I worry that it might be heavy.

In an internal project we use StackFuture that allocates futures on the stack. Makes a big array of u8s and casts the future into that. You get inline storage up to a certain size. Could I write an adapter like that?

This is sort of confusing to me. I see

#![allow(unused)]
fn main() {
async fn caller() {
    let mut f = HttpFetch { .. };
    fetch_and_process_dyn(&mut f); // 👈 ERROR!
}
}

and I see the error that this wants to be boxed, but I'm like, "Why, it's confusing?" I would look at it and say "I don't need to box the thing, I want to output something that needs to be boxed", but even that, it feels weird that I have to do it myself, I wish you could say it somewhere else? (from the chat: Hear hear!)

You want to a way for fn to say "give me something that will box?" Answer: it feels to me like impl Into<> where you're saying, give me something that's convertible.

We tried to make it so that you don't care, as the one who takes the dyn, whether it will box or not.

What I really heard was "it would be nice if I didn't have to say give me Boxing here".

Yeah it'd be nice if you could give people the "option" to do boxing without having to say it as explicitly. But then you'd have the issue where libraries have to interoperate.

My thought when I see Boxing is...I agree that it seems dangerous to implicitly box, but this Boxing thing seems weird. I don't know if there's a better solution, I don't see one, but it seems weird to me. It makes sense to me, but it seems hard to teach.

nikomatsakis: another idea would be have a warning by default that people can opt to silence on their project if they don't care.

#![allow(unused)]
fn main() {
async fn caller() {
    let mut f = HttpFetch { .. };
    fetch_and_process_dyn(&mut f); // 👈 ERROR!
    fetch_and_process_dyn(Boxing::new(&mut f)); // Works
    fetch_and_process_dyn(&mut InlineAdapter::new(f)); // Works, but doesn't box later, instead uses stack allocate
}
}

^ why is that Boxing::new(..) instead of &mut Boxing::new(..)? -- not entirely sure

#![allow(unused)]
fn main() {
async fn caller() {
    let mut f = HttpFetch { .. };
    fetch_and_process_dyn(&mut f); // 👈 works, but lints
    fetch_and_process_dyn(&mut InlineAdapter::new(f)); // works, but doesn't box later, instead uses stack allocate, and never lints
}
}

Yosh: what about the inverse: inline by default, opt-in to boxing?

Problem is that inlining doesn't always work. Each future needs to be pre-allocated, there may be infinite calls to &self methods which means the pre-allocated size is potentially unbounded.

Eric: it feels like boxing is being made at a weird place in the program. At the point where it is converted from static to dynamic, but you really want to make that decision at the call site-- isn't that the place you want to be in control over where it gets stored? I get that this is hard, and that's why were moving towards this boxing thing.

I suspect there's no right answer, but at least some of the time, you want to be able to write code that works equally well in multiple contexts.

You could inline the &mut and box the &.

Other downside: takes up a lot of stack space. (Edit: only the max of all the futures you call.)

What is the language level requirement for something to be convertible to dyn?

Basically: you'd have to have an impl where each method returns a "single pointer", or really something that implements what is required for box* creation.

Interesting language idea: e.g. could we have a way to return a Vec for cases that are not ...

It would work for anything that returns -> impl Trait, so maybe -> Vec<T>, but yes -> impl Sequence<T>.

This is recurring problem in no-std contexts, because returning vecs is a painful thing.

#![allow(unused)]
fn main() {
//

trait GetElements {
    fn elements(&self) -> Vec<Element>;
}

// but this generalized form is potentially no-std compatible

trait GetElements {
    fn elements(&self) -> impl Sequence<Element>;
}
}

Alternatives, reasons to avoid boxing? Maybe you want to cache the same box over and over again?

Needed it in microsoft beacuse allocation was really expensive in this particular context, it happened to be a very bad allocator. But this was a pretty niche scenario, probably not something your typical async programmer runs into all the time.

Common enough perhaps that it shouldn't be disregarded.

Scenario needs to be supported but it will be quite unusual.

Depends on the environment. e.g., drivers for the linux kernel? They probably share a lot of similarities.

Audio path for bluetooth is a similar use case.

Android binder currently allocates for every binder (IPC) call, but they're supposed to be low latency.

How widespread are the coercions etc?

In one case, scsi driver, have a vector of dyn, and stack future is inlining the target to a big future.

Ah, ok, so it's more the case of: hard-coding on the dyn how much space the future requires.

Don't know that we have to solve it now.

stack-future came out of: see problem, bring in calvalry, doing detailed work. How can we make it so we don't need Rust experts to solve that sort of problem of tweaking the sizes? Users can figure it out through diagnostics?

I think it has to be PGO.

Discoverability?

I'd like it to be a tool, but it's kind of science fiction at this time.

what about async has been giving most trouble lately?

AWS SDK would like to support other runtimes but often things are inextricably linked together. Very difficult to take async-std runtime without also using surf, which has its own HTTP types, vs tokio, which uses hyper. You end up with these confusing errors where there are competing things with the same name but slight differences. You runtime into the core abstraction stuff. Biggest change if things for reading bytes asynchronously could solidify around a single thing.

Read trait sufficient or more? Answer: reading from a body is the most important thing. Users have to do that very often.

Libraries often want the ability to spawn tasks or set timers?

Yep! That's a common thing. One of the ways we're trying to enable people to swap out runtimes. We've abstracted around how we ask executor "has this time passed", we have "async sleep impl", but because of all the other stuff, it's not possible. Anytime you're doing some kind of I/O or clock thing, you need some abstraction for how the executor likes to deal with the system. It's kind of meaningless to have abstractions at that level. So even though we're splitting it, it's kind of a false split.

How common is that as a request? Most common request.

Dealing with polling, lifetime across closures. Sometimes you pass a lifetime in, and you know the closure will run here, and not need the lifetime past that, but there's no way to explain that to Rust. Could maybe drop the closure? (nikomatsakis: I'd love to see an example.)

An issue I've seen a few times that arises when porting blocking code to async: no way to write async closures that borrow things from the caller. For example....

#![allow(unused)]
fn main() {
fn transaction(
    &mut self,
    f: impl FnOnce(&mut Foo),
);
}

if I have to port this to async, no way to specify it. It would have to be higher-ranked and lots of strange borrow checker errors. I've never been able to get it to work. There are some wild-hacks, the higher-ranked closure hack, but it only works in narrow cases.

Comes up on the tokio questions forum from time to time. There's a hack where you can make a helper trait, but the type inferences fails if there's a closure, basically requires that you can put the lifetimes in all the right places and I don't know how to do it.

Example where this is very painful:

blocking: https://github.com/rust-embedded/embedded-hal/blob/master/src/spi/blocking.rs#L199

async: https://github.com/rust-embedded/embedded-hal/blob/master/embedded-hal-async/src/spi.rs#L57

Ultimately gave up and used a raw pointer.

#![allow(unused)]
fn main() {
fn transaction(
    &mut self,
    f: impl FnOnce(&mut Foo) -> impl Future<Output = ()> + '_
);
}

Yosh: Oli and I were messing around today with what async closuress could potentially look like using keyword generics:

#![allow(unused)]
fn main() {
async<A> trait Fn<Args> {
    type Output;
    async<A> fn call(&self, args: Args) -> Self::Output;
}
}

which would allow

#![allow(unused)]
fn main() {
fn transaction(
    &mut self,
    f: impl async FnOnce(&mut Foo) // or something
);
}

There's this hack you can do:

#![allow(unused)]
fn main() {
trait MyAsyncFn<'a, Arg, Out>: FnOnce(Arg) -> Self::Fut {
    type Fut: Future<Output = Out>;
}

 + an impl block
}

^ I tried that but then type inference breaks when calling a function taking impl MyAsyncFn, you have to use named functions or manually write the types of args+return in the closure (I think...?) ~Dario

The fact that Waker must be Sync is not great if runtimes are not send/sync by default

https://github.com/rust-lang/rust/issues/66481

async fn fundamentals initiative

Async Stakeholders Sep 2022: Notes

Agenda

Function return syntax

`Boxing::new`

Work stealing

Other possible topics

Open ended

Notes

Function return syntax

How common is thread-per-core setups? How often will people interact with Send?

Lifetimes of manual desugaring (yosh)

Mentioning the return type of fn (alice)

Boxing

what about async has been giving most trouble lately?