Futures-rs

Compatibility Layer

2019-04-18T00:00:00+00:00

Futures 0.1 Compatibility Layer

Rust’s futures ecosystem is currently split in two: On the one hand we have the vibrant ecosystem built around futures@0.1 with its many libraries working on stable Rust and on the other hand there’s the unstable std::future ecosystem with support for the ergonomic and powerful async/await language feature. To bridge the gap between these two worlds we have introduced a compatibility layer as part of the futures@0.3 extension to std::future. This blog post aims to give an overview over how to use it.

`Cargo.toml`

The compatibility layer can be enabled by setting the compat feature in your Cargo.toml:

futures-preview = { version = "0.3.0-alpha.14", features = ["compat"] }

To use futures@0.1 and futures@0.3 together in a single project, we can make use of the new cargo feature for renaming dependencies. Why? Because, even though the futures@0.3 crate is called futures-preview on crates.io, it’s lib name is also futures. By renaming futures version 0.1 to futures01, we can avoid a name collision:

[dependencies]
futures01 = { package = "futures", version = "0.1", optional = true }

Note: Renaming the crate is only required if you specify it as a dependency. If your project only depends on Tokio and thus only indirectly on futures@0.1, then no renaming is required.

Async functions on 0.1 executors

The compatibility layer makes it possible to run std futures on executors built for futures@0.1. This makes it for instance possible to run futures created via async/await! on Tokio’s executor. Here’s how this looks like:

#![feature(async_await, await_macro, futures_api)]
use futures::future::{FutureExt, TryFutureExt};

let future03 = async {
    println!("Running on the pool");
};

let future01 = future03
    .unit_error()
    .boxed()
    .compat();

tokio::run(future01);

Turning a std future into a 0.1 future requires three steps:

First, the future needs to be a TryFuture, i.e. a future with Output = Result<T, E>. If your future isn’t a TryFuture yet, you can quickly make it one using the unit_error combinator which wraps the output in a Result<T, ()>.
Next, the future needs to be Unpin. If your future isn’t Unpin yet, you can use the boxed combinator which wraps the future in a Pin<Box>.
The final step is to call the compat combinator which wraps the std future into a 0.1 future that can run on any 0.1 executor.

0.1 futures in async functions

The conversion from a 0.1 future to a std future also works via a compat combinator method:

use futures::compat::Future01CompatExt;

let future03 = future01.compat();

It converts a 0.1 Future<Item = T, Error = E> into a std Future<Output = Result<T, E>>.

Streams and Sinks

Converting between 0.1 and 0.3 streams and sinks are possible via the TryStreamExt::compat, Stream01CompatExt::compat, SinkExt::compat and Sink01CompatExt::sink_compat methods. These combinators work analogously to their future equivalents.

Spawning

Finally from the core compat feature, there’s also conversions between futures 0.1 Executor trait and futures 0.3 Spawn trait. Again, these function near identically to the futures compatibility functions with SpawnExt::compat and Executor01CompatExt::compat providing the conversions in each direction.

Async IO

With futures 0.1 IO was almost exclusively provided by Tokio, this was reflected in the core IO traits being provided by tokio-io. With 0.3 instead the futures crate is providing these core abstractions along with generic utilities that work with them. To convert between these traits you can enable the io-compat feature in your Cargo.toml (this will implicitly enable the base compat feature as well):

futures-preview = { version = "0.3.0-alpha.14", features = ["io-compat"] }

This provides the 4 expected conversions: AsyncReadExt::compat and AsyncRead01CompatExt::compat to convert between futures::io::AsyncRead and tokio-io::AsyncRead; and AsyncWriteExt::compat_write and AsyncWrite01CompatExt::compat to convert between futures::io::AsyncWrite and tokio-io::AsyncWrite.

Conclusion

The compatibility layer offers conversions in both directions and thus enables gradual migrations and experiments with std::future. With that it manages to bridge the gap between the futures 0.1 and futures 0.3 ecosystems.

Finally a self contained example that shows using futures 0.1 based hyper and tokio with a std async/await layer in between:

[dependencies]
futures-preview = { version = "0.3.0-alpha.14", features = ["io-compat"] }
hyper = "0.12.25"
tokio = "0.1.16"
pin-utils = "0.1.0-alpha.4"

#![feature(async_await, await_macro, futures_api)]

use futures::{
  compat::{Future01CompatExt, Stream01CompatExt, AsyncWrite01CompatExt},
  future::{FutureExt, TryFutureExt},
  io::AsyncWriteExt,
  stream::StreamExt,
};
use pin_utils::pin_mut;
use std::error::Error;

fn main() {
    let future03 = async {
        let url = "http://httpbin.org/ip".parse()?;

        let client = hyper::Client::new();
        let res = await!(client.get(url).compat())?;
        println!("{}", res.status());

        let body = res.into_body().compat();
        pin_mut!(body);

        let mut stdout = tokio::io::stdout().compat();
        while let Some(Ok(chunk)) = await!(body.next()) {
            await!(stdout.write_all(&chunk))?;
        }

        Ok(())
    };

    tokio::run(future03.map_err(|e: Box<dyn Error>| panic!("{}", e)).boxed().compat())
}

Special thanks goes to the main authors of the compatibility layer:

Futures 0.3.0-alpha.4

2018-09-02T00:00:00+00:00

0.3.0-alpha.4 requires a recent nightly (2018-08-31 or newer):

$ rustup update

`futures-test`

With futures 0.3.0-alpha.4 we’re releasing a new independent crate called futures-test (under the name futures-test-preview on crates.io). The crate contains various testing utilities that were previously only used internally in the futures crate. We polished them and made them more ergonomic to use and are now releasing them as a public standalone crate. What’s a little different compared to the other crates that comprise futures-rs is that this one is meant to be used as a dev dependency when implementing custom futures. So, unlike the other crates, its content is not reexported from the futures facade.

Testing without `futures-test`

Before getting into the details of futures-test, it’s useful to go over a basic technique for testing asynchronous code that may not be known to everyone.

A lot of the time you don’t care about all the details of how some async operation executes, just that it performs a certain task. In these cases you can simply use an async block run via futures::executor::block_on and have all your assertions in there. Here’s a quick example:

async fn some_operation<F: FnMut(usize)>(callback: F) -> io::Result<usize> {
    await!(async { /* Some IO stuff */ });
    callback(5);
    await!(async { /* Some more IO */ });
    callback(6);
    await!(async { /* ... */ });
    return 7;
}

#[test]
fn test_some_operation() {
    futures::executor::block_on(async {
        let mut values = vec![];
        let result = await!(some_operation(|value| values.push(value)));
        assert_eq!(result, Ok(7);
        assert_eq!(values, vec![5, 6]);
    })
}

Test `Context`s

While you can get a long way by just defining and running async tests via block_on, sometimes you need to be able to inspect what is happening more precisely. E.g. when testing FutureExt::fuse we want to check that after it completes, poll correctly returns Poll::Pending on subsequent calls. However, to call poll we need to somehow acquire a task::Context. futures-test makes this easy. It provides you with all sorts of useful contexts depending on what you are testing. You have two choices to make when you want to create a context:

What should happen when a future calls cx.waker().wake()?
What should happen when a future calls cx.spawner().spawn()?

For both of these there are two very basic implementations provided: panicking and noop (short for “no operation”). Sometimes this is all you need. For example when testing FutureExt::fuse we use futures_test::task::panic_context which gives a context that will panic on either operation:

#![feature(pin, arbitrary_self_types, futures_api)]

use futures::future::{self, FutureExt};
use futures_test::task::panic_context;

#[test]
fn fuse() {
    let mut future = future::ready::<i32>(2).fuse();
    let cx = &mut panic_context();
    assert!(future.poll_unpin(cx).is_ready());
    assert!(future.poll_unpin(cx).is_pending());
}

`WakeCounter`

One slightly more specialized Wake implementation is to count how many times a task has been woken. This has been used to check that AbortHandle correctly awakens a waiting task when it is aborted. You can see from this example that setting up a context with this implementation is slightly more work. We start with a basic panic_context, then swap out the waker with one provided by the WakeCounter:

#![feature(pin, arbitrary_self_types, futures_api)]

use futures::channel::oneshot;
use futures::future::{abortable, Aborted, FutureExt};
use futures::task::Poll;
use futures_test::task::{panic_context, WakeCounter};

#[test]
fn abortable_awakens() {
    let (_tx, a_rx) = oneshot::channel::<()>();
    let (mut abortable_rx, abort_handle) = abortable(a_rx);

    let wake_counter = WakeCounter::new();
    let mut cx = panic_context();
    let cx = &mut cx.with_waker(wake_counter.local_waker());

    assert_eq!(0, wake_counter.count());
    assert_eq!(Poll::Pending, abortable_rx.poll_unpin(cx));
    assert_eq!(0, wake_counter.count());
    abort_handle.abort();
    assert_eq!(1, wake_counter.count());
    assert_eq!(Poll::Ready(Err(Aborted)), abortable_rx.poll_unpin(cx));
}

There are some other implementations provided, take a look in the futures_test::task module docs for more info.

`FutureTestExt`

Along with the context utilities there are some more test specific combinators provided. These come on an additional FutureTestExt trait. For now there are three provided:

assert_unmoved: When writing custom futures that wrap other futures you want to be sure that they’re not accidentally moved after they were polled because that’s not allowed. This combinator can create a future that can detect such accidental moves.
pending_once: When testing, a lot of the time you will be working with instantly ready futures like future::ready(5) or async { 6 }. Frequently you need to have a future that actually acts like a future. This combinator will return Poll::Pending once and instantly wake its task to emulate a real async operation.
run_in_background: Sometimes you just want to run some future to completion and not care about the details. This will spin up a dedicated executor on a background thread to run it to completion for you.

`assert_stream_*!`

The final testing utility is trio of macros for working with streams. Together, these macros allow you to poll a stream one poll at a time and ensure it behaves exactly as you imagined:

#![feature(async_await, futures_api, pin)]
use futures::stream;
use futures_test::future::FutureTestExt;
use futures_test::{
    assert_stream_pending, assert_stream_next, assert_stream_done,
};
use pin_utils::pin_mut;

let mut stream = stream::once((async { 5 }).pending_once());
pin_mut!(stream);

assert_stream_pending!(stream);
assert_stream_next!(stream, 5);
assert_stream_done!(stream);

Improvements to come

The work on this crate is not done. We already have some ideas for other useful testing utilities. If you have existing utilities that you’re using in your project or ideas about what would be useful to you please let us know.

Further changes

A complete list of changes can be found in our changelog.

Fahrenheit

2018-08-17T00:00:00+00:00

Note: fahrenheit was previously called “toykio”, but it caused great confusion amongst reddit users.

In this blog post I’d like to present fahrenheit, a simple futures executor intended for learning about how executors with an event loop work. Fahrenheit only provides a very minimal feature set: An event loop and TCP streams and listeners. However, it turns out that due to the fact that futures are composable, this is enough to build complex clients and servers.

In the following, I’d like to give you a quick overview of fahrenheit’s components.

`AsyncTcpStream`

Fahrenheit defines the AsyncTcpStream type, a newtype wrapper around TcpStream from the standard library. Just like in std, the connect function opens a connection, and sets the socket to non-blocking mode. This means that the read() and write() methods will return immediately, either with data or an error. If there’s not enough data (for reads) or buffer space (for writes) available, a special kind of error - WouldBlock - is returned. We’ll discuss how to handle it in the next section.

`AsyncRead` & `AsyncWrite`

The AsyncRead and AsyncWrite traits are the foundation of all I/O futures. The AsyncReadExt and AsyncWriteExt extension methods (like read and write_all) are built on top of them. These traits provide a way to connect futures to an event loop while keeping them independent from any particular event loop implementation.

Let’s take a look at the AsyncRead implementation for AsyncTcpStream:

impl AsyncRead for AsyncTcpStream {
    fn poll_read(&mut self, cx: &mut Context, buf: &mut [u8]) -> Poll<Result<usize, Error>> {
        match self.0.read(buf) {
            Ok(len) => Poll::Ready(Ok(len)),
            Err(ref err) if err.kind() == std::io::ErrorKind::WouldBlock => {
                // Get TcpStream file descriptor
                let fd = self.0.as_raw_fd();
                let waker = cx.waker();

                REACTOR.with(|reactor| reactor.add_read_interest(fd, waker.clone()));

                Poll::Pending
            }
            Err(err) => panic!("error {:?}", err),
        }
    }
}

It tries to perform a read on the underlying TcpStream and if the read succeeds, the slice will be filled with data. If however the read fails with a WouldBlock error, then it registers the current task’s waker so that it gets awoken when the data becomes available. More about this in the next section.

The AsyncWrite implementation does something analogous for writes.

Event loop

The EventLoop (often also called “reactor”) is the core of the executor. It is defined like this:

struct InnerEventLoop {
    read: RefCell<BTreeMap<RawFd, Waker>>,
    write: RefCell<BTreeMap<RawFd, Waker>>,
    counter: Cell<usize>,
    wait_queue: RefCell<BTreeMap<TaskId, Task>>,
    run_queue: RefCell<VecDeque<Wakeup>>,
}

read and write are BTreeMaps which map file descriptors to wakers.
wait_queue stores blocked tasks which wait for an event
and run_queue stores wake-up notifications.

The event loop provides methods to register (and remove) interest in read and write events. Let’s look at what add_read_interest does:

fn add_read_interest(&self, fd: RawFd, waker: Waker) {
    if !self.read.borrow().contains_key(&fd) {
        self.read.borrow_mut().insert(fd, waker);
    }
}

But it just inserts fd and waker into the read map! So where does all the magic happen? In the main loop. The event loop is called a loop for a reason. Let’s take a look:

loop {
    // Event loop iteration timeout. If no descriptor is ready we continue iterating
    let mut tv: timeval = timeval {
        tv_sec: 1,
        tv_usec: 0,
    };

    // Initialize fd_sets (file descriptor sets)
    let mut read_fds: fd_set = unsafe { std::mem::zeroed() };
    let mut write_fds: fd_set = unsafe { std::mem::zeroed() };

    unsafe { FD_ZERO(&mut read_fds) };
    unsafe { FD_ZERO(&mut write_fds) };

Woah, woah, a lot of unsafe here! Well, don’t worry, it’s just how C FFI works. We need to initialize some C structures, a timeout and fd_sets. They will be passed to the select(2) function later.

    // Add read interests to read fd_sets
    for fd in self.read.borrow().keys() {
        unsafe { FD_SET(*fd, &mut read_fds as *mut fd_set) };
        nfds = std::cmp::max(nfds, fd + 1);
    }

    // Add write interests to write fd_sets
    for fd in self.write.borrow().keys() {
        unsafe { FD_SET(*fd, &mut write_fds as *mut fd_set) };
        nfds = std::cmp::max(nfds, fd + 1);
    }

Here we add file descriptors from our read and write maps from before to the fd_sets.

    // `select` will block until some event happens on the fds or the timeout triggers
    let rv = unsafe {
        select(
            nfds,
            &mut read_fds,
            &mut write_fds,
            std::ptr::null_mut(),
            &mut tv,
        )
    };

    // Don't care for errors
    if rv == -1 {
	    panic!("select()");
    } else if rv == 0 {
	    debug!("timeout");
    } else {
	    debug!("data available on {} fds", rv);
    }

Finally we call select with arguments we prepared earlier. select() accepts 3 fd_sets (we ignore the third for this example) and a timeout and returns something non-zero in case one (or more) of the file descriptors in the sets are ready. We should then go and find which one it was!

    // Check which fd it was and put appropriate future on run queue
    for (fd, waker) in self.read.borrow().iter() {
        let is_set = unsafe { FD_ISSET(*fd, &mut read_fds as *mut fd_set) };
        if is_set {
            waker.wake();
        }
    }

    // Same for write
    for (fd, waker) in self.write.borrow().iter() {
        let is_set = unsafe { FD_ISSET(*fd, &mut write_fds as *mut fd_set) };
        if is_set {
            waker.wake();
        }
    }

We go through our maps again and check if they’re set (ready) in fd_sets. When they are, we call wake on their associated wakers, which in turn puts Wakeup events on the ready-to-run queue.

    let mut tasks_done = Vec::new();

    // Now pop tasks from the run queue and poll them
    while let Some(wakeup) = self.run_queue.borrow_mut().pop_front() {
        let mut handle = self.handle();

        if let Some(ref mut task) = self.wait_queue.borrow_mut().get_mut(&wakeup.index) {
            // If a task returns `Poll::Ready`, we're done with it
            if task.poll(wakeup.waker, &mut handle).is_ready() {
                tasks_done.push(wakeup.index);
            }
        }
    }

    // Remove completed tasks
    for idx in tasks_done {
	    self.wait_queue.borrow_mut().remove(&idx);
    }

    // Stop the loop if there are no more tasks in the `wait_queue`
    if self.wait_queue.borrow().is_empty() {
	    return;
    }

We drain the run_queue, retrieve the task index on the wait_queue and poll the tasks. Ready (done) tasks are removed from the wait_queue.

The life of the future

In this section, I’d like to recap how a future (let’s take Read for example) is executed on the event loop:

first it’s created using read() method on AsyncTcpStream. This method is implemented for all types that implement AsyncRead trait.
then it’s spawned on the executor, using either run() or spawn() method
the executor calls poll() method on the future. poll implementation for Read calls poll_read() on the AsyncTcpStream, which registers its interest in readable events
when an event occurs, future is polled again. This cycle is repeated until future is ready.

Thank you

Thanks to all futures team, and specially @aturon for encouragement and @MajorBreakfast for edits.

That’s all for today! You can find fahrenheit on github and crates.io. Happy hacking!

Futures 0.3.0-alpha.3

2018-08-15T00:00:00+00:00

0.3.0-alpha.3 requires a recent nightly (2018-08-13 or newer):

$ rustup update

Compatibility layer

A compatibility layer between 0.3 and 0.1 was developed. It is now possible to convert an 0.3 future into an 0.1 future and vice versa. Similar conversions for streams and sinks are also supported. Additionally, it is now possible to run 0.3 futures and async functions on Tokio’s executor. We have a dedicated blog post coming up that explains this in more detail.

Spawning

`Spawn` trait

The std::task::Executor trait is now called std::task::Spawn. We chose to rename the trait to Spawn because the functionality exposed by this trait is not the ability to execute, but the ability to spawn.

As a consequence of this renaming, some functions on Context were adjusted:

impl<'a> Context<'a> {
    pub fn new(local_waker: &'a LocalWaker, spawner: &'a mut dyn Spawn) -> Context<'a> { ... }

    pub fn spawner(&mut self) -> &mut dyn Spawn { ... }

    pub fn with_spawner<'b, Sp: Spawn>(&'b mut self, spawner: &'b mut Sp) -> Context<'b> { ... }

    // ...
}

Spawn macros

This release of futures-rs introduces the spawn! and spawn_with_handle! macros for use in async functions, closures and blocks. Both macros let you spawn a task that polls the given future to completion. The task is spawned via the current context’s spawner.

spawn! lets you spawn futures with Output = ():

#![feature(async_await, await_macro, futures_api)]
use futures::spawn;

spawn!(future).unwrap();

spawn_with_handle! lets you spawn futures with any Output: Send type and returns a future that resolves to the output of the spawned future:

use futures::spawn_with_handle;

let join_handle = spawn_with_handle!(future).unwrap();
let output = await!(join_handle);

spawn! is slightly cheaper than spawn_with_handle! since it doesn’t need to provide a way to get the output.

Should spawning fail, a SpawnError error is returned which contains information about why it failed.

So what futures can be spawned? There are some requirements: The future needs to be Send and 'static and the output needs to be Send. Here’s why:

The 'static lifetime means that the future cannot contain any non-static references. This is required so that the spawned task can be run independently to the task that spawns it.
The Send bounds mean that the future and its output can be sent to other threads. This is required so that multithreaded executors can run the spawned task on any thread they choose.

Note: You can still execute non-Send, non-'static futures concurrently via the join! macro.

Spawn methods

For use outside of async functions, there’s also something new: The SpawnExt trait. It provides the methods spawn and spawn_with_handle:

use futures::task::SpawnExt;

// spawn()
spawner.spawn(future).unwrap();

// spawn_with_handle()
let join_handle = spawner.spawn_with_handle(future).unwrap();

Inside poll, you can use these methods by accessing the context’s spawner via Context::spawner:

fn poll(self: PinMut<Self>, cx: &mut task::Context) -> Poll<T> {
    cx.spawner().spawn(future).unwrap();
}

The plan: `spawn` with `dyn Future`

We plan to eventually replace SpawnExt::spawn with Spawn::spawn. This method will live directly in the Spawn trait defined in libcore. It’s signature will look like this:

fn spawn(
    &mut self,
    future: dyn Future<Output = ()> + Send + 'static,
) -> Result<(), SpawnError>

This method can’t be implemented just yet due to compiler limitations:

Arbitrary self types are currently not object-safe: Future::poll uses PinMut as arbitrary self type. This unfortunately makes it impossible to create trait objects for Future. Therefore, Box<dyn Future> and &dyn Future are currently not supported.
Unsized rvalues are not yet available: Trait objects (dyn Trait) are !Sized because the size of a trait object depends on the size of the concrete type behind it and can therefore only be known at runtime. Currently we often use Box<dyn Trait> which stores the trait object on the heap to circumvent that. The unsized rvalues feature will make it possible to store !Sized types on the stack.

To circumvent these limitations, we currently have FutureObj and Spawn::spawn_obj. These are, however, just a temporary solution until we can introduce the planned spawn method. The planned spawn method will make it possible to have one fewer heap allocation because the future is kept on the stack for as long as possible.

`pin-utils` crate

The pin macros now live in their own crate, the pin-utils crate. The macros are general purpose and can be used in any project that uses the pinning API.

The crate includes the macros for (un)pin projections unsafe_pinned! and unsafe_unpinned! and the macro for stack pinning pin_mut!:

let foo = Foo { ... };
pin_mut!(foo); // PinMut<Foo>

Version 0.1.0-alpha.1 is now available on crates.io and the API docs are available on docs.rs.

Further changes

A complete list of changes can be found in our changelog.

A big thanks goes to every one who contributed to this release! This includes @tinaun, @Nemo157, @cramertj and @MajorBreakfast and our new contributors @gbonik and @dbcfd!

Futures 0.3.0-alpha.2

2018-07-30T00:00:00+00:00

0.3.0-alpha.2 requires a recent nightly (2018-07-29 or newer):

$ rustup update

What’s new?

Refined definition for “task”

The documentation for the Executor trait was updated. It now defines a “task executor” like this:

Futures are polled until completion by tasks, a kind of lightweight “thread”. A task executor is responsible for the creation of these tasks and the coordination of their execution on real operating system threads. In particular, whenever a task signals that it can make further progress via a wake-up notification, it is the responsibility of the task executor to put the task into a queue to continue executing it, i.e. polling the future in it, later.

The notable difference to before is that the term “task” no longer refers to the future that the task runs. Instead, from now on, the term “task” only refers to the concept of a lightweight thread, a kind of thread that doesn’t directly correspond to a thread provided by the operating system. This concept is sometimes also referred to as “green thread” - We, however, do not use either of these terms because they tend to cause confusion with real operating system threads. Instead, we call this concept a “task” and the thing that runs them a “task executor” or often just “executor”.

`Try` impl for `Poll`

Poll<Result<T, E>> now implements the Try trait. This means you can now do this:

fn poll(mut self: PinMut<Self>, cx: &mut task::Context) -> Poll<Result<T, E>> {
    let poll: Poll<Result<T, E>> = self.future().poll();
    let poll: Poll<T> = poll?; // Returns early if there's an error
    let ok_value: T = ready!(poll); // Returns early if the value is pending

    // Or short:
    let ok_value = ready!(self.future().poll()?);
}

Additionally Try was also implemented for Poll<Option<Result<T, E>>> to make the ?-operator useful in Stream::poll implentations:

fn poll_next(mut self: PinMut<Self>, cx: &mut task::Context) -> Poll<Option<Result<T, E>>> {
    let poll: Poll<Option<Result<T, E>>> = self.stream().poll_next();
    let poll: Poll<Option<T>> = poll?; // Returns early if there's an error
    let ok_item: Option<T> = ready!(poll); // Returns early if the value is pending

    // Or short:
    let ok_item = ready!(self.stream().poll_next()?);
}

You can find additional details about this change in the pull request.

Changes to futures-rs

We revived the changelog!

No major changes happened to the futures crate since the last release. There were, however, numerous little changes. Take a look at the changelog to get an overview.

What are we currently working on?

Compatiblity layer for futures 0.1

We are currently working on an officially supported compatibility layer between 0.1 and 0.3. This shim will make it possible to mix 0.1 futures with those from the 0.3 branch. The plan is to make it possible to use crates from the existing futures 0.1 ecosystem together with async functions and futures from the 0.3 crate. Additionally, this compatibility layer will also make it possible to gradually migrate exisiting applications and libraries to 0.3. If you’re curious, take a look the compatibility layer PR and the issue in the net-wg repository.

Futures 0.3.0-alpha.1

2018-07-19T17:00:00+00:00

Welcome to the inaugural post of the new futures-rs blog!

After several months of work, we’re happy to announce an alpha release of the new edition of futures-rs, version 0.3. The immediate goal of this work is to support async/await notation (with borrowing) in Rust itself, which has entailed significant changes to the futures crate.

TL;DR

The 0.3.0-alpha.1 release is available in the new futures-preview family of crates.
It requires a nightly compiler, and works with rustc’s new support for async/await notation.
Tokio and Hyper support is not yet available for 0.3.0-alpha.1, but that’s the next area of work.
Neither the futures crate nor async/await notation will be stabilized for the initial release of the Rust 2018 Edition.
Futures 0.1 continues to be maintained and is the primary way to write production async code today. We plan to continue maintaining 0.1 for some time after 0.3.0 is released, as well.

In short, this alpha release sets the stage for kicking off integration and documentation, but is not yet widely usable.

The futures-preview crate

We’re publishing a 0.3.0-alpha.1 release of futures today, but rather than doing it within the existing futures crate, we’re publishing within the new futures-preview family of crates. This allows us to more clearly signal that this version is a work in progress, and to keep crates.io and docs.rs pages pointing to the 0.1 series.

In other words:

Futures 0.1 is the production-ready way to use futures today, and is actively maintained.
Ongoing work for async/await will live in the futures-preview crates until it is ready to be promoted to 0.3.0.

What is futures 0.3?

Async/await

With futures 0.3 alpha, it is possible to use the new async/await notation that recently landed in nightly:

cargo-features = ["edition"]

[package]
edition = "2018"

#![feature(async_await, await_macro)]

use futures::executor::block_on;
use futures::future::{self, FutureExt};

fn main() {
  block_on(async {
    let fut = future::lazy(|_| vec![0, 1, 2, 3]);
    let shared1 = fut.shared();
    let shared2 = shared1.clone();

    assert_eq!(await!(shared1).len(), 4);
    assert_eq!(await!(shared2).len(), 4);
  })
}

The notation works as described in the corresponding RFC; more documentation for it is forthcoming. There are several open questions on this notation, but these are largely on hold until we have achieved integration with Tokio and Hyper, at which point more serious code can be written against the library.

While we had originally hoped to ship async/await notation as part of Rust 2018, there’s no chance at this point of having adequate feedback and confidence to do so in time. However, the Networking WG plans to continue pushing hard in this space up to and after the Rust 2018 launch, to provide the strongest story on nightly Rust that we can, and to get us on a path for stabilization.

Ecosystem integration

As of this writing, futures 0.3.0-alpha.1 is not integrated in any way with other libraries like Tokio or Hyper. Achieving such integration is a crucial part of getting real experience with the design and ultimately heading toward stabilization. The Futures Team intends to work closely with the Tokio Team to determine the best way forward here — either a shim crate for compatibility with futures 0.1, or direct integration via a feature-flag.

This integration is our main next area of work, and we’d love to have help! If you’re interested in getting involved, drop by the #wg-net channel on Discord.

Crate structure and relation to `std`

To support the built-in async/await notation, the Future trait and associated machinery had to move into the standard library. As such, the futures 0.3.0-alpha.1 release just re-exports those definitions, and then adds a host of useful extras, including streams, sinks, I/O traits, and the various combinators.

The overall organization of the crate is similar to the 0.2 release:

futures-core-preview: This crate re-exports the futures-related items from std, and defines Stream, TryFuture and TryStream.
futures-executor-preview: This crate provides basic executors (a thread pool and a thread-local executor).
futures-io-preview: This crate defines core traits for I/O.
futures-sink-preview: This crate defines the core Sink trait.
futures-channel-preview: This crate defines basic Sync channels with end points implementing Future, Stream or Sink.
futures-util-preview: This crate defines a collection of extension traits with useful adapters (aka combinators).
futures-preview: This crate is a “facade” combining and re-exporting all of the above crates for convenience.

You can read more about the rationale for the breakdown into multiple crates in the 0.2 notes.

Changes from 0.2

Note: Changes between 0.1 and 0.2 are outlined in the 0.2 notes.

Aside from some minor structural changes, there are two major changes from the 0.2 release:

Using pinned types to support borrowing within async blocks.
Factoring out the Error type into separate traits (TryFuture and TryStream).

Both of these changes are described and motivated in detail in the companion RFC.

The RFC, however, covers only the core Future trait and executor system. The bulk of the work in 0.3.0-alpha.1 has been pushing through these ideas through the rest of the crates, adapting the combinators and other traits to fit with pinning and the ability to borrow across yield points. For example, the revised AsyncReadExt trait now provides a read method that operates on borrowed data, just like in the Read trait in std:

trait AsyncReadExt: AsyncRead {
    /// Tries to read some bytes directly into the given `buf` in asynchronous
    /// manner, returning a future type.
    ///
    /// The returned future will resolve to the number of bytes read once the read
    /// operation is completed.
    fn read<'a>(&'a mut self, buf: &'a mut [u8]) -> Read<'a, Self> {
        Read::new(self, buf)
    }
}

In contrast, the 0.1-style API must operate on owned buffers, which are threaded through the futures code. You can learn more about how this all works in this blog post on borrowing and async/await.

The road ahead

While futures 0.3.0-alpha.1 is a major milestone, we still have a ton of work to do! The immediate next steps include:

Work out the integration story for Tokio, Hyper, and other important crates.
Write documentation for async/await notation.
Flesh out the migration path from futures 0.1 code.
Generally gain experience and feedback using this new design and async/await notation, so that we can get on track for stabilization.

More generally, the futures team plans on communicating via this blog much more regularly about progress and ways to get involved. And now that the alpha is at the door, the broader Networking WG is hoping to reboot, bring in a lot more people, and start focusing on ecosystem concerns.

As always, if you’re interested in getting involved in this space, reach out on #wg-net on Discord.

Thank you!

This alpha release is the result of work by a growing set of people contributing to the futures crate (some of whom have joined the team officially):

New Website

2018-07-19T16:00:00+00:00

Welcome!

Futures-rs now has a website and a logo. We hope you like it! This site contains our brand new blog with which we plan to keep you up-to-date about the project.

Enjoy reading!

BTW The logo was inspired by a movie, we’ll leave it up to the reader to guess which one :)

Futures-rs

Compatibility Layer

Futures 0.1 Compatibility Layer

Cargo.toml

Async functions on 0.1 executors

0.1 futures in async functions

Streams and Sinks

Spawning

Async IO

Conclusion

Futures 0.3.0-alpha.4

futures-test

Testing without futures-test

Test Contexts

WakeCounter

FutureTestExt

assert_stream_*!

Improvements to come

Further changes

Fahrenheit

AsyncTcpStream

AsyncRead & AsyncWrite

Event loop

The life of the future

Thank you

Futures 0.3.0-alpha.3

Compatibility layer

Spawning

Spawn trait

Spawn macros

Spawn methods

The plan: spawn with dyn Future

pin-utils crate

Further changes

Futures 0.3.0-alpha.2

What’s new?

Refined definition for “task”

Try impl for Poll

Changes to futures-rs

What are we currently working on?

Compatiblity layer for futures 0.1

Futures 0.3.0-alpha.1

TL;DR

The futures-preview crate

What is futures 0.3?

Async/await

Ecosystem integration

Crate structure and relation to std

Changes from 0.2

The road ahead

Thank you!

New Website

Welcome!

`Cargo.toml`

`futures-test`

Testing without `futures-test`

Test `Context`s

`WakeCounter`

`FutureTestExt`

`assert_stream_*!`

`AsyncTcpStream`

`AsyncRead` & `AsyncWrite`

`Spawn` trait

The plan: `spawn` with `dyn Future`

`pin-utils` crate

`Try` impl for `Poll`

Crate structure and relation to `std`