Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Summary

Add a f16b type to core::num for the 16-bit Brain Floating Point format.

Similar to the f32 type, a f16b has a similar layout to the f32, keeping the same 1-bit sign and 8-bit exponent layout but only storing the top 7 fraction bits. Despite these similarities the f16b is not an IEEE-754 floating point format.

The primary role of the f16b is representing bfloat16 values in memory, in APIs, in FFI signatures where the target ABI supports bfloat16, and as an element type for SIMD and matrix operations. It is expected to be uncommon for a f16b to be used as a general-purpose arithmetic type in the same way as f32 or f64.

Scalar operations, casts and such are not proposed as part of this RFC. These are, however, considered a future possibility, though initially the f16b type will only be meaningfully usable by SIMD intrinsics.

Acknowledgements

Thanks to Folkert de Vries, Trevor Gross, Tyler Mandry, Ralf Jung and Sayantan Chakraborty for providing early feedback on a draft of this RFC.

Motivation

Increasingly the f16b type is found in hardware, across architectures, to support AI and machine learning workloads. Current and emerging instructions, for example x86_64’s VADDBF16, prove tricky to implement in stdarch without a f16b type. Rust today cannot name this type directly, so FFI signatures, SIMD element types, and codegen route around the gap with u16 wrapper types using ad-hoc lowering rules, such as the half crate or core::arch::x86::bf16 (see Current Rust support). A driving motivation for a f16b type is to close that interop gap, in a similar spirit to RFC 3892, which proposes core::num::Complex to match C99 _Complex at the FFI boundary.

Beyond raw hardware support, f16b is already supported in the surrounding ecosystem. A f16b type is provided by stdfloat in C++23 in C++23 as std::bfloat16_t, as a vendor extension in GCC and Clang with __bf16, and as a first-class scalar type in LLVM IR (bfloat).

The essence of this RFC is to make SIMD intrinsics, in a similar fashion to the following, possible to define;

// Example Neon intrinsic that would be added to `std::arch::aarch64`
fn vbfdot_lane_f32<const LANE: i32>(r: float32x2_t, a: bfloat16x4_t, b: bfloat16x4_t) -> float32x2_t;

Guide-level explanation

  • f16b would be defined in core::num and re-exported in std::num, for example use core::num::f16b or use std::num::f16b.
  • f16b will be Rust’s only language-provided brain floating-point type. The existing core::arch::x86::bf16 type will instead use core::num::f16b and be re-exported from core::arch as the bf16 alias.
  • core::arch::x86::bf16’s should use the f16b.
  • There should only be one representation of a brain floating point type in Rust, namely the f16b.
  • There would be no scalar arithmetic or casts defined on the f16b see rationale.

Defining vector types could be done as follows;

pub struct bfloat16x1_t(1 x f16b);
pub struct bfloat16x2_t(2 x f16b);
// etc...

Naming the 16-bit Brain floating-point f16b

The proposed name for the Brain Floating Point type is f16b, see rationale for alternatives.

Reference-level explanation

The core crate would define a struct along the lines of;

#[lang = "f16b"]
#[repr(C, align(2))]
#[derive(Copy, Clone)]
struct f16b { _do_not_look_at_fields: () }

// Ensure that the size of this struct is `2`
const _: () = assert!(size_of::<f16b>() == 2);

Alternatively, some other compiler magic could map the struct to LLVM’s bfloat type, the example above illustrates this through #[lang = "b16b"]. The struct has no constructor however it is proposed that it can be instantiated through the following methods;

impl f16b {
    pub const fn from_bits(bits: u16) -> f16b;
    pub const fn to_bits(self) -> u16;
}

Constructing a f16b can initially only be done through either f16b::from_bits(...) or vendor intrinsics.

Implemented Traits

This list is not exhaustive, but is intended to be the bare minimum required to make the type reasonably user-friendly.

  • Default, defaulting to 0x0.
  • Copy
  • Clone
  • Debug
  • UpperExp
  • LowerExp
  • PartialEq
  • PartialOrd
  • From<f16b> for f32

The traits From<f16b> for f32, PartialEq and PartialOrd despite requiring a conversion can reasonably be implemented with a lossless conversion from f16b to f32, doing the comparison and then returning the result.

Display and FromStr are deferred, possible concerns and implementations are discussed as future possibilities. However if printing a f16b is required, it would be possible to do so by converting an f16b to an f32 via a vendor intrinsic and then printing the resultant f32. Note that this may not be the expected value due to how narrowing is performed.

ABI

This RFC does not propose a new platform ABI for f16b.

On AArch64, the AAPCS64 specification defines __bf16 as the Brain floating-point format with byte size 2 and natural alignment 2. On x86_64, f16b ABI compatibility should likewise follow the platform psABI and toolchain support for __bf16, which is stated as having “the same size, alignment, parameter passing and return rules as _Float16”.

Within Rust, f16b should usually have a size 2 and alignment 2, all bit patterns are valid values, and its memory representation is the 16-bit Brain floating-point encoding.

For extern "C" and other foreign ABIs, Rust should not independently choose how f16b is passed or returned. Instead, on targets where the platform defines and toolchains implement an ABI for the corresponding C/C++ __bf16 type, Rust’s f16b should be ABI-compatible with that type. On platforms where the ABI is undefined, see Unresolved Questions.

Drawbacks

  • As with RFC 3453, support for this type has a fairly specific use case as opposed to general purpose programming. f16b is important for machine learning, numerical code, storage, conversion, and target-specific SIMD or matrix operations, but most Rust code will not use it directly.

  • The initial implementation of the f16b type, as specified in this RFC, will be quite lean. This means that some of the more challenging implementation questions are being deferred. Where possible, however, these have been explored in some depth in the future possibilities section, especially around portable scalar arithmetic and native vs emulated conversions.

Rationale and alternatives

Primitive type versus core::num

Initially the proposed f16b type will not be a primitive type living in the global namespace. There have been difficult challenges with the f16 type which almost certainly would crop up when implementing a primitive f16b type. Thus for the time being the f16b would live in core::num::f16b and could, at a later date, be included in the prelude and its functionality extended.

Having only the path core::num::f16b signals that this is a specialist interchange type rather than a general-purpose arithmetic type, although a f16b arguably is one to some extent, and helps prevent misuse of the type.

Neither of these opinions is to say that the f16b could not be implemented as a primitive and be added to the prelude at a later point in time, nor do these proposed decisions box a f16b into a particular path.

Omitting Scalar Arithmetic

Omitting scalar arithmetic keeps the initial implementation of f16b simpler and avoids potentially complex or inefficient implementations as well as the minefield of conversion issues discussed in the conversions section.

Portable scalar arithmetic could prove useful if predictable behaviour is expected. Baking this into the language could lessen the possible footguns from the disparate and somewhat confusing hardware support available. Thus inefficiencies in using the type for scalar arithmetic portably could be an acceptable tradeoff favouring predictable behaviour. Some of these issues are discussed in this RFC here. Target-dependent instructions, which are not portable, would remain available through core::arch. Some portability issues are presently observable in Clang when converting from an f32 to f16b which are discussed in Prior Art c++ compiler support.

If portability was not a concern, target-dependent behaviour for scalar arithmetic could plausibly produce faster code. For instance the aforementioned VADDBF16 instruction flushes subnormals to zero, which does not follow IEEE-754 and is not portable. However, the instruction could ostensibly be more performant than converting to f32 doing the operation and narrowing back to f16b, being the portable alternative.

Naming f16b

Other such proposed names have been; bf16, f16b, fb16, bfloat16, and __bf16.

This RFC prefers f16b.

The name f16b is in keeping with the Rust naming convention for floating point types which all start with an f prefix.

Prior Art

C++ compiler support

The C++23 standard defines a std::bfloat16_t type in <stdfloat> as part of the extended floating-point types introduced by P1467. This was motivated by the prevalence of availability of bf16 on hardware, library support through class wrappers being cumbersome to use and a desire for more portable code. (for the section on adding extended floating point types see here)

GCC 13 implements the C++23 P1467 extended floating-point types, including std::bfloat16_t and the bf16 literal suffix. Clang’s support for extended floating-point types, at the time of writing, is more limited and varies by feature; see the Clang C++ status page (then search the page for P1467). Both GCC and Clang nevertheless support bfloat16 through the __bf16 type.

Furthermore the standard states:

An implementation may also provide additional types that represent floating-point values and define them (and cv-qualified versions thereof) to be extended floating-point types. The standard and extended floating-point types are collectively termed floating-point types. […] Except as specified in [basic.extended.fp], the object and value representations and accuracy of operations of floating-point types is implementation-defined.

As such, during the time of writing, depending on whether compilation for x86_64 uses emulated or hardware-specific instructions on the primitive std::bfloat16_t type subnormals will be handled differently. With native hardware support on a bf16 Clang will compile code to use VCVTNEPS2BF16 which flushes subnormals to zero, this is discussed in further detail here

Current Rust support

Rust already has an experimental core::arch::x86::bf16 / core::arch::x86_64::bf16 type for AVX-512 BF16 intrinsics. This type is nightly-only, x86-specific, and documented as “the BFloat16 type used in AVX-512 intrinsics” rather than as a general-purpose type. Its API is limited to raw from_bits and to_bits operations, and the stdarch tracking issue describes it as “just a wrapper around u16” intended to make intrinsic signatures more readable.

There is also a code generation reason not to treat this as merely an x86 stdarch convention. Some LLVM intrinsics use LLVM’s first-class bf16 / bf16xN types in their signatures, but Rust cannot currently name those types directly. As a result, Rust intrinsic signatures have to describe bfloat16 operands using integer-like types such as i16, u16, or vectors of those types, and rustc then needs special-case lowering rules to connect those signatures to LLVM intrinsics that actually expect bf16 or bf16xN. This is an ad hoc workaround: it applies only to selected intrinsics and does not give users or libraries a general Rust type for the format.

PR #140763 is an example of the current workaround: it adds temporary codegen bypasses so Rust signatures using i16 / i16xN can call LLVM intrinsics whose real signatures use bf16 / bf16xN. The need for that bypass illustrates the gap this RFC is intended to close.

A f16b would eventually give Rust one common type for scalar values, memory layout, conversions, FFI where the target ABI supports bfloat16, and target-specific SIMD or matrix intrinsics.

This RFC has been split, and expanded upon, from an RFC proposing bf16, f64f64 and f80 types. This RFC focuses specifically on f16b and addresses feedback about naming the type, floating-point semantics, especially the behaviour of scalar arithmetic and conversions to and from other floating-point types.

Unresolved Questions

ABI on targets without a defined f16b calling convention

The reference ABI section above focuses on platforms where a specified ABI exists. That leaves open what Rust should do on targets whose ABI does not (yet) specify how f16b is passed and returned. f16 faces similar problems and several questions carry over:

  • Should f16b be rejected in extern "C" signatures on such targets, via an error, or should Rust pick a provisional lowering?
  • If provisional, then should Rust follow Clang’s behaviour on that target? However, Clang and GCC have historically differed on how _Float16 is passed on some targets so, “match the C toolchain” is not a single answer and a provisional choice may risk a silent ABI break if the platform standardises something different.
  • Most issues stem from f16b being available on a target without the required FP/SIMD registers being available. This problem is not exclusive to f16b. f32 32-bit targets without SSE have unsound floating point behaviour. As mentioned f16 has similar issues without SSE registers on a 32-bit x86 target.

Where it is not possible to fix these issues, a reasonable compromise is suggested in RFC-3514 for floating point semantics where compatibility is documented for the target in a proposed “support page”. Presently platform support information is documented in platform-support.md along with platform support errata.

Future possibilities

In short, scalar arithmetic, a literal suffix, portable f16b vector operations, conversions and constants. This RFC originally proposed f16b as a primitive type following IEEE-754 semantics for scalar arithmetic, rounding and subnormals. As such, this section is rather long as considerations and findings here seem worthwhile preserving but can be brought up at a later date.

Literal Suffix

let a = 0.0f16b;

Unifying f16b usages

For example, in stdarch the core::arch::x86_64::bf16 type should be transitioned away in favour of using the f16b proposed here. This approach should, where possible, be followed for other wrapper types that are aiming to create a f16b type.

f16b Scalar Operations

The following has been adapted from a previous version of this RFC which proposed a f16b primitive type following the IEEE-754 standard for floating point types for scalar operations (albeit with a non-standard binary format) and only diverging when using vendor intrinsics gated by a target feature.

Typically, at the time of writing, hardware does not provide instructions for f16b scalar arithmetic. Instead, scalar operations would typically follow widening to f32 then narrowing to f16b. Consider this example;

let a: f16b = 1.1f16b;
let b: f16b = 2.2f16b;
let c: f16b = 3.3f16b;

let d = a + b + c;

This could semantically equate to either of the following;

// If an `f16b` ADD exists
let d: f16b = a + b + c;

// Each intermediary step is converted to `f32` and narrowed to `f16b`
let d: f16b = (f32::from((f32::from(a) + f32::from(b)) as f16b) + f32::from(c)) as f16b;

The as f16b cast performs the narrowing under round-to-nearest, ties-to-even, as specified later. An instance where the precision is not enough to perform the operation, a suitable library call should be emitted, such as with the case of f16b’s FMA in LLVM where casting to an f64 does not have enough precision to perform a fused-multiply-add without double rounding becoming an issue.

Narrowing: with native conversion support

Some architectures, including AArch64, x86_64, and RISC-V, provide dedicated instructions for converting to and from f16b. These instructions are not interchangeable: their behaviour can differ in rounding mode selection, subnormal handling, NaN handling, and interaction with floating-point control registers.

The examples below cover AArch64, x86_64, and RISC-V, but the same considerations apply to any target with native f16b conversion instructions. The word “conforming” in this section is used to indicate following the semantics of RNE and not flushing subnormals to zero, following IEEE-754’s specification for handling floating point numbers.

x86_64

On x86_64 with AVX512_BF16, VCVTNEPS2BF16 performs RNE and does not respect or update MXCSR. This makes it independent of the dynamic MXCSR.RC rounding mode. For subnormals, however, the instruction flushes to zero. Thus, VCVTNEPS2BF16 under what is proposed in this RFC is not a conforming lowering for an f32 -> f16b narrowing operation that is required to preserve representable f16b subnormal results. If this behaviour is required for a program, one can use the relevant platform specific intrinsic. Thus, this instruction, if portable behaviour is desired, could not be used.

AArch64

On AArch64 with the bf16 architectural extension, narrowing from f32 to f16b can be performed by BFCVT. In the normal FPCR mode (FPCR.AH == 0), the BFCVT-family instructions use the rounding mode selected by FPCR.RMode, where RMode = 0b00 selects RNE. If FPCR.AH == 1, the BFCVT-family instructions use RNE regardless of RMode. Therefore, for an unconditional RNE specification, BFCVT is a conforming lowering only when the compilation/runtime model guarantees that the relevant execution observes RNE, for example because FPCR.RMode == 0b00 or because FPCR.AH == 1 gives BFCVT-family instructions RNE semantics regardless of RMode. Subnormal preservation is a separate requirement from the rounding mode. AArch64 BFCVT is a conforming lowering for this specification only when the execution environment both selects RNE for the conversion and does not enable a mode that flushes the relevant denormal/subnormal results to zero. In the normal FPCR-zero state, this means FPCR.RMode == 0b00 and FPCR.FZ == 0.

On Linux/arm64, a newly exec’d task starts with zeroed user FPSIMD state. The kernel’s arm64 FPSIMD code stores FPSR and FPCR, then the exec/reset path initializes this user state to zero. Consequently, at process entry after execve(), the user FPCR is zero: FPCR.RMode[23:22] == 0b00, which is Round to Nearest, i.e. nearest-even. This is an initial process state, not a guarantee that FPCR cannot later be changed by user code or libraries. See source.

Rust assumes FPCR remains in a conforming state across the lifetime of a process; code that mutates FPCR.RMode or FPCR.FZ may observe non-conforming f16b results, analogous to existing expectations about MXCSR on x86_64 or the effects the FPCR register has on f32. Thus BFCVT would be a portable instruction.

RISC-V

On RISC-V with Zfbfmin, narrowing from f32 to f16b is performed by FCVT.BF16.S, which is a standard narrowing floating-point-to-floating-point conversion that rounds according to the frm field (either the static rounding mode encoded in the instruction or the dynamic mode in the frm CSR). Setting frm to RNE (0b000) selects round-to-nearest, ties-to-even, making FCVT.BF16.S a conforming lowering under this specification when the execution environment selects RNE rounding for the conversion. Static encoding of the rounding mode in the instruction itself avoids dependence on the dynamic frm CSR and is the preferred lowering where the compiler can emit it directly.

Narrowing Comparison

The three architectures above each provide a hardware narrow from f32 to f16b, but each has its nuances:

ArchitectureInstructionRoundingSubnormal behaviourConforming as default lowering?
AArch64 (bf16 ext.)BFCVTPer FPCR.RMode when FPCR.AH == 0; RNE when FPCR.AH == 1Per FPCR.FZOnly when FPCR.RMode == 0b00 and FPCR.FZ == 0 (or FPCR.AH == 1 with FPCR.FZ == 0)
x86_64 (AVX512_BF16)VCVTNEPS2BF16RNE, independent of MXCSR.RCFlushes subnormal outputs to zero, ignores MXCSRNo, fails to preserve subnormals
RISC-V (Zfbfmin)FCVT.BF16.SPer instruction-encoded RM, or frm CSRPreservedYes, when RM/frm selects RNE

Narrowing: without native conversion support or where hardware does not provide a portable solution

Where no hardware instruction performs an RNE narrow from f32 to f16b or hardware does not provide a portable solution, the narrowing should be emulated. A straightforward RNE implementation inspects bit 15 of the f32 significand and the sticky bits below it to decide whether to round up, with the usual ties-to-even tiebreak; this is a handful of integer ops. This is the same kind of operation normally provided by soft-float conversion helpers such as __truncsfbf2.

Constants

f16b could expose the same style of constants as other primitive float types:

f16b::BITS
f16b::DIGITS          // approximately 2 decimal digits
f16b::EPSILON
f16b::INFINITY
f16b::MANTISSA_DIGITS // 8, including the implicit leading bit
f16b::MAX
f16b::MAX_10_EXP
f16b::MAX_EXP
f16b::MIN
f16b::MIN_10_EXP
f16b::MIN_EXP
f16b::MIN_POSITIVE
f16b::NAN
f16b::NEG_INFINITY
f16b::RADIX           // 2

Conversions

From Conversions

Conversions from f16b to a wider floating-point type should preserve the represented value by expanding the f16b bit pattern into the destination floating-point format.

At minimum:

impl From<f16b> for f64 { /* exact widening */ }

impl From<u8> for f16b { /* ... */ }
impl From<i8> for f16b { /* ... */ }

let x: f16b = 1.0f16b;
let y: f32 = f32::from(x);

println!("Size of f16b in bytes: {}", core::mem::size_of::<f16b>()); // 2

From<f16> for f16b and vice versa should not exist in either direction as neither direction is value preserving. f16 has 11 bits of precision while f16b has 8. A f16b’s range vastly exceeds that of an f16 and From impl’s are for lossless conversions. Ostensibly, the naming, including 16 makes them look similar when reading the type however how they use those 16 bits differs wildly.

Lossy Conversions

Lossy conversions to f16b could be available through casts, performing down casting using round-to-nearest, ties to even:

let x: f16b = 3.1415927f32 as f16b;
let y: f16b = 3.141592653589793f64 as f16b;

Display & FromStr

Display and FromStr for f16b should follow the same principles as Rust’s existing primitive floats: producing a decimal representation that parses back to the same f16b value via FromStr, preferably using the shortest such representation.

The maximum number of decimal digits needed to uniquely round-trip any f16b value is given by the standard formula ceil(1 + p × log10(2)) from IEEE-754, 2019, where p = 8 for f16b, yielding 4. In practice, shortest-round-trip formatters use fewer digits where the value permits, so most f16b values will display with 1-3 significant digits.

Special values follow the existing primitive-float conventions: NaN, inf, -inf. Debug should be consistent with Display for ordinary values; its exact format is not specified here.

As with f32 and f64, displayed values may differ from the source literal when the literal is not exactly representable in the format. This effect is more visible for f16b because of its lower precision.

Portable f16b vector operations

Specifying these operations is out of scope for this RFC. However, problems emerge even when two architectures provide an instruction with the same source-level shape, their handling of rounding, subnormals, and intermediate precision can differ. Arm’s BFDOT and x86_64’s VDPBF16PS both compute a f16b dot product accumulating into f32, but they do not produce bit-identical results.

Appendix

f16b hardware support

On AArch64, f16b is supported by Armv8.6-A and later implementations that include the bf16 architectural extension, including platforms based on Neoverse V1, Neoverse N2, and Neoverse V2. On x86_64, f16b is supported through AVX512_BF16 on platforms such as Intel Cooper Lake, Intel Sapphire Rapids and later platforms with AMX_BF16, and AMD Zen 4 and later platforms with AVX-512 BF16 support. On RISC-V, bf16 is supported through the ratified Zfbfmin, Zvfbfmin, and Zvfbfwma extensions, providing scalar conversion, vector conversion, and vector widening multiply-accumulate respectively.

f16b compared to other floating point types

Below is a brief introduction as to how floats are represented and a comparison between the representations of a f16b and f32.

Intro to Floating Point bit representation

Floating-point types represent real numbers approximately using a fixed number of bits. These bits are divided into three fields: a sign bit, which determines whether the value is positive or negative. Exponent bits, which determine the scale or range of the value and fraction bits, also commonly called mantissa or significand bits, which determine the precision of the value within that range. Below is the bit representation of an f32.

    31 30             23 22                                      0
  +---+-----------------+----------------------------------------+
  | S |     Exponent    |               Mantissa                 |
  +---+-----------------+----------------------------------------+
    1       8 bits                       23 bits

Trade-offs between representations

Increasing the number of exponent bits allows a format to represent much larger and much smaller values. Increasing the number of fraction bits allows it to represent values more precisely. Different floating-point formats therefore make different trade-offs between range, precision, storage size, and hardware efficiency.

f16b is a 16-bit floating-point format intended to preserve the dynamic range of f32 while using half the storage. It does this by using the same number of exponent bits as f32, but fewer mantissa bits. This makes f16b useful in machine learning and numerical workloads where range is often more important than precision, and where data movement, memory bandwidth, and hardware throughput matter.

Compared with IEEE-754 binary16 / f16, f16b trades precision for range:

data typesign bitsexponent bitsfraction bits
f161510
f16b187
f321823

That makes the essence of f16b feel more like a compact storage/compute companion to f32, rather than a smaller-range replacement for it. f16b will generate the bfloat type in LLVM IR. It is also equivalent to C++ std::bfloat16_t, and GCC’s and Clang’s __bf16 vendor extension.

The f16b layout is conceptually similar to taking the high 16 bits of an IEEE-754 binary32 / f32 value:

  f32, 32 bits:

    31 30             23 22                                      0
  +---+-----------------+----------------------------------------+
  | S |     Exponent    |               Mantissa                 |
  +---+-----------------+----------------------------------------+
    1           8 bits                          23 bits


  f16b, 16 bits:

    15 14              7 6              0
  +---+-----------------+---------------+
  | S | Exponent        |    Mantissa   |
  +---+-----------------+---------------+
    1           8 bits                 7 bits


  Relationship:

  f32:
  +---+-----------------+---------------+--------------------------+
  | S | E E E E E E E E | M M M M M M M | discarded lower f32 frac |
  +---+-----------------+---------------+--------------------------+
    \___________________ _______________/
            kept as f16b high 16 bits

  f16b:
  +---+-----------------+---------------+
  | S | E E E E E E E E | M M M M M M M |
  +---+-----------------+---------------+