Rust RFCs - RFC Book - Active RFC List

The “RFC” (request for comments) process is intended to provide a consistent and controlled path for changes to Rust (such as new features) so that all stakeholders can be confident about the direction of the project.

Many changes, including bug fixes and documentation improvements can be implemented and reviewed via the normal GitHub pull request workflow.

Some changes though are “substantial”, and we ask that these be put through a bit of a design process and produce a consensus among the Rust community and the sub-teams.

Table of Contents

• Opening
• Table of Contents
• When you need to follow this process
• Sub-team specific guidelines
• Before creating an RFC
• What the process is
• The RFC life-cycle
• Reviewing RFCs
• Implementing an RFC
• RFC Postponement
• Help this is all too informal!
• License
• Contributions

When you need to follow this process

You need to follow this process if you intend to make “substantial” changes to Rust, Cargo, Crates.io, or the RFC process itself. What constitutes a “substantial” change is evolving based on community norms and varies depending on what part of the ecosystem you are proposing to change, but may include the following.

• Any semantic or syntactic change to the language that is not a bugfix.
• Removing language features, including those that are feature-gated.
• Changes to the interface between the compiler and libraries, including lang items and intrinsics.
• Additions to std.

Some changes do not require an RFC:

• Rephrasing, reorganizing, refactoring, or otherwise “changing shape does not change meaning”.
• Additions that strictly improve objective, numerical quality criteria (warning removal, speedup, better platform coverage, more parallelism, trap more errors, etc.)
• Additions only likely to be noticed by other developers-of-rust, invisible to users-of-rust.

If you submit a pull request to implement a new feature without going through the RFC process, it may be closed with a polite request to submit an RFC first.

Sub-team specific guidelines

For more details on when an RFC is required for the following areas, please see the Rust community’s sub-team specific guidelines for:

• language changes,
• library changes,
• compiler changes.

Before creating an RFC

A hastily-proposed RFC can hurt its chances of acceptance. Low quality proposals, proposals for previously-rejected features, or those that don’t fit into the near-term roadmap, may be quickly rejected, which can be demotivating for the unprepared contributor. Laying some groundwork ahead of the RFC can make the process smoother.

Although there is no single way to prepare for submitting an RFC, it is generally a good idea to pursue feedback from other project developers beforehand, to ascertain that the RFC may be desirable; having a consistent impact on the project requires concerted effort toward consensus-building.

The most common preparations for writing and submitting an RFC include talking the idea over on our official Zulip server, discussing the topic on our developer discussion forum, and occasionally posting “pre-RFCs” on the developer forum. You may file issues on this repo for discussion, but these are not actively looked at by the teams.

As a rule of thumb, receiving encouraging feedback from long-standing project developers, and particularly members of the relevant sub-team is a good indication that the RFC is worth pursuing.

What the process is

In short, to get a major feature added to Rust, one must first get the RFC merged into the RFC repository as a markdown file. At that point the RFC is “active” and may be implemented with the goal of eventual inclusion into Rust.

• Fork the RFC repo RFC repository
• Copy 0000-template.md to text/0000-my-feature.md (where “my-feature” is descriptive). Don’t assign an RFC number yet; This is going to be the PR number and we’ll rename the file accordingly if the RFC is accepted.
• Fill in the RFC. Put care into the details: RFCs that do not present convincing motivation, demonstrate lack of understanding of the design’s impact, or are disingenuous about the drawbacks or alternatives tend to be poorly-received.
• Submit a pull request. As a pull request the RFC will receive design feedback from the larger community, and the author should be prepared to revise it in response.
• Now that your RFC has an open pull request, use the issue number of the PR to rename the file: update your 0000- prefix to that number. Also update the “RFC PR” link at the top of the file.
• Each pull request will be labeled with the most relevant sub-team, which will lead to its being triaged by that team in a future meeting and assigned to a member of the subteam.
• Build consensus and integrate feedback. RFCs that have broad support are much more likely to make progress than those that don’t receive any comments. Feel free to reach out to the RFC assignee in particular to get help identifying stakeholders and obstacles.
• The sub-team will discuss the RFC pull request, as much as possible in the comment thread of the pull request itself. Offline discussion will be summarized on the pull request comment thread.
• RFCs rarely go through this process unchanged, especially as alternatives and drawbacks are shown. You can make edits, big and small, to the RFC to clarify or change the design, but make changes as new commits to the pull request, and leave a comment on the pull request explaining your changes. Specifically, do not squash or rebase commits after they are visible on the pull request.
• At some point, a member of the subteam will propose a “motion for final comment period” (FCP), along with a disposition for the RFC (merge, close, or postpone).
  • This step is taken when enough of the tradeoffs have been discussed that the subteam is in a position to make a decision. That does not require consensus amongst all participants in the RFC thread (which is usually impossible). However, the argument supporting the disposition on the RFC needs to have already been clearly articulated, and there should not be a strong consensus against that position outside of the subteam. Subteam members use their best judgment in taking this step, and the FCP itself ensures there is ample time and notification for stakeholders to push back if it is made prematurely.
  • For RFCs with lengthy discussion, the motion to FCP is usually preceded by a summary comment trying to lay out the current state of the discussion and major tradeoffs/points of disagreement.
  • Before actually entering FCP, all members of the subteam must sign off; this is often the point at which many subteam members first review the RFC in full depth.
• The FCP lasts ten calendar days, so that it is open for at least 5 business days. It is also advertised widely, e.g. in This Week in Rust. This way all stakeholders have a chance to lodge any final objections before a decision is reached.
• In most cases, the FCP period is quiet, and the RFC is either merged or closed. However, sometimes substantial new arguments or ideas are raised, the FCP is canceled, and the RFC goes back into development mode.

The RFC life-cycle

Once an RFC becomes “active” then authors may implement it and submit the feature as a pull request to the Rust repo. Being “active” is not a rubber stamp, and in particular still does not mean the feature will ultimately be merged; it does mean that in principle all the major stakeholders have agreed to the feature and are amenable to merging it.

Furthermore, the fact that a given RFC has been accepted and is “active” implies nothing about what priority is assigned to its implementation, nor does it imply anything about whether a Rust developer has been assigned the task of implementing the feature. While it is not necessary that the author of the RFC also write the implementation, it is by far the most effective way to see an RFC through to completion: authors should not expect that other project developers will take on responsibility for implementing their accepted feature.

Modifications to “active” RFCs can be done in follow-up pull requests. We strive to write each RFC in a manner that it will reflect the final design of the feature; but the nature of the process means that we cannot expect every merged RFC to actually reflect what the end result will be at the time of the next major release.

In general, once accepted, RFCs should not be substantially changed. Only very minor changes should be submitted as amendments. More substantial changes should be new RFCs, with a note added to the original RFC. Exactly what counts as a “very minor change” is up to the sub-team to decide; check Sub-team specific guidelines for more details.

Reviewing RFCs

While the RFC pull request is up, the sub-team may schedule meetings with the author and/or relevant stakeholders to discuss the issues in greater detail, and in some cases the topic may be discussed at a sub-team meeting. In either case a summary from the meeting will be posted back to the RFC pull request.

A sub-team makes final decisions about RFCs after the benefits and drawbacks are well understood. These decisions can be made at any time, but the sub-team will regularly issue decisions. When a decision is made, the RFC pull request will either be merged or closed. In either case, if the reasoning is not clear from the discussion in thread, the sub-team will add a comment describing the rationale for the decision.

Implementing an RFC

Some accepted RFCs represent vital features that need to be implemented right away. Other accepted RFCs can represent features that can wait until some arbitrary developer feels like doing the work. Every accepted RFC has an associated issue tracking its implementation in the Rust repository; thus that associated issue can be assigned a priority via the triage process that the team uses for all issues in the Rust repository.

The author of an RFC is not obligated to implement it. Of course, the RFC author (like any other developer) is welcome to post an implementation for review after the RFC has been accepted.

If you are interested in working on the implementation for an “active” RFC, but cannot determine if someone else is already working on it, feel free to ask (e.g. by leaving a comment on the associated issue).

RFC Postponement

Some RFC pull requests are tagged with the “postponed” label when they are closed (as part of the rejection process). An RFC closed with “postponed” is marked as such because we want neither to think about evaluating the proposal nor about implementing the described feature until some time in the future, and we believe that we can afford to wait until then to do so. Historically, “postponed” was used to postpone features until after 1.0. Postponed pull requests may be re-opened when the time is right. We don’t have any formal process for that, you should ask members of the relevant sub-team.

Usually an RFC pull request marked as “postponed” has already passed an informal first round of evaluation, namely the round of “do we think we would ever possibly consider making this change, as outlined in the RFC pull request, or some semi-obvious variation of it.” (When the answer to the latter question is “no”, then the appropriate response is to close the RFC, not postpone it.)

Help this is all too informal!

The process is intended to be as lightweight as reasonable for the present circumstances. As usual, we are trying to let the process be driven by consensus and community norms, not impose more structure than necessary.

License

This repository is currently in the process of being licensed under either of:

• Apache License, Version 2.0, (LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or https://opensource.org/licenses/MIT)

at your option. Some parts of the repository are already licensed according to those terms. For more see RFC 2044 and its tracking issue.

Contributions

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

RFC policy - the compiler

Compiler RFCs will be managed by the compiler sub-team, and tagged T-compiler. The compiler sub-team will do an initial triage of new PRs within a week of submission. The result of triage will either be that the PR is assigned to a member of the sub-team for shepherding, the PR is closed because the sub-team believe it should be done without an RFC, or closed because the sub-team feel it should clearly not be done and further discussion is not necessary. We’ll follow the standard procedure for shepherding, final comment period, etc.

Most compiler decisions that go beyond the scope of a simple PR are done using MCPs, not RFCs. It is therefore likely that you should file an MCP instead of an RFC for your problem.

Changes which need an RFC

• Significant user-facing changes to the compiler with a complex design space, especially if they involve other teams as well (for example, path sanitization).
• Any other change which causes significant backwards incompatible changes to stable behaviour of the compiler, language, or libraries

Changes which don’t need an RFC

• Bug fixes, improved error messages, etc.
• Minor refactoring/tidying up
• Large internal refactorings or redesigns of the compiler (needs an MCP)
• Implementing language features which have an accepted RFC.
• New lints (these fall under the lang team). Lints are best first tried out in clippy and then uplifted later.
• Changing the API presented to syntax extensions or other compiler plugins in non-trivial ways
• Adding, removing, or changing a stable compiler flag (needs an FCP somewhere, like on an MCP or just on a PR)
• Adding unstable API for tools (note that all compiler API is currently unstable)
• Adding, removing, or changing an unstable compiler flag (if the compiler flag is widely used there should be at least some discussion on discuss, or an RFC in some cases)

If in doubt it is probably best to just announce the change you want to make to the compiler subteam on Zulip, and see if anyone feels it needs an RFC.

RFC policy - language design

Pretty much every change to the language needs an RFC. Note that new lints (or major changes to an existing lint) are considered changes to the language.

Language RFCs are managed by the language sub-team, and tagged T-lang. The language sub-team will do an initial triage of new PRs within a week of submission. The result of triage will either be that the PR is assigned to a member of the sub-team for shepherding, the PR is closed as postponed because the subteam believe it might be a good idea, but is not currently aligned with Rust’s priorities, or the PR is closed because the sub-team feel it should clearly not be done and further discussion is not necessary. In the latter two cases, the sub-team will give a detailed explanation. We’ll follow the standard procedure for shepherding, final comment period, etc.

Amendments

Sometimes in the implementation of an RFC, changes are required. In general these don’t require an RFC as long as they are very minor and in the spirit of the accepted RFC (essentially bug fixes). In this case implementers should submit an RFC PR which amends the accepted RFC with the new details. Although the RFC repository is not intended as a reference manual, it is preferred that RFCs do reflect what was actually implemented. Amendment RFCs will go through the same process as regular RFCs, but should be less controversial and thus should move more quickly.

When a change is more dramatic, it is better to create a new RFC. The RFC should be standalone and reference the original, rather than modifying the existing RFC. You should add a comment to the original RFC with referencing the new RFC as part of the PR.

Obviously there is some scope for judgment here. As a guideline, if a change affects more than one part of the RFC (i.e., is a non-local change), affects the applicability of the RFC to its motivating use cases, or there are multiple possible new solutions, then the feature is probably not ‘minor’ and should get a new RFC.

RFC guidelines - libraries sub-team

Motivation

• RFCs are heavyweight:

  • RFCs generally take at minimum 2 weeks from posting to land. In practice it can be more on the order of months for particularly controversial changes.
  • RFCs are a lot of effort to write; especially for non-native speakers or for members of the community whose strengths are more technical than literary.
  • RFCs may involve pre-RFCs and several rewrites to accommodate feedback.
  • RFCs require a dedicated shepherd to herd the community and author towards consensus.
  • RFCs require review from a majority of the subteam, as well as an official vote.
  • RFCs can’t be downgraded based on their complexity. Full process always applies. Easy RFCs may certainly land faster, though.
  • RFCs can be very abstract and hard to grok the consequences of (no implementation).

• PRs are low overhead but potentially expensive nonetheless:

  • Easy PRs can get insta-merged by any rust-lang contributor.
  • Harder PRs can be easily escalated. You can ping subject-matter experts for second opinions. Ping the whole team!
  • Easier to grok the full consequences. Lots of tests and Crater to save the day.
  • PRs can be accepted optimistically with bors, buildbot, and the trains to guard us from major mistakes making it into stable. The size of the nightly community at this point in time can still mean major community breakage regardless of trains, however.
  • HOWEVER: Big PRs can be a lot of work to make only to have that work rejected for details that could have been hashed out first.

• RFCs are only meaningful if a significant and diverse portion of the community actively participates in them. The official teams are not sufficiently diverse to establish meaningful community consensus by agreeing amongst themselves.

• If there are tons of RFCs – especially trivial ones – people are less likely to engage with them. Official team members are super busy. Domain experts and industry professionals are super busy and have no responsibility to engage in RFCs. Since these are exactly the most important people to get involved in the RFC process, it is important that we be maximally friendly towards their needs.

Is an RFC required?

The overarching philosophy is: do whatever is easiest. If an RFC would be less work than an implementation, that’s a good sign that an RFC is necessary. That said, if you anticipate controversy, you might want to short-circuit straight to an RFC. For instance new APIs almost certainly merit an RFC. Especially as std has become more conservative in favour of the much more agile cargoverse.

• Submit a PR if the change is a:
  • Bugfix
  • Docfix
  • Obvious API hole patch, such as adding an API from one type to a symmetric type. e.g. Vec<T> -> Box<[T]> clearly motivates adding String -> Box<str>
  • Minor tweak to an unstable API (renaming, generalizing)
  • Implementing an “obvious” trait like Clone/Debug/etc
• Submit an RFC if the change is a:
  • New API
  • Semantic Change to a stable API
  • Generalization of a stable API (e.g. how we added Pattern or Borrow)
  • Deprecation of a stable API
  • Nontrivial trait impl (because all trait impls are insta-stable)
• Do the easier thing if uncertain. (choosing a path is not final)

Non-RFC process

• A (non-RFC) PR is likely to be closed if clearly not acceptable:

  • Disproportionate breaking change (small inference breakage may be acceptable)
  • Unsound
  • Doesn’t fit our general design philosophy around the problem
  • Better as a crate
  • Too marginal for std
  • Significant implementation problems

• A PR may also be closed because an RFC is appropriate.

• A (non-RFC) PR may be merged as unstable. In this case, the feature should have a fresh feature gate and an associated tracking issue for stabilisation. Note that trait impls and docs are insta-stable and thus have no tracking issue. This may imply requiring a higher level of scrutiny for such changes.

However, an accepted RFC is not a rubber-stamp for merging an implementation PR. Nor must an implementation PR perfectly match the RFC text. Implementation details may merit deviations, though obviously they should be justified. The RFC may be amended if deviations are substantial, but are not generally necessary. RFCs should favour immutability. The RFC + Issue + PR should form a total explanation of the current implementation.

• Once something has been merged as unstable, a shepherd should be assigned to promote and obtain feedback on the design.

• Every time a release cycle ends, the libs teams assesses the current unstable APIs and selects some number of them for potential stabilization during the next cycle. These are announced for FCP at the beginning of the cycle, and (possibly) stabilized just before the beta is cut.

• After the final comment period, an API should ideally take one of two paths:

  • Stabilize if the change is desired, and consensus is reached
  • Deprecate is the change is undesired, and consensus is reached
  • Extend the FCP is the change cannot meet consensus
    • If consensus still can’t be reached, consider requiring a new RFC or just deprecating as “too controversial for std”.

• If any problems are found with a newly stabilized API during its beta period, strongly favour reverting stability in order to prevent stabilizing a bad API. Due to the speed of the trains, this is not a serious delay (~2-3 months if it’s not a major problem).

• Start Date: 2014-03-11
• RFC PR: rust-lang/rfcs#1
• Rust Issue: rust-lang/rust#8122

Summary

This is an RFC to make all struct fields private by default. This includes both tuple structs and structural structs.

Motivation

Reasons for default private visibility

• Visibility is often how soundness is achieved for many types in rust. These types are normally wrapping unsafe behavior of an FFI type or some other rust-specific behavior under the hood (such as the standard Vec type). Requiring these types to opt-in to being sound is unfortunate.

• Forcing tuple struct fields to have non-overridable public visibility greatly reduces the utility of such types. Tuple structs cannot be used to create abstraction barriers as they can always be easily destructed.

• Private-by-default is more consistent with the rest of the Rust language. All other aspects of privacy are private-by-default except for enum variants. Enum variants, however, are a special case in that they are inserted into the parent namespace, and hence naturally inherit privacy.

• Public fields of a struct must be considered as part of the API of the type. This means that the exact definition of all structs is by default the API of the type. Structs must opt-out of this behavior if the priv keyword is required. By requiring the pub keyword, structs must opt-in to exposing more surface area to their API.

Reasons for inherited visibility (today’s design)

• Public definitions like pub struct Point { x: int, y: int } are concise and easy to read.
• Private definitions certainly want private fields (to hide implementation details).

Detailed design

Currently, rustc has two policies for dealing with the privacy of struct fields:

• Tuple structs have public fields by default (including “newtype structs”)
• Fields of structural structs (struct Foo { ... }) inherit the same privacy of the enclosing struct.

This RFC is a proposal to unify the privacy of struct fields with the rest of the language by making them private by default. This means that both tuple variants and structural variants of structs would have private fields by default. For example, the program below is accepted today, but would be rejected with this RFC.

mod inner {
    pub struct Foo(u64);
    pub struct Bar { field: u64 }
}

fn main() {
    inner::Foo(10);
    inner::Bar { field: 10 };
}

Refinements to structural structs

Public fields are quite a useful feature of the language, so syntax is required to opt out of the private-by-default semantics. Structural structs already allow visibility qualifiers on fields, and the pub qualifier would make the field public instead of private.

Additionally, the priv visibility will no longer be allowed to modify struct fields. Similarly to how a priv fn is a compiler error, a priv field will become a compiler error.

Refinements on tuple structs

As with their structural cousins, it’s useful to have tuple structs with public fields. This RFC will modify the tuple struct grammar to:

tuple_struct := 'struct' ident '(' fields ')' ';'
fields := field | field ',' fields
field := type | visibility type

For example, these definitions will be added to the language:

// a "newtype wrapper" struct with a private field
struct Foo(u64);

// a "newtype wrapper" struct with a public field
struct Bar(pub u64);

// a tuple struct with many fields, only the first and last of which are public
struct Baz(pub u64, u32, f32, pub int);

Public fields on tuple structs will maintain the semantics that they currently have today. The structs can be constructed, destructed, and participate in pattern matches.

Private fields on tuple structs will prevent the following behaviors:

• Private fields cannot be bound in patterns (both in irrefutable and refutable contexts, i.e. let and match statements).
• Private fields cannot be specified outside of the defining module when constructing a tuple struct.

These semantics are intended to closely mirror the behavior of private fields for structural structs.

Statistics gathered

A brief survey was performed over the entire mozilla/rust repository to gather these statistics. While not representative of all projects, this repository should give a good indication of what most structs look like in the real world. The repository has both libraries (libstd) as well as libraries which don’t care much about privacy (librustc).

These numbers tally up all structs from all locations, and only take into account structural structs, not tuple structs.

The numbers clearly show that the predominant pattern is to have all-private structs, and that there are many public fields today which can be private (and perhaps should!). Additionally, there is on the order of 1418 instances of the word priv today, when in theory there should be around 1852. With this RFC, there would need to be 1602 instances of the word pub. A very large portion of structs requiring pub fields are FFI structs defined in the libc module.

Impact on enums

This RFC does not impact enum variants in any way. All enum variants will continue to inherit privacy from the outer enum type. This includes both the fields of tuple variants as well as fields of struct variants in enums.

Alternatives

The main alternative to this design is what is currently implemented today, where fields inherit the privacy of the outer structure. The pros and cons of this strategy are discussed above.

Unresolved questions

As the above statistics show, almost all structures are either all public or all private. This RFC provides an easy method to make struct fields all private, but it explicitly does not provide a method to make struct fields all public. The statistics show that pub will be written less often than priv is today, and it’s always possible to add a method to specify a struct as all-public in the future in a backwards-compatible fashion.

That being said, it’s an open question whether syntax for an “all public struct” is necessary at this time.

• Start Date: 2014-03-11
• RFC PR: rust-lang/rfcs#2, rust-lang/rfcs#6
• Rust Issue: N/A

Summary

The “RFC” (request for comments) process is intended to provide a consistent and controlled path for new features to enter the language and standard libraries, so that all stakeholders can be confident about the direction the language is evolving in.

Motivation

The freewheeling way that we add new features to Rust has been good for early development, but for Rust to become a mature platform we need to develop some more self-discipline when it comes to changing the system. This is a proposal for a more principled RFC process to make it a more integral part of the overall development process, and one that is followed consistently to introduce features to Rust.

Detailed design

Many changes, including bug fixes and documentation improvements can be implemented and reviewed via the normal GitHub pull request workflow.

Some changes though are “substantial”, and we ask that these be put through a bit of a design process and produce a consensus among the Rust community and the core team.

When you need to follow this process

You need to follow this process if you intend to make “substantial” changes to the Rust distribution. What constitutes a “substantial” change is evolving based on community norms, but may include the following.

• Any semantic or syntactic change to the language that is not a bugfix.
• Removing language features, including those that are feature-gated.
• Changes to the interface between the compiler and libraries, including lang items and intrinsics.
• Additions to std

Some changes do not require an RFC:

• Rephrasing, reorganizing, refactoring, or otherwise “changing shape does not change meaning”.
• Additions that strictly improve objective, numerical quality criteria (warning removal, speedup, better platform coverage, more parallelism, trap more errors, etc.)
• Additions only likely to be noticed by other developers-of-rust, invisible to users-of-rust.

If you submit a pull request to implement a new feature without going through the RFC process, it may be closed with a polite request to submit an RFC first.

What the process is

In short, to get a major feature added to Rust, one must first get the RFC merged into the RFC repo as a markdown file. At that point the RFC is ‘active’ and may be implemented with the goal of eventual inclusion into Rust.

• Fork the RFC repo https://github.com/rust-lang/rfcs
• Copy 0000-template.md to text/0000-my-feature.md (where ‘my-feature’ is descriptive. don’t assign an RFC number yet).
• Fill in the RFC
• Submit a pull request. The pull request is the time to get review of the design from the larger community.
• Build consensus and integrate feedback. RFCs that have broad support are much more likely to make progress than those that don’t receive any comments.

Eventually, somebody on the core team will either accept the RFC by merging the pull request, at which point the RFC is ‘active’, or reject it by closing the pull request.

Whomever merges the RFC should do the following:

• Assign an id, using the PR number of the RFC pull request. (If the RFC has multiple pull requests associated with it, choose one PR number, preferably the minimal one.)
• Add the file in the text/ directory.
• Create a corresponding issue on Rust repo
• Fill in the remaining metadata in the RFC header, including links for the original pull request(s) and the newly created Rust issue.
• Add an entry in the Active RFC List of the root README.md.
• Commit everything.

Once an RFC becomes active then authors may implement it and submit the feature as a pull request to the Rust repo. An ‘active’ is not a rubber stamp, and in particular still does not mean the feature will ultimately be merged; it does mean that in principle all the major stakeholders have agreed to the feature and are amenable to merging it.

Modifications to active RFC’s can be done in followup PR’s. An RFC that makes it through the entire process to implementation is considered ‘complete’ and is removed from the Active RFC List; an RFC that fails after becoming active is ‘inactive’ and moves to the ‘inactive’ folder.

Alternatives

Retain the current informal RFC process. The newly proposed RFC process is designed to improve over the informal process in the following ways:

• Discourage unactionable or vague RFCs
• Ensure that all serious RFCs are considered equally
• Give confidence to those with a stake in Rust’s development that they understand why new features are being merged

As an alternative, we could adopt an even stricter RFC process than the one proposed here. If desired, we should likely look to Python’s PEP process for inspiration.

Unresolved questions

1. Does this RFC strike a favorable balance between formality and agility?
2. Does this RFC successfully address the aforementioned issues with the current informal RFC process?
3. Should we retain rejected RFCs in the archive?

• Start Date: 2012-03-20
• RFC PR: rust-lang/rfcs#3
• Rust Issue: rust-lang/rust#14373

Summary

Rust currently has an attribute usage lint but it does not work particularly well. This RFC proposes a new implementation strategy that should make it significantly more useful.

Motivation

The current implementation has two major issues:

• There are very limited warnings for valid attributes that end up in the wrong place. Something like this will be silently ignored:

#[deriving(Clone)]; // Shouldn't have put a ; here
struct Foo;

#[ignore(attribute-usage)] // Should have used #[allow(attribute-usage)] instead!
mod bar {
    //...
}

• ItemDecorators can now be defined outside of the compiler, and there’s no way to tag them and associated attributes as valid. Something like this requires an #[allow(attribute-usage)]:

#[feature(phase)];
#[phase(syntax, link)]
extern crate some_orm;

#[ormify]
pub struct Foo {
    #[column(foo_)]
    #[primary_key]
    foo: int
}

Detailed design

The current implementation is implemented as a simple fold over the AST, comparing attributes against a whitelist. Crate-level attributes use a separate whitelist, but no other distinctions are made.

This RFC would change the implementation to actually track which attributes are used during the compilation process. syntax::ast::Attribute_ would be modified to add an ID field:

pub struct AttrId(uint);

pub struct Attribute_ {
    id: AttrId,
    style: AttrStyle,
    value: @MetaItem,
    is_sugared_doc: bool,
}

syntax::ast::parse::ParseSess will generate new AttrIds on demand. I believe that attributes will only be created during parsing and expansion, and the ParseSess is accessible in both.

The AttrIds will be used to create a side table of used attributes. This will most likely be a thread local to make it easily accessible during all stages of compilation by calling a function in syntax::attr:

fn mark_used(attr: &Attribute) { }

The attribute-usage lint would run at the end of compilation and warn on all attributes whose ID does not appear in the side table.

One interesting edge case is attributes like doc that are used, but not in the normal compilation process. There could either be a separate fold pass to mark all doc attributes as used or doc could simply be whitelisted in the attribute-usage lint.

Attributes in code that has been eliminated with #[cfg()] will not be linted, but I feel that this is consistent with the way #[cfg()] works in general (e.g. the code won’t be type-checked either).

Alternatives

An alternative would be to rewrite rustc::middle::lint to robustly check that attributes are used where they’re supposed to be. This will be fairly complex and be prone to failure if/when more nodes are added to the AST. This also doesn’t solve motivation #2, which would require externally loaded lint support.

Unresolved questions

• This implementation doesn’t allow for a distinction between “unused” and “unknown” attributes. The #[phase(syntax)] crate loading infrastructure could be extended to pull a list of attributes from crates to use in the lint pass, but I’m not sure if the extra complexity is worth it.
• The side table could be threaded through all of the compilation stages that need to use it instead of being a thread local. This would probably require significantly more work than the thread local approach, however. The thread local approach should not negatively impact any future parallelization work as each thread can keep its own side table, which can be merged into one for the lint pass.

• Start Date: 2014-03-14
• RFC PR: rust-lang/rfcs#8
• Rust Issue:

Note: this RFC was never implemented and has been retired. The design may still be useful in the future, but before implementing we would prefer to revisit it so as to be sure it is up to date.

Summary

The way our intrinsics work forces them to be wrapped in order to behave like normal functions. As a result, rustc is forced to inline a great number of tiny intrinsic wrappers, which is bad for both compile-time performance and runtime performance without optimizations. This proposal changes the way intrinsics are surfaced in the language so that they behave the same as normal Rust functions by removing the “rust-intrinsic” foreign ABI and reusing the “Rust” ABI.

Motivation

A number of commonly-used intrinsics, including transmute, forget, init, uninit, and move_val_init, are accessed through wrappers whose only purpose is to present the intrinsics as normal functions. As a result, rustc is forced to inline a great number of tiny intrinsic wrappers, which is bad for both compile-time performance and runtime performance without optimizations.

Intrinsics have a differently-named ABI from Rust functions (“rust-intrinsic” vs. “Rust”) though the actual ABI implementation is identical. As a result one can’t take the value of an intrinsic as a function:

// error: the type of transmute is `extern "rust-intrinsic" fn ...`
let transmute: fn(int) -> uint = intrinsics::transmute;

This incongruity means that we can’t just expose the intrinsics directly as part of the public API.

Detailed design

extern "Rust" fn is already equivalent to fn, so if intrinsics have the “Rust” ABI then the problem is solved.

Under this scheme intrinsics will be declared as extern "Rust" functions and identified as intrinsics with the #[lang = "..."] attribute:

extern "Rust" {
    #[lang = "transmute"]
    fn transmute<T, U>(T) -> U;
}

The compiler will type check and translate intrinsics the same as today. Additionally, when trans sees a “Rust” extern tagged as an intrinsic it will not emit a function declaration to LLVM bitcode.

Because intrinsics will be lang items, they can no longer be redeclared arbitrary number of times. This will require a small amount of existing library code to be refactored, and all intrinsics to be exposed through public abstractions.

Currently, “Rust” foreign functions may not be generic; this change will require a special case that allows intrinsics to be generic.

Alternatives

1. Instead of making intrinsics lang items we could create a slightly different mechanism, like an #[intrinsic] attribute, that would continue letting intrinsics to be redeclared.

2. While using lang items to identify intrinsics, intrinsic lang items could be allowed to be redeclared.

3. We could also make “rust-intrinsic” coerce or otherwise be the same as “Rust” externs and normal Rust functions.

Unresolved questions

None.

• Start Date: 2014-03-20
• RFC PR: rust-lang/rfcs#16
• Rust Issue: rust-lang/rust#15701

Summary

Allow attributes on more places inside functions, such as statements, blocks and expressions.

Motivation

One sometimes wishes to annotate things inside functions with, for example, lint #[allow]s, conditional compilation #[cfg]s, and even extra semantic (or otherwise) annotations for external tools.

For the lints, one can currently only activate lints at the level of the function which is possibly larger than one needs, and so may allow other “bad” things to sneak through accidentally. E.g.

#[allow(uppercase_variable)]
let L = List::new(); // lowercase looks like one or capital i

For the conditional compilation, the work-around is duplicating the whole containing function with a #[cfg], or breaking the conditional code into a its own function. This does mean that any variables need to be explicitly passed as arguments.

The sort of things one could do with other arbitrary annotations are

#[allowed_unsafe_actions(ffi)]
#[audited="2014-04-22"]
unsafe { ... }

and then have an external tool that checks that that unsafe block’s only unsafe actions are FFI, or a tool that lists blocks that have been changed since the last audit or haven’t been audited ever.

The minimum useful functionality would be supporting attributes on blocks and let statements, since these are flexible enough to allow for relatively precise attribute handling.

Detailed design

Normal attribute syntax on let statements, blocks and expressions.

fn foo() {
    #[attr1]
    let x = 1;

    #[attr2]
    {
        // code
    }

    #[attr3]
    unsafe {
        // code
    }
    #[attr4] foo();

    let x = #[attr5] 1;

    qux(3 + #[attr6] 2);

    foo(x, #[attr7] y, z);
}

Attributes bind tighter than any operator, that is #[attr] x op y is always parsed as (#[attr] x) op y.

cfg

It is definitely an error to place a #[cfg] attribute on a non-statement expressions, that is, attr1–attr4 can possibly be #[cfg(foo)], but attr5–attr7 cannot, since it makes little sense to strip code down to let x = ;.

However, like #ifdef in C/C++, widespread use of #[cfg] may be an antipattern that makes code harder to read. This RFC is just adding the ability for attributes to be placed in specific places, it is not mandating that #[cfg] actually be stripped in those places (although it should be an error if it is ignored).

Inner attributes

Inner attributes can be placed at the top of blocks (and other structure incorporating a block) and apply to that block.

{
    #![attr11]

    foo()
}

match bar {
    #![attr12]

    _ => {}
}

// are the same as

#[attr11]
{
    foo()
}

#[attr12]
match bar {
    _ => {}
}

if

Attributes would be disallowed on if for now, because the interaction with if/else chains are funky, and can be simulated in other ways.

#[cfg(not(foo))]
if cond1 {
} else #[cfg(not(bar))] if cond2 {
} else #[cfg(not(baz))] {
}

There is two possible interpretations of such a piece of code, depending on if one regards the attributes as attaching to the whole if ... else chain (“exterior”) or just to the branch on which they are placed (“interior”).

• --cfg foo: could be either removing the whole chain (exterior) or equivalent to if cond2 {} else {} (interior).
• --cfg bar: could be either if cond1 {} (e) or if cond1 {} else {} (i)
• --cfg baz: equivalent to if cond1 {} else if cond2 {} (no subtlety).
• --cfg foo --cfg bar: could be removing the whole chain (e) or the two if branches (leaving only the else branch) (i).

(This applies to any attribute that has some sense of scoping, not just #[cfg], e.g. #[allow] and #[warn] for lints.)

As such, to avoid confusion, attributes would not be supported on if. Alternatives include using blocks:

#[attr] if cond { ... } else ...
// becomes, for an exterior attribute,
#[attr] {
    if cond { ... } else ...
}
// and, for an interior attribute,
if cond {
    #[attr] { ... }
} else ...

And, if the attributes are meant to be associated with the actual branching (e.g. a hypothetical #[cold] attribute that indicates a branch is unlikely), one can annotate match arms:

match cond {
    #[attr] true => { ... }
    #[attr] false => { ... }
}

Drawbacks

This starts mixing attributes with nearly arbitrary code, possibly dramatically restricting syntactic changes related to them, for example, there was some consideration for using @ for attributes, this change may make this impossible (especially if @ gets reused for something else, e.g. Python is using it for matrix multiplication). It may also make it impossible to use # for other things.

As stated above, allowing #[cfg]s everywhere can make code harder to reason about, but (also stated), this RFC is not for making such #[cfg]s be obeyed, it just opens the language syntax to possibly allow it.

Alternatives

These instances could possibly be approximated with macros and helper functions, but to a low degree degree (e.g. how would one annotate a general unsafe block).

Only allowing attributes on “statement expressions” that is, expressions at the top level of a block, this is slightly limiting; but we can expand to support other contexts backwards compatibly in the future.

The if/else issue may be able to be resolved by introducing explicit “interior” and “exterior” attributes on if: by having #[attr] if cond { ... be an exterior attribute (applying to the whole if/else chain) and if cond #[attr] { ... be an interior attribute (applying to only the current if branch). There is no difference between interior and exterior for an else { branch, and so else #[attr] { is sufficient.

Unresolved questions

Are the complications of allowing attributes on arbitrary expressions worth the benefits?

• Start Date: 2014-09-18
• RFC PR #: rust-lang/rfcs#19, rust-lang/rfcs#127
• Rust Issue #: rust-lang/rust#13231

Note: The Share trait described in this RFC was later renamed to Sync.

Summary

The high-level idea is to add language features that simultaneously achieve three goals:

1. move Send and Share out of the language entirely and into the standard library, providing mechanisms for end users to easily implement and use similar “marker” traits of their own devising;
2. make “normal” Rust types sendable and sharable by default, without the need for explicit opt-in; and,
3. continue to require “unsafe” Rust types (those that manipulate unsafe pointers or implement special abstractions) to “opt-in” to sendability and sharability with an unsafe declaration.

These goals are achieved by two changes:

1. Unsafe traits: An unsafe trait is a trait that is unsafe to implement, because it represents some kind of trusted assertion. Note that unsafe traits are perfectly safe to use. Send and Share are examples of unsafe traits: implementing these traits is effectively an assertion that your type is safe for threading.

2. Default and negative impls: A default impl is one that applies to all types, except for those types that explicitly opt out. For example, there would be a default impl for Send, indicating that all types are Send “by default”.

   To counteract a default impl, one uses a negative impl that explicitly opts out for a given type T and any type that contains T. For example, this RFC proposes that unsafe pointers *T will opt out of Send and Share. This implies that unsafe pointers cannot be sent or shared between threads by default. It also implies that any structs which contain an unsafe pointer cannot be sent. In all examples encountered thus far, the set of negative impls is fixed and can easily be declared along with the trait itself.

   Safe wrappers like Arc, Atomic, or Mutex can opt to implement Send and Share explicitly. This will then make them be considered sendable (or sharable) even though they contain unsafe pointers etc.

Based on these two mechanisms, we can remove the notion of Send and Share as builtin concepts. Instead, these would become unsafe traits with default impls (defined purely in the library). The library would explicitly opt out of Send/Share for certain types, like unsafe pointers (*T) or interior mutability (Unsafe<T>). Any type, therefore, which contains an unsafe pointer would be confined (by default) to a single thread. Safe wrappers around those types, like Arc, Atomic, or Mutex, can then opt back in by explicitly implementing Send (these impls would have to be designed as unsafe).

Motivation

Since proposing opt-in builtin traits, I have become increasingly concerned about the notion of having Send and Share be strictly opt-in. There are two main reasons for my concern:

1. Rust is very close to being a language where computations can be parallelized by default. Making Send, and especially Share, opt-in makes that harder to achieve.
2. The model followed by Send/Share cannot easily be extended to other traits in the future nor can it be extended by end-users with their own similar traits. It is worrisome that I have come across several use cases already which might require such extension (described below).

To elaborate on those two points: With respect to parallelization: for the most part, Rust types are threadsafe “by default”. To make something non-threadsafe, you must employ unsynchronized interior mutability (e.g., Cell, RefCell) or unsynchronized shared ownership (Rc). In both cases, there are also synchronized variants available (Mutex, Arc, etc). This implies that we can make APIs to enable intra-task parallelism and they will work ubiquitously, so long as people avoid Cell and Rc when not needed. Explicit opt-in threatens that future, however, because fewer types will implement Share, even if they are in fact threadsafe.

With respect to extensibility, it is particularly worrisome that if a library forgets to implement Send or Share, downstream clients are stuck. They cannot, for example, use a newtype wrapper, because it would be illegal to implement Send on the newtype. This implies that all libraries must be vigilant about implementing Send and Share (even more so than with other pervasive traits like Eq or Ord). The current plan is to address this via lints and perhaps some convenient deriving syntax, which may be adequate for Send and Share. But if we wish to add new “classification” traits in the future, these new traits won’t have been around from the start, and hence won’t be implemented by all existing code.

Another concern of mine is that end users cannot define classification traits of their own. For example, one might like to define a trait for “tainted” data, and then test to ensure that tainted data doesn’t pass through some generic routine. There is no particular way to do this today.

More examples of classification traits that have come up recently in various discussions:

• Snapshot (nee Freeze), which defines logical immutability rather than physical immutability. Rc<int>, for example, would be considered Snapshot. Snapshot could be useful because Snapshot+Clone indicates a type whose value can be safely “preserved” by cloning it.
• NoManaged, a type which does not contain managed data. This might be useful for integrating garbage collection with custom allocators which do not wish to serve as potential roots.
• NoDrop, a type which does not contain an explicit destructor. This can be used to avoid nasty GC quandries.

All three of these (Snapshot, NoManaged, NoDrop) can be easily defined using traits with default impls.

A final, somewhat weaker, motivator is aesthetics. Ownership has allowed us to move threading almost entirely into libraries. The one exception is that the Send and Share types remain built-in. Opt-in traits makes them less built-in, but still requires custom logic in the “impl matching” code as well as special safety checks when Safe or Share are implemented.

After the changes I propose, the only traits which would be specifically understood by the compiler are Copy and Sized. I consider this acceptable, since those two traits are intimately tied to the core Rust type system, unlike Send and Share.

Detailed design

Unsafe traits

Certain traits like Send and Share are critical to memory safety. Nonetheless, it is not feasible to check the thread-safety of all types that implement Send and Share. Therefore, we introduce a notion of an unsafe trait – this is a trait that is unsafe to implement, because implementing it carries semantic guarantees that, if compromised, threaten memory safety in a deep way.

An unsafe trait is declared like so:

unsafe trait Foo { ... }

To implement an unsafe trait, one must mark the impl as unsafe:

unsafe impl Foo for Bar { ... }

Designating an impl as unsafe does not automatically mean that the body of the methods is an unsafe block. Each method in the trait must also be declared as unsafe if it to be considered unsafe.

Unsafe traits are only unsafe to implement. It is always safe to reference an unsafe trait. For example, the following function is safe:

fn foo<T:Send>(x: T) { ... }

It is also safe to opt out of an unsafe trait (as discussed in the next section).

Default and negative impls

We add a notion of a default impl, written:

impl Trait for .. { }

Default impls are subject to various limitations:

1. The default impl must appear in the same module as Trait (or a submodule).
2. Trait must not define any methods.

We further add the notion of a negative impl, written:

impl !Trait for Foo { }

Negative impls are only permitted if Trait has a default impl. Negative impls are subject to the usual orphan rules, but they are permitting to be overlapping. This makes sense because negative impls are not providing an implementation and hence we are not forced to select between them. For similar reasons, negative impls never need to be marked unsafe, even if they reference an unsafe trait.

Intuitively, to check whether a trait Foo that contains a default impl is implemented for some type T, we first check for explicit (positive) impls that apply to T. If any are found, then T implements Foo. Otherwise, we check for negative impls. If any are found, then T does not implement Foo. If neither positive nor negative impls were found, we proceed to check the component types of T (i.e., the types of a struct’s fields) to determine whether all of them implement Foo. If so, then Foo is considered implemented by T.

Oe non-obvious part of the procedure is that, as we recursively examine the component types of T, we add to our list of assumptions that T implements Foo. This allows recursive types like

struct List<T> { data: T, next: Option<List<T>> }

to be checked successfully. Otherwise, we would recursive infinitely. (This procedure is directly analogous to what the existing TypeContents code does.)

Note that there exist types that expand to an infinite tree of types. Such types cannot be successfully checked with a recursive impl; they will simply overflow the builtin depth checking. However, such types also break code generation under monomorphization (we cannot create a finite set of LLVM types that correspond to them) and are in general not supported. Here is an example of such a type:

struct Foo<A> {
    data: Option<Foo<Vec<A>>>
}

The difference between Foo and List above is that Foo<A> references Foo<Vec<A>>, which will then in turn reference Foo<Vec<Vec<A>>> and so on.

Modeling Send and Share using default traits

The Send and Share traits will be modeled entirely in the library as follows. First, we declare the two traits as follows:

unsafe trait Send { }
unsafe impl Send for .. { }

unsafe trait Share { }
unsafe impl Share for .. { }

Both traits are declared as unsafe because declaring that a type if Send and Share has ramifications for memory safety (and data-race freedom) that the compiler cannot, itself, check.

Next, we will add opt out impls of Send and Share for the various unsafe types:

impl<T> !Send for *T { }
impl<T> !Share for *T { }

impl<T> !Send for *mut T { }
impl<T> !Share for *mut T { }

impl<T> !Share for Unsafe<T> { }

Note that it is not necessary to write unsafe to opt out of an unsafe trait, as that is the default state.

Finally, we will add opt in impls of Send and Share for the various safe wrapper types as needed. Here I give one example, which is Mutex. Mutex is interesting because it has the property that it converts a type T from being Sendable to something Sharable:

unsafe impl<T:Send> Send for Mutex<T> { }
unsafe impl<T:Send> Share for Mutex<T> { }

The Copy and Sized traits

The final two builtin traits are Copy and Share. This RFC does not propose any changes to those two traits but rather relies on the specification from the original opt-in RFC.

Controlling copy vs move with the Copy trait

The Copy trait is “opt-in” for user-declared structs and enums. A struct or enum type is considered to implement the Copy trait only if it implements the Copy trait. This means that structs and enums would move by default unless their type is explicitly declared to be Copy. So, for example, the following code would be in error:

struct Point { x: int, y: int }
...
let p = Point { x: 1, y: 2 };
let q = p;  // moves p
print(p.x); // ERROR

To allow that example, one would have to impl Copy for Point:

struct Point { x: int, y: int }
impl Copy for Point { }
...
let p = Point { x: 1, y: 2 };
let q = p;  // copies p, because Point is Pod
print(p.x); // OK

Effectively, there is a three step ladder for types:

1. If you do nothing, your type is linear, meaning that it moves from place to place and can never be copied in any way. (We need a better name for that.)
2. If you implement Clone, your type is cloneable, meaning that it moves from place to place, but it can be explicitly cloned. This is suitable for cases where copying is expensive.
3. If you implement Copy, your type is copyable, meaning that it is just copied by default without the need for an explicit clone. This is suitable for small bits of data like ints or points.

What is nice about this change is that when a type is defined, the user makes an explicit choice between these three options.

Determining whether a type is Sized

Per the DST specification, the array types [T] and object types like Trait are unsized, as are any structs that embed one of those types. The Sized trait can never be explicitly implemented and membership in the trait is always automatically determined.

Matching and coherence for the builtin types Copy and Sized

In general, determining whether a type implements a builtin trait can follow the existing trait matching algorithm, but it will have to be somewhat specialized. The problem is that we are somewhat limited in the kinds of impls that we can write, so some of the implementations we would want must be “hard-coded”.

Specifically we are limited around tuples, fixed-length array types, proc types, closure types, and trait types:

• Fixed-length arrays: A fixed-length array [T, ..n] is Copy if T is Copy. It is always Sized as T is required to be Sized.
• Tuples: A tuple (T_0, ..., T_n) is Copy/Sized depending if, for all i, T_i is Copy/Sized.
• Trait objects (including procs and closures): A trait object type Trait:K (assuming DST here ;) is never Copy nor Sized.

We cannot currently express the above conditions using impls. We may at some point in the future grow the ability to express some of them. For now, though, these “impls” will be hardcoded into the algorithm as if they were written in libstd.

Per the usual coherence rules, since we will have the above impls in libstd, and we will have impls for types like tuples and fixed-length arrays baked in, the only impls that end users are permitted to write are impls for struct and enum types that they define themselves. Although this rule is in the general spirit of the coherence checks, it will have to be written specially.

Design discussion

Why unsafe traits

Without unsafe traits, it would be possible to create data races without using the unsafe keyword:

struct MyStruct { foo: Cell<int> }
impl Share for MyStruct { }

Balancing abstraction, safety, and convenience.

In general, the existence of default traits is anti-abstraction, in the sense that it exposes implementation details a library might prefer to hide. Specifically, adding new private fields can cause your types to become non-sendable or non-sharable, which may break downstream clients without your knowing. This is a known challenge with parallelism: knowing whether it is safe to parallelize relies on implementation details we have traditionally tried to keep secret from clients (often it is said that parallelism is “anti-modular” or “anti-compositional” for this reason).

I think this risk must be weighed against the limitations of requiring total opt in. Requiring total opt in not only means that some types will accidentally fail to implement send or share when they could, but it also means that libraries which wish to employ marker traits cannot be composed with other libraries that are not aware of those marker traits. In effect, opt-in is anti-modular in its own way.

To be more specific, imagine that library A wishes to define a Untainted trait, and it specifically opts out of Untainted for some base set of types. It then wishes to have routines that only operate on Untainted data. Now imagine that there is some other library B that defines a nifty replacement for Vector, NiftyVector. Finally, some library C wishes to use a NiftyVector<uint>, which should not be considered tainted, because it doesn’t reference any tainted strings. However, NiftyVector<uint> does not implement Untainted (nor can it, without either library A or library B knowing about one another). Similar problems arise for any trait, of course, due to our coherence rules, but often they can be overcome with new types. Not so with Send and Share.

Other use cases

Part of the design involves making space for other use cases. I’d like to sketch out how some of those use cases can be implemented briefly. This is not included in the Detailed design section of the RFC because these traits generally concern other features and would be added under RFCs of their own.

Isolating snapshot types. It is useful to be able to identify types which, when cloned, result in a logical snapshot. That is, a value which can never be mutated. Note that there may in fact be mutation under the covers, but this mutation is not visible to the user. An example of such a type is Rc<T> – although the ref count on the Rc may change, the user has no direct access and so Rc<T> is still logically snapshotable. However, not all Rc instances are snapshottable – in particular, something like Rc<Cell<int>> is not.

trait Snapshot { }
impl Snapshot for .. { }

// In general, anything that can reach interior mutability is not
// snapshotable.
impl<T> !Snapshot for Unsafe<T> { }

// But it's ok for Rc<T>.
impl<T:Snapshot> Snapshot for Rc<T> { }

Note that these definitions could all occur in a library. That is, the Rc type itself doesn’t need to know about the Snapshot trait.

Preventing access to managed data. As part of the GC design, we expect it will be useful to write specialized allocators or smart pointers that explicitly do not support tracing, so as to avoid any kind of GC overhead. The general idea is that there should be a bound, let’s call it NoManaged, that indicates that a type cannot reach managed data and hence does not need to be part of the GC’s root set. This trait could be implemented as follows:

unsafe trait NoManaged { }
unsafe impl NoManaged for .. { }
impl<T> !NoManaged for Gc<T> { }

Preventing access to destructors. It is generally recognized that allowing destructors to escape into managed data – frequently referred to as finalizers – is a bad idea. Therefore, we would generally like to ensure that anything is placed into a managed box does not implement the drop trait. Instead, we would prefer to regular the use of drop through a guardian-like API, which basically means that destructors are not asynchronously executed by the GC, as they would be in Java, but rather enqueued for the mutator thread to run synchronously at its leisure. In order to handle this, though, we presumably need some sort of guardian wrapper types that can take a value which has a destructor and allow it to be embedded within managed data. We can summarize this in a trait GcSafe as follows:

unsafe trait GcSafe { }
unsafe impl GcSafe for .. { }

// By default, anything which has drop trait is not GcSafe.
impl<T:Drop> !GcSafe for T { }

// But guardians are, even if `T` has drop.
impl<T> GcSafe for Guardian<T> { }

Why are Copy and Sized different?

The Copy and Sized traits remain builtin to the compiler. This makes sense because they are intimately tied to analyses the compiler performs. For example, the running of destructors and tracking of moves requires knowing which types are Copy. Similarly, the allocation of stack frames need to know whether types are fully Sized. In contrast, sendability and sharability has been fully exported to libraries at this point.

In addition, opting in to Copy makes sense for several reasons:

• Experience has shown that “data-like structs”, for which Copy is most appropriate, are a very small percentage of the total.
• Changing a public API from being copyable to being only movable has a outsized impact on users of the API. It is common however that as APIs evolve they will come to require owned data (like a Vec), even if they do not initially, and hence will change from being copyable to only movable. Opting in to Copy is a way of saying that you never foresee this coming to pass.
• Often it is useful to create linear “tokens” that do not themselves have data but represent permissions. This can be done today using markers but it is awkward. It becomes much more natural under this proposal.

Drawbacks

API stability. The main drawback of this approach over the existing opt-in approach seems to be that a type may be “accidentally” sendable or sharable. I discuss this above under the heading of “balancing abstraction, safety, and convenience”. One point I would like to add here, as it specifically pertains to API stability, is that a library may, if they choose, opt out of Send and Share pre-emptively, in order to “reserve the right” to add non-sendable things in the future.

Alternatives

• The existing opt-in design is of course an alternative.

• We could also simply add the notion of unsafe traits and not default impls and then allow types to unsafely implement Send or Share, bypassing the normal safety guidelines. This gives an escape valve for a downstream client to assert that something is sendable which was not declared as sendable. However, such a solution is deeply unsatisfactory, because it rests on the downstream client making an assertion about the implementation of the library it uses. If that library should be updated, the client’s assumptions could be invalidated, but no compilation errors will result (the impl was already declared as unsafe, after all).

Phasing

Many of the mechanisms described in this RFC are not needed immediately. Therefore, we would like to implement a minimal “forwards compatible” set of changes now and then leave the remaining work for after the 1.0 release. The builtin rules that the compiler currently implements for send and share are quite close to what is proposed in this RFC. The major change is that unsafe pointers and the UnsafeCell type are currently considered sendable.

Therefore, to be forwards compatible in the short term, we can use the same hybrid of builtin and explicit impls for Send and Share that we use for Copy, with the rule that unsafe pointers and UnsafeCell are not considered sendable. We must also implement the unsafe trait and unsafe impl concept.

What this means in practice is that using *const T, *mut T, and UnsafeCell will make a type T non-sendable and non-sharable, and T must then explicitly implement Send or Share.

Unresolved questions

• The terminology of “unsafe trait” seems somewhat misleading, since it seems to suggest that “using” the trait is unsafe, rather than implementing it. One suggestion for an alternate keyword was trusted trait, which might dovetail with the use of trusted to specify a trusted block of code. If we did use trusted trait, it seems that all impls would also have to be trusted impl.
• Perhaps we should declare a trait as a “default trait” directly, rather than using the impl Drop for .. syntax. I don’t know precisely what syntax to use, though.
• Currently, there are special rules relating to object types and the builtin traits. If the “builtin” traits are no longer builtin, we will have to generalize object types to be simply a set of trait references. This is already planned but merits a second RFC. Note that no changes here are required for the 1.0, since the phasing plan dictates that builtin traits remain special until after 1.0.

• Start Date: 2014-03-31
• RFC PR: rust-lang/rfcs#26
• Rust Issue: rust-lang/rust#13535

Summary

This RFC is a proposal to remove the usage of the keyword priv from the Rust language.

Motivation

By removing priv entirely from the language, it significantly simplifies the privacy semantics as well as the ability to explain it to newcomers. The one remaining case, private enum variants, can be rewritten as such:

// pub enum Foo {
//     Bar,
//     priv Baz,
// }

pub enum Foo {
    Bar,
    Baz(BazInner)
}

pub struct BazInner(());

// pub enum Foo2 {
//     priv Bar2,
//     priv Baz2,
// }

pub struct Foo2 {
    variant: FooVariant
}

enum FooVariant {
    Bar2,
    Baz2,
}

Private enum variants are a rarely used feature of the language, and are generally not regarded as a strong enough feature to justify the priv keyword entirely.

Detailed design

There remains only one use case of the priv visibility qualifier in the Rust language, which is to make enum variants private. For example, it is possible today to write a type such as:

pub enum Foo {
    Bar,
    priv Baz
}

In this example, the variant Bar is public, while the variant Baz is private. This RFC would remove this ability to have private enum variants.

In addition to disallowing the priv keyword on enum variants, this RFC would also forbid visibility qualifiers in front of enum variants entirely, as they no longer serve any purpose.

Status of the identifier priv

This RFC would demote the identifier priv from being a keyword to being a reserved keyword (in case we find a use for it in the future).

Alternatives

• Allow private enum variants, as-is today.
• Add a new keyword for enum which means “my variants are all private” with controls to make variants public.

Unresolved questions

• Is the assertion that private enum variants are rarely used true? Are there legitimate use cases for keeping the priv keyword?

• Start Date: 2014-04-05
• RFC PR: rust-lang/rfcs#34
• Rust Issue: rust-lang/rust#15759

Summary

Check all types for well-formedness with respect to the bounds of type variables.

Allow bounds on formal type variable in structs and enums. Check these bounds are satisfied wherever the struct or enum is used with actual type parameters.

Motivation

Makes type checking saner. Catches errors earlier in the development process. Matches behaviour with built-in bounds (I think).

Currently formal type variables in traits and functions may have bounds and these bounds are checked whenever the item is used against the actual type variables. Where these type variables are used in types, these types should be checked for well-formedness with respect to the type definitions. E.g.,

trait U {}
trait T<X: U> {}
trait S<Y> {
    fn m(x: ~T<Y>) {}  // Should be flagged as an error
}

Formal type variables in structs and enums may not have bounds. It is possible to use these type variables in the types of fields, and these types cannot be checked for well-formedness until the struct is instantiated, where each field must be checked.

struct St<X> {
    f: ~T<X>, // Cannot be checked
}

Likewise, impls of structs are not checked. E.g.,

impl<X> St<X> {  // Cannot be checked
    ...
}

Here, no struct can exist where X is replaced by something implementing U, so in the impl, X can be assumed to have the bound U. But the impl does not indicate this. Note, this is sound, but does not indicate programmer intent very well.

Detailed design

Whenever a type is used it must be checked for well-formedness. For polymorphic types we currently check only that the type exists. I would like to also check that any actual type parameters are valid. That is, given a type T<U> where T is declared as T<X: B>, we currently only check that T does in fact exist somewhere (I think we also check that the correct number of type parameters are supplied, in this case one). I would also like to check that U satisfies the bound B.

Work on built-in bounds is (I think) in the process of adding this behaviour for built-in bounds. I would like to apply this to user-specified bounds too.

I think no fewer programs can be expressed. That is, any errors we catch with this new check would have been caught later in the existing scheme, where exactly would depend on where the type was used. The only exception would be if the formal type variable was not used.

We would allow bounds on type variable in structs and enums. Wherever a concrete struct or enum type appears, check the actual type variables against the bounds on the formals (the type well-formedness check).

From the above examples:

trait U {}
trait T<X: U> {}
trait S1<Y> {
    fn m(x: ~T<Y>) {}  //~ ERROR
}
trait S2<Y: U> {
    fn m(x: ~T<Y>) {}
}

struct St<X: U> {
    f: ~T<X>,
}

impl<X: U> St<X> {
    ...
}

Alternatives

Keep the status quo.

We could add bounds on structs, etc. But not check them in impls. This is safe since the implementation is more general than the struct. It would mean we allow impls to be un-necessarily general.

Unresolved questions

Do we allow and check bounds in type aliases? We currently do not. We should probably continue not to since these type variables (and indeed the type aliases) are substituted away early in the type checking process. So if we think of type aliases as almost macro-like, then not checking makes sense. OTOH, it is still a little bit inconsistent.

• Start Date: 2014-04-08
• RFC PR: rust-lang/rfcs#40
• Rust Issue: rust-lang/rust#13851

Summary

Split the current libstd into component libraries, rebuild libstd as a facade in front of these component libraries.

Motivation

Rust as a language is ideal for usage in constrained contexts such as embedding in applications, running on bare metal hardware, and building kernels. The standard library, however, is not quite as portable as the language itself yet. The standard library should be as usable as it can be in as many contexts as possible, without compromising its usability in any context.

This RFC is meant to expand the usability of the standard library into these domains where it does not currently operate easily

Detailed design

In summary, the following libraries would make up part of the standard distribution. Each library listed after the colon are the dependent libraries.

• libmini
• liblibc
• liballoc: libmini liblibc
• libcollections: libmini liballoc
• libtext: libmini liballoc libcollections
• librustrt: libmini liballoc liblibc
• libsync: libmini liballoc liblibc librustrt
• libstd: everything above

libmini

Note: The name libmini warrants bikeshedding. Please consider it a placeholder for the name of this library.

This library is meant to be the core component of all rust programs in existence. This library has very few external dependencies, and is entirely self contained.

Current modules in std which would make up libmini would include the list below. This list was put together by actually stripping down libstd to these modules, so it is known that it is possible for libmini to compile with these modules.

• atomics
• bool
• cast
• char
• clone
• cmp
• container
• default
• finally
• fmt
• intrinsics
• io, stripped down to its core
• iter
• kinds
• mem
• num (and related modules), no float support
• ops
• option
• ptr
• raw
• result
• slice, but without any ~[T] methods
• tuple
• ty
• unit

This list may be a bit surprising, and it’s makeup is discussed below. Note that this makeup is selected specifically to eliminate the need for the dreaded “one off extension trait”. This pattern, while possible, is currently viewed as subpar due to reduced documentation benefit and sharding implementation across many locations.

Strings

In a post-DST world, the string type will actually be a library-defined type, Str (or similarly named). Strings will no longer be a language feature or a language-defined type. This implies that any methods on strings must be in the same crate that defined the Str type, or done through extension traits.

In the spirit of reducing extension traits, the Str type and module were left out of libmini. It’s impossible for libmini to support all methods of Str, so it was entirely removed.

This decision does have ramifications on the implementation of libmini.

• String literals are an open question. In theory, making a string literal would require the Str lang item to be present, but is not present in libmini. That being said, libmini would certainly create many literal strings (for error messages and such). This may be adequately circumvented by having literal strings create a value of type &'static [u8] if the string lang item is not present. While difficult to work with, this may get us 90% of the way there.

• The fmt module must be tweaked for the removal of strings. The only major user-facing detail is that the pad function on Formatter would take a byte-slice and a character length, and then not handle the precision (which truncates the byte slice with a number of characters). This may be overcome by possibly having an extension trait could be added for a Formatter adding a real pad function that takes strings, or just removing the function altogether in favor of str.fmt(formatter).

• The IoError type suffers from the removal of strings. Currently, this type is inhabited with three fields, an enum, a static description string, and an optionally allocated detail string. Removal of strings would imply the IoError type would be just the enum itself. This may be an acceptable compromise to make, defining the IoError type upstream and providing easy constructors from the enum to the struct. Additionally, the OtherIoError enum variant would be extended with an i32 payload representing the error code (if it came from the OS).

• The ascii module is omitted, but it would likely be defined in the crate that defines Str.

Formatting

While not often thought of as “ultra-core” functionality, this module may be necessary because printing information about types is a fundamental problem that normally requires no dependencies.

Inclusion of this module is the reason why I/O is included in the module as well (or at least a few traits), but the module can otherwise be included with little to no overhead required in terms of dependencies.

Neither print! nor format! macros to be a part of this library, but the write! macro would be present.

I/O

The primary reason for defining the io module in the libmini crate would be to implement the fmt module. The ramification of removing strings was previously discussed for IoError, but there are further modifications that would be required for the io module to exist in libmini:

• The Buffer, Listener, Seek, and Acceptor traits would all be defined upstream instead of in libmini. Very little in libstd uses these traits, and nothing in libmini requires them. They are of questionable utility when considering their applicability to all rust code in existence.

• Some extension methods on the Reader and Writer traits would need to be removed. Methods such as push_exact, read_exact, read_to_end, write_line, etc., all require owned vectors or similar unimplemented runtime requirements. These can likely be moved to extension traits upstream defined for all readers and writers. Note that this does not apply to the integral reading and writing methods. These are occasionally overwritten for performance, but removal of some extension methods would strongly suggest to me that these methods should be removed. Regardless, the remaining methods could live in essentially any location.

Slices

The only method lost on mutable slices would currently be the sorting method. This can be circumvented by implementing a sorting algorithm that doesn’t require allocating a temporary buffer. If intensive use of a sorting algorithm is required, Rust can provide a libsort crate with a variety of sorting algorithms apart from the default sorting algorithm.

FromStr

This trait and module are left out because strings are left out. All types in libmini can have their implementation of FromStr in the crate which implements strings

Floats

This current design excludes floats entirely from libmini (implementations of traits and such). This is another questionable decision, but the current implementation of floats heavily leans on functions defined in libm, so it is unacceptable for these functions to exist in libmini.

Either libstd or a libfloat crate will define floating point traits and such.

Failure

It is unacceptable for Option to reside outside of libmini, but it is also also unacceptable for unwrap to live outside of the Option type. Consequently, this means that it must be possible for libmini to fail.

While impossible for libmini to define failure, it should simply be able to declare failure. While currently not possible today, this extension to the language is possible through “weak lang items”.

Implementation-wise, the failure lang item would have a predefined symbol at which it is defined, and libraries which declare but to not define failure are required to only exist in the rlib format. This implies that libmini can only be built as an rlib. Note that today’s linkage rules do not allow for this (because building a dylib with rlib dependencies is not possible), but the rules could be tweaked to allow for this use case.

tl;dr; The implementation of libmini can use failure, but it does not define failure. All usage of libmini would require an implementation of failure somewhere.

liblibc

This library will exist to provide bindings to libc. This will be a highly platform-specific library, containing an entirely separate api depending on which platform it’s being built for.

This crate will be used to provide bindings to the C language in all forms, and would itself essentially be a giant metadata blob. It conceptually represents the inclusion of all C header files.

Note that the funny name of the library is to allow extern crate libc; to be the form of declaration rather than extern crate c; which is consider to be too short for its own good.

Note that this crate can only exist in rlib or dylib form.

liballoc

Note: This name liballoc is questionable, please consider it a placeholder.

This library would define the allocator traits as well as bind to libc malloc/free (or jemalloc if we decide to include it again). This crate would depend on liblibc and libmini.

Pointers such as ~ and Rc would move into this crate using the default allocator. The current Gc pointers would move to libgc if possible, or otherwise librustrt for now (they’re feature gated currently, not super pressing).

Primarily, this library assumes that an allocation failure should trigger a failure. This makes the library not suitable for use in a kernel, but it is suitable essentially everywhere else.

With today’s libstd, this crate would likely mostly be made up by the global_heap module. Its purpose is to define the allocation lang items required by the compiler.

Note that this crate can only exist in rlib form.

libcollections

This crate would not depend on libstd, it would only depend on liballoc and libmini. These two foundational crates should provide all that is necessary to provide a robust set of containers (what you would expect today). Each container would likely have an allocator parameter, and the default would be the default allocator provided by liballoc.

When using the containers from libcollections, it is implicitly assumed that all allocation succeeds, and this will be reflected in the api of each collection.

The contents of this crate would be the entirety of libcollections as it is today, as well as the vec module from the standard library. This would also implement any relevant traits necessary for ~[T].

Note that this crate can only exist in rlib form.

libtext

This crate would define all functionality in rust related to strings. This would contain the definition of the Str type, as well as implementations of the relevant traits from libmini for the string type.

The crucial assumption of this crate is that allocation does not fail, and the rest of the string functionality could be built on top of this. Note that this crate will depend on libcollections for the Vec type as the underlying building block for string buffers and the string type.

This crate would be composed of the str, ascii, and unicode modules which live in libstd today, but would allow for the extension of other text-related functionality.

librustrt

This library would be the crate where the rt module is almost entirely implemented. It will assume that allocation succeeds, and it will assume a libc implementation to run on.

The current libstd modules which would be implemented as part of this crate would be:

• rt
• task
• local_data

Note that comm is not on this list. This crate will additionally define failure (as unwinding for each task). This crate can exist in both rlib and dylib form.

libsync

This library will largely remain what it is today, with the exception that the comm implementation would move into this crate. The purpose of doing so would be to consolidate all concurrency-related primitives in this crate, leaving none out.

This crate would depend on the runtime for task management (scheduling and descheduling).

The libstd facade

A new standard library would be created that would primarily be a facade which would expose the underlying crates as a stable API. This library would depend on all of the above libraries, and would predominately be a grouping of pub use statements.

This library would also be the library to contain the prelude which would include types from the previous crates. All remaining functionality of the standard library would be filled in as part of this crate.

Note that all rust programs will by default link to libstd, and hence will transitively link to all of the upstream crates mentioned above. Many more apis will be exposed through libstd directly, however, such as HashMap, Arc, etc.

The exact details of the makeup of this crate will change over time, but it can be considered as “the current libstd plus more”, and this crate will be the source of the “batteries included” aspect of the rust standard library. The API (reexported paths) of the standard library would not change over time. Once a path is reexported and a release is made, all the path will be forced to remain constant over time.

One of the primary reasons for this facade is to provide freedom to restructure the underlying crates. Once a facade is established, it is the only stable API. The actual structure and makeup of all the above crates will be fluid until an acceptable design is settled on. Note that this fluidity does not apply to libstd, only to the structure of the underlying crates.

Updates to rustdoc

With today’s incarnation of rustdoc, the documentation for this libstd facade would not be as high quality as it is today. The facade would just provide hyperlinks back to the original crates, which would have reduced quantities of documentation in terms of navigation, implemented traits, etc. Additionally, these reexports are meant to be implementation details, not facets of the api. For this reason, rustdoc would have to change in how it renders documentation for libstd.

First, rustdoc would consider a cross-crate reexport as inlining of the documentation (similar to how it inlines reexports of private types). This would allow all documentation in libstd to remain in the same location (even the same urls!). This would likely require extensive changes to rustdoc for when entire module trees are reexported.

Secondly, rustdoc will have to be modified to collect implementors of reexported traits all in one location. When libstd reexports trait X, rustdoc will have to search libstd and all its dependencies for implementors of X, listing them out explicitly.

These changes to rustdoc should place it in a much more presentable space, but it is an open question to what degree these modifications will suffice and how much further rustdoc will have to change.

Remaining crates

There are many more crates in the standard distribution of rust, all of which currently depend on libstd. These crates would continue to depend on libstd as most rust libraries would.

A new effort would likely arise to reduce dependence on the standard library by cutting down to the core dependencies (if necessary). For example, the libnative crate currently depend on libstd, but it in theory doesn’t need to depend on much other than librustrt and liblibc. By cutting out dependencies, new use cases will likely arise for these crates.

Crates outside of the standard distribution of rust will like to link to the above crates as well (and specifically not libstd). For example, crates which only depend on libmini are likely candidates for being used in kernels, whereas crates only depending on liballoc are good candidates for being embedded into other languages. Having a clear delineation for the usability of a crate in various environments seems beneficial.

Alternatives

• There are many alternatives to the above sharding of libstd and its dependent crates. The one that is most rigid is likely libmini, but the contents of all other crates are fairly fluid and able to shift around. To this degree, there are quite a few alternatives in how the remaining crates are organized. The ordering proposed is simply one of many.

• Compilation profiles. Instead of using crate dependencies to encode where a crate can be used, crates could instead be composed of cfg(foo) attributes. In theory, there would be one libstd crate (in terms of source code), and this crate could be compiled with flags such as --cfg libc, --cfg malloc, etc. This route has may have the problem of “multiple standard libraries” in that code compatible with the “libc libstd” is not necessarily compatible with the “no libc libstd”. Asserting that a crate is compatible with multiple profiles would involve requiring multiple compilations.

• Removing libstd entirely. If the standard library is simply a facade, the compiler could theoretically only inject a select number of crates into the prelude, or possibly even omit the prelude altogether. This works towards elimination the question of “does this belong in libstd”, but it would possibly be difficult to juggle the large number of crates to choose from where one could otherwise just look at libstd.

Unresolved questions

• Compile times. It’s possible that having so many upstream crates for each rust crate will increase compile times through reading metadata and invoking the system linker. Would sharding crates still be worth it? Could possible problems that arise be overcome? Would extra monomorphization in all these crates end up causing more binary bloat?

• Binary bloat. Another possible side effect of having many upstream crates would be increasing binary bloat of each rust program. Our current linkage model means that if you use anything from a crate that you get everything in that crate (in terms of object code). It is unknown to what degree this will become a concern, and to what degree it can be overcome.

• Should floats be left out of libmini? This is largely a question of how much runtime support is required for floating point operations. Ideally functionality such as formatting a float would live in libmini, whereas trigonometric functions would live in an external crate with a dependence on libm.

• Is it acceptable for strings to be left out of libmini? Many common operations on strings don’t require allocation. This is currently done out of necessity of having to define the Str type elsewhere, but this may be seen as too limiting for the scope of libmini.

• Does liblibc belong so low in the dependency tree? In the proposed design, only the libmini crate doesn’t depend on liblibc. Crates such as libtext and libcollections, however, arguably have no dependence on libc itself, they simply require some form of allocator. Answering this question would be figuring how how to break liballoc’s dependency on liblibc, but it’s an open question as to whether this is worth it or not.

• Reexporting macros. Currently the standard library defines a number of useful macros which are used throughout the implementation of libstd. There is no way to reexport a macro, so multiple implementations of the same macro would be required for the core libraries to all use the same macro. Is there a better solution to this situation? How much of an impact does this have?

• Start Date: 2014-04-12
• RFC PR: rust-lang/rfcs#42
• Rust Issue: rust-lang/rust#13700

Summary

Add a regexp crate to the Rust distribution in addition to a small regexp_macros crate that provides a syntax extension for compiling regular expressions during the compilation of a Rust program.

The implementation that supports this RFC is ready to receive feedback: https://github.com/BurntSushi/regexp

Documentation for the crate can be seen here: http://burntsushi.net/rustdoc/regexp/index.html

regex-dna benchmark (vs. Go, Python): https://github.com/BurntSushi/regexp/tree/master/benchmark/regex-dna

Other benchmarks (vs. Go): https://github.com/BurntSushi/regexp/tree/master/benchmark

(Perhaps the links should be removed if the RFC is accepted, since I can’t guarantee they will always exist.)

Motivation

Regular expressions provide a succinct method of matching patterns against search text and are frequently used. For example, many programming languages include some kind of support for regular expressions in its standard library.

The outcome of this RFC is to include a regular expression library in the Rust distribution and resolve issue #3591.

Detailed design

(Note: This is describing an existing design that has been implemented. I have no idea how much of this is appropriate for an RFC.)

The first choice that most regular expression libraries make is whether or not to include backreferences in the supported syntax, as this heavily influences the implementation and the performance characteristics of matching text.

In this RFC, I am proposing a library that closely models Russ Cox’s RE2 (either its C++ or Go variants). This means that features like backreferences or generalized zero-width assertions are not supported. In return, we get O(mn) worst case performance (with m being the size of the search text and n being the number of instructions in the compiled expression).

My implementation currently simulates an NFA using something resembling the Pike VM. Future work could possibly include adding a DFA. (N.B. RE2/C++ includes both an NFA and a DFA, but RE2/Go only implements an NFA.)

The primary reason why I chose RE2 was that it seemed to be a popular choice in issue #3591, and its worst case performance characteristics seemed appealing. I was also drawn to the limited set of syntax supported by RE2 in comparison to other regexp flavors.

With that out of the way, there are other things that inform the design of a regexp library.

Unicode

Given the already existing support for Unicode in Rust, this is a no-brainer. Unicode literals should be allowed in expressions and Unicode character classes should be included (e.g., general categories and scripts).

Case folding is also important for case insensitive matching. Currently, this is implemented by converting characters to their uppercase forms and then comparing them. Future work includes applying at least a simple fold, since folding one Unicode character can produce multiple characters.

Normalization is another thing to consider, but like most other regexp libraries, the one I’m proposing here does not do any normalization. (It seems the recommended practice is to do normalization before matching if it’s needed.)

A nice implementation strategy to support Unicode is to implement a VM that matches characters instead of bytes. Indeed, my implementation does this. However, the public API of a regular expression library should expose byte indices corresponding to match locations (which ought to be guaranteed to be UTF8 codepoint boundaries by construction of the VM). My reason for this is that byte indices result in a lower cost abstraction. If character indices are desired, then a mapping can be maintained by the client at their discretion.

Additionally, this makes it consistent with the std::str API, which also exposes byte indices.

Word boundaries, word characters and Unicode

At least Python and D define word characters, word boundaries and space characters with Unicode character classes. My implementation does the same by augmenting the standard Perl character classes \d, \s and \w with corresponding Unicode categories.

Leftmost-first

As of now, my implementation finds the leftmost-first match. This is consistent with PCRE style regular expressions.

I’ve pretty much ignored POSIX, but I think it’s very possible to add leftmost-longest semantics to the existing VM. (RE2 supports this as a parameter, but I believe still does not fully comply with POSIX with respect to picking the correct submatches.)

Public API

There are three main questions that can be asked when searching text:

1. Does the string match this expression?
2. If so, where?
3. Where are its submatches?

In principle, an API could provide a function to only answer (3). The answers to (1) and (2) would immediately follow. However, keeping track of submatches is expensive, so it is useful to implement an optimization that doesn’t keep track of them if it doesn’t have to. For example, submatches do not need to be tracked to answer questions (1) and (2).

The rabbit hole continues: answering (1) can be more efficient than answering (2) because you don’t have to keep track of any capture groups ((2) requires tracking the position of the full match). More importantly, (1) enables early exit from the VM. As soon as a match is found, the VM can quit instead of continuing to search for greedy expressions.

Therefore, it’s worth it to segregate these operations. The performance difference can get even bigger if a DFA were implemented (which can answer (1) and (2) quickly and even help with (3)). Moreover, most other regular expression libraries provide separate facilities for answering these questions separately.

Some libraries (like Python’s re and RE2/C++) distinguish between matching an expression against an entire string and matching an expression against part of the string. My implementation favors simplicity: matching the entirety of a string requires using the ^ and/or $ anchors. In all cases, an implicit .*? is added the beginning and end of each expression evaluated. (Which is optimized out in the presence of anchors.)

Finally, most regexp libraries provide facilities for splitting and replacing text, usually making capture group names available with some sort of $var syntax. My implementation provides this too. (These are a perfect fit for Rust’s iterators.)

This basically makes up the entirety of the public API, in addition to perhaps a quote function that escapes a string so that it may be used as a literal in an expression.

The regexp! macro

With syntax extensions, it’s possible to write an regexp! macro that compiles an expression when a Rust program is compiled. This includes translating the matching algorithm to Rust code specific to the expression given. This “ahead of time” compiling results in a performance increase. Namely, it elides all heap allocation.

I’ve called these “native” regexps, whereas expressions compiled at runtime are “dynamic” regexps. The public API need not impose this distinction on users, other than requiring the use of a syntax extension to construct a native regexp. For example:

let re = regexp!("a*");

After construction, re is indistinguishable from an expression created dynamically:

let re = Regexp::new("a*").unwrap();

In particular, both have the same type. This is accomplished with a representation resembling:

enum MaybeNative {
    Dynamic(~[Inst]),
    Native(fn(MatchKind, &str, uint, uint) -> ~[Option<uint>]),
}

This syntax extension requires a second crate, regexp_macros, where the regexp! macro is defined. Technically, this could be provided in the regexp crate, but this would introduce a runtime dependency on libsyntax for any use of the regexp crate.

@alexcrichton remarks that this state of affairs is a wart that will be corrected in the future.

Untrusted input

Given worst case O(mn) time complexity, I don’t think it’s worth worrying about unsafe search text.

Untrusted regular expressions are another matter. For example, it’s very easy to exhaust a system’s resources with nested counted repetitions. For example, ((a{100}){100}){100} tries to create 100^3 instructions. My current implementation does nothing to mitigate against this, but I think a simple hard limit on the number of instructions allowed would work fine. (Should it be configurable?)

Name

The name of the crate being proposed is regexp and the type describing a compiled regular expression is Regexp. I think an equally good name would be regex (and Regex). Either name seems to be frequently used, e.g., “regexes” or “regexps” in colloquial use. I chose regexp over regex because it matches the name used for the corresponding package in Go’s standard library.

Other possible names are regexpr (and Regexpr) or something with underscores: reg_exp (and RegExp). However, I perceive these to be more ugly and less commonly used than either regexp or regex.

Finally, we could use re (like Python), but I think the name could be ambiguous since it’s so short. regexp (or regex) unequivocally identifies the crate as providing regular expressions.

For consistency’s sake, I propose that the syntax extension provided be named the same as the crate. So in this case, regexp!.

Summary

My implementation is pretty much a port of most of RE2. The syntax should be identical or almost identical. I think matching an existing (and popular) library has benefits, since it will make it easier for people to pick it up and start using it. There will also be (hopefully) fewer surprises. There is also plenty of room for performance improvement by implementing a DFA.

Alternatives

I think the single biggest alternative is to provide a backtracking implementation that supports backreferences and generalized zero-width assertions. I don’t think my implementation precludes this possibility. For example, a backtracking approach could be implemented and used only when features like backreferences are invoked in the expression. However, this gives up the blanket guarantee of worst case O(mn) time. I don’t think I have the wisdom required to voice a strong opinion on whether this is a worthwhile endeavor.

Another alternative is using a binding to an existing regexp library. I think this was discussed in issue #3591 and it seems like people favor a native Rust implementation if it’s to be included in the Rust distribution. (Does the regexp! macro require it? If so, that’s a huge advantage.) Also, a native implementation makes it maximally portable.

Finally, it is always possible to persist without a regexp library.

Unresolved questions

The public API design is fairly simple and straight-forward with no surprises. I think most of the unresolved stuff is how the backend is implemented, which should be changeable without changing the public API (sans adding features to the syntax).

I can’t remember where I read it, but someone had mentioned defining a trait that declared the API of a regexp engine. That way, anyone could write their own backend and use the regexp interface. My initial thoughts are YAGNI—since requiring different backends seems like a super specialized case—but I’m just hazarding a guess here. (If we go this route, then we might want to expose the regexp parser and AST and possibly the compiler and instruction set to make writing your own backend easier. That sounds restrictive with respect to making performance improvements in the future.)

I personally think there’s great value in keeping the standard regexp implementation small, simple and fast. People who have more specialized needs can always pick one of the existing C or C++ libraries.

For now, we could mark the API as #[unstable] or #[experimental].

Future work

I think most of the future work for this crate is to increase the performance, either by implementing different matching algorithms (e.g., a DFA) or by improving the code generator that produces native regexps with regexp!.

If and when a DFA is implemented, care must be taken when creating a code generator, as the size of the code required can grow rapidly.

Other future work (that is probably more important) includes more Unicode support, specifically for simple case folding.

• Start Date: 2014-06-10
• RFC PR: rust-lang/rfcs#48
• Rust Issue: rust-lang/rust#5527

Summary

Cleanup the trait, method, and operator semantics so that they are well-defined and cover more use cases. A high-level summary of the changes is as follows:

1. Generalize explicit self types beyond &self and &mut self etc, so that self-type declarations like self: Rc<Self> become possible.
2. Expand coherence rules to operate recursively and distinguish orphans more carefully.
3. Revise vtable resolution algorithm to be gradual.
4. Revise method resolution algorithm in terms of vtable resolution.

This RFC excludes discussion of associated types and multidimensional type classes, which will be the subject of a follow-up RFC.

Motivation

The current trait system is ill-specified and inadequate. Its implementation dates from a rather different language. It should be put onto a surer footing.

Use cases

Poor interaction with overloadable deref and index

Addressed by: New method resolution algorithm.

The deref operator * is a flexible one. Imagine a pointer p of type ~T. This same * operator can be used for three distinct purposes, depending on context.

1. Create an immutable referent to the referent: &*p.
2. Create a mutable reference to the referent: &mut *p.
3. Copy/move the contents of the referent: consume(*p).

Not all of these operations are supported by all types. In fact, because most smart pointers represent aliasable data, they will only support the creation of immutable references (e.g., Rc, Gc). Other smart pointers (e.g., the RefMut type returned by RefCell) support mutable or immutable references, but not moves. Finally, a type that owns its data (like, indeed, ~T) might support #3.

To reflect this, we use distinct traits for the various operators. (In fact, we don’t currently have a trait for copying/moving the contents, this could be a distinct RFC (ed., I’m still thinking this over myself, there are non-trivial interactions)).

Unfortunately, the method call algorithm can’t really reliably choose mutable vs immutable deref. The challenge is that the proper choice will sometimes not be apparent until quite late in the process. For example, imagine the expression p.foo(): if foo() is defined with &self, we want an immutable deref, otherwise we want a mutable deref.

Note that in this RFC I do not completely address this issue. In particular, in an expression like (*p).foo(), where the dereference is explicit and not automatically inserted, the sense of the dereference is not inferred. For the time being, the sense can be manually specified by making the receiver type fully explicit: (&mut *p).foo() vs (&*p).foo(). I expect in a follow-up RFC to possibly address this problem, as well as the question of how to handle copies and moves of the referent (use #3 in my list above).

Lack of backtracking

Addressed by: New method resolution algorithm.

Issue #XYZ. When multiple traits define methods with the same name, it is ambiguous which trait is being used:

trait Foo { fn method(&self); }
trait Bar { fn method(&self); }

In general, so long as a given type only implements Foo or Bar, these ambiguities don’t present a problem (and ultimately Universal Function Call Syntax or UFCS will present an explicit resolution). However, this is not guaranteed. Sometimes we see “blanket” impls like the following:

impl<A:Base> Foo for A { }

This impl basically says “any type T that implements Base automatically implements Foo”. Now, we expect an ambiguity error if we have a type T that implements both Base and Bar. But in fact, we’ll get an ambiguity error even if a type only implements Bar. The reason for this is that the current method resolution doesn’t “recurse” and check additional dependencies when deciding if an impl is applicable. So it will decide, in this case, that the type T could implement Foo and then record for later that T must implement Base. This will lead to weird errors.

Overly conservative coherence

Addressed by: Expanded coherence rules.

The job of coherence is to ensure that, for any given set of type parameters, a given trait is implemented at most once (it may of course not be implemented at all). Currently, however, coherence is more conservative that it needs to be. This is partly because it doesn’t take into account the very property that it itself is enforcing.

The problems arise due to the “blanket impls” I discussed in the previous section. Consider the following two traits and a blanket impl:

trait Base { }
trait Derived { }
impl<A:Base> Derived for A { }

Here we have two traits Base and Derived, and a blanket impl which implements the Derived trait for any type A that also implements Base.

This implies that if you implement Base for a type S, then S automatically implements Derived:

struct S;
impl Base for S { } // Implement Base => Implements Derived

On a related note, it’d be an error to implement both Base and Derived for the same type T:

// Illegal
struct T;
impl Base for T { }
impl Derived for T { }

This is illegal because now there are two implements of Derived for T. There is the direct one, but also an indirect one. We do not assign either higher precedence, we just report it as an error.

So far, all is in agreement with the current rules. However, problems arise if we imagine a type U that only implements Derived:

struct U;
impl Derived for U { } // Should be OK, currently not.

In this scenario, there is only one implementation of Derived. But the current coherence rules still report it as an error.

Here is a concrete example where a rule like this would be useful. We currently have the Copy trait (aka Pod), which states that a type can be memcopied. We also have the Clone trait, which is a more heavyweight version for types where copying requires allocation. It’d be nice if all types that could be copied could also be cloned – it’d also be nice if we knew for sure that copying a value had the same semantics as cloning it, in that case. We can guarantee both using a blanket impl like the following:

impl<T:Copy> Clone for T {
    fn clone(&self) -> T {
        *self
    }
}

Unfortunately, writing such an impl today would imply that no other types could implement Clone. Obviously a non-starter.

There is one not especially interesting ramification of this. Permitting this rule means that adding impls to a type could cause coherence errors. For example, if I had a type which implements Copy, and I add an explicit implementation of Clone, I’d get an error due to the blanket impl. This could be seen as undesirable (perhaps we’d like to preserve that property that one can always add impls without causing errors).

But of course we already don’t have the property that one can always add impls, since method calls could become ambiguous. And if we were to add “negative bounds”, which might be nice, we’d lose that property. And the popularity and usefulness of blanket impls cannot be denied. Therefore, I think this property (“always being able to add impls”) is not especially useful or important.

Hokey implementation

Addressed by: Gradual vtable resolution algorithm

In an effort to improve inference, the current implementation has a rather ad-hoc two-pass scheme. When performing a method call, it will immediately attempt “early” trait resolution and – if that fails – defer checking until later. This helps with some particular scenarios, such as a trait like:

trait Map<E> {
    fn map(&self, op: |&E| -> E) -> Self;
}

Given some higher-order function like:

fn some_mapping<E,V:Map<E>>(v: &V, op: |&E| -> E) { ... }

If we were then to see a call like:

some_mapping(vec, |elem| ...)

the early resolution would be helpful in connecting the type of elem with the type of vec. The reason to use two phases is that often we don’t need to resolve each trait bound to a specific impl, and if we wait till the end then we will have more type information available.

In my proposed solution, we eliminate the phase distinction. Instead, we simply track pending constraints. We are free to attempt to resolve pending constraints whenever desired. In particular, whenever we find we need more type information to proceed with some type-overloaded operation, rather than reporting an error we can try and resolve pending constraints. If that helps give more information, we can carry on. Once we reach the end of the function, we must then resolve all pending constraints that have not yet been resolved for some other reason.

Note that there is some interaction with the distinction between input and output type parameters discussed in the previous example. Specifically, we must never infer the value of the Self type parameter based on the impls in scope. This is because it would cause crate concatenation to potentially lead to compilation errors in the form of inference failure.

Properties

There are important properties I would like to guarantee:

• Coherence or No Overlapping Instances: Given a trait and values for all of its type parameters, there should always be at most one applicable impl. This should remain true even when unknown, additional crates are loaded.
• Crate concatenation: It should always be possible to take two creates and combine them without causing compilation errors. This property

Here are some properties I do not intend to guarantee:

• Crate divisibility: It is not always possible to divide a crate into two crates. Specifically, this may incur coherence violations due to the orphan rules.
• Decidability: Haskell has various sets of rules aimed at ensuring that the compiler can decide whether a given trait is implemented for a given type. All of these rules wind up preventing useful implementations and thus can be turned off with the undecidable-instances flag. I don’t think decidability is especially important. The compiler can simply keep a recursion counter and report an error if that level of recursion is exceeded. This counter can be adjusted by the user on a crate-by-crate basis if some bizarre impl pattern happens to require a deeper depth to be resolved.

Detailed design

In general, I won’t give a complete algorithmic specification. Instead, I refer readers to the prototype implementation. I would like to write out a declarative and non-algorithmic specification for the rules too, but that is work in progress and beyond the scope of this RFC. Instead, I’ll try to explain in “plain English”.

Method self-type syntax

Currently methods must be declared using the explicit-self shorthands:

fn foo(self, ...)
fn foo(&self, ...)
fn foo(&mut self, ...)
fn foo(~self, ...)

Under this proposal we would keep these shorthands but also permit any function in a trait to be used as a method, so long as the type of the first parameter is either Self or something derefable Self:

fn foo(self: Gc<Self>, ...)
fn foo(self: Rc<Self>, ...)
fn foo(self: Self, ...)      // equivalent to `fn foo(self, ...)
fn foo(self: &Self, ...)     // equivalent to `fn foo(&self, ...)

It would not be required that the first parameter be named self, though it seems like it would be useful to permit it. It’s also possible we can simply make self not be a keyword (that would be my personal preference, if we can achieve it).

Coherence

The coherence rules fall into two categories: the orphan restriction and the overlapping implementations restriction.

trait Show { ... }
impl Show for int { ... }
impl Show for uint { ... }

trait Iterator<E> { ... }
impl Iterator<char> for ~str  { ... }
impl Iterator<u8> for ~str { ... }

trait Clone { ... }
impl<A:Copy> Clone for A { ... }
impl<B:Clone> Clone for ~B { ... }

Since trait resolution is not fully decidable, it is possible to concoct scenarios in which coherence can neither confirm nor deny the possibility that two impls are overlapping. One way for this to happen is when there are two traits which the user knows are mutually exclusive; mutual exclusion is not currently expressible in the type system [7] however, and hence the coherence check will report errors. For example:

trait Even { } // Naturally can't be Even and Odd at once!
trait Odd { }
impl<T:Even> Foo for T { }
impl<T:Odd> Foo for T { }

Another possible scenario is infinite recursion between impls. For example, in the following scenario, the coherence checked would be unable to decide if the following impls overlap:

impl<A:Foo> Bar for A { ... }
impl<A:Bar> Foo for A { ... }

In such cases, the recursion bound is exceeded and an error is conservatively reported. (Note that recursion is not always so easily detected.)

Method resolution

Let us assume the method call is r.m(...) and the type of the receiver r is R. We will resolve the call in two phases. The first phase checks for inherent methods [4] and the second phase for trait methods. Both phases work in a similar way, however. We will just describe how trait method search works and then express the inherent method search in terms of traits.

The core method search looks like this:

METHOD-SEARCH(R, m):
    let TRAITS = the set consisting of any in-scope trait T where:
        1. T has a method m and
        2. R implements T<...> for any values of Ty's type parameters

    if TRAITS is an empty set:
        if RECURSION DEPTH EXCEEDED:
            return UNDECIDABLE
        if R implements Deref<U> for some U:
            return METHOD-SEARCH(U, m)
        return NO-MATCH

    if TRAITS is the singleton set {T}:
        RECONCILE(R, T, m)

    return AMBIGUITY(TRAITS)

Basically, we will continuously auto-dereference the receiver type, searching for some type that implements a trait that offers the method m. This gives precedence to implementations that require fewer autodereferences. (There exists the possibility of a cycle in the Deref chain, so we will only autoderef so many times before reporting an error.)

Receiver reconciliation

Once we find a trait that is implemented for the (adjusted) receiver type R and which offers the method m, we must reconcile the receiver with the self type declared in m. Let me explain by example.

Consider a trait Mob (anyone who ever hacked on the MUD source code will surely remember Mobs!):

trait Mob {
    fn hit_points(&self) -> int;
    fn take_damage(&mut self, damage: int) -> int;
    fn move_to_room(self: GC<Self>, room: &Room);
}

Let’s say we have a type Monster, and Monster implements Mob:

struct Monster { ... }
impl Mob for Monster { ... }

And now we see a call to hit_points() like so:

fn attack(victim: &mut Monster) {
    let hp = victim.hit_points();
    ...
}

Our method search algorithm above will proceed by searching for an implementation of Mob for the type &mut Monster. It won’t find any. It will auto-deref &mut Monster to yield the type Monster and search again. Now we find a match. Thus far, then, we have a single autoderef *victims, yielding the type Monster – but the method hit_points() actually expects a reference (&Monster) to be given to it, not a by-value Monster.

This is where self-type reconciliation steps in. The reconciliation process works by unwinding the adjustments and adding auto-refs:

RECONCILE(R, T, m):
    let E = the expected self type of m in trait T;

    // Case 1.
    if R <: E:
      we're done.

    // Case 2.
    if &R <: E:
      add an autoref adjustment, we're done.

    // Case 3.
    if &mut R <: E:
      adjust R for mutable borrow (if not possible, error).
      add a mut autoref adjustment, we're done.

    // Case 4.
    unwind one adjustment to yield R' (if not possible, error).
    return RECONCILE(R', T, m)

In this case, the expected self type E would be &Monster. We would first check for case 1: is Monster <: &Monster? It is not. We would then proceed to case 2. Is &Monster <: &Monster? It is, and hence add an autoref. The final result then is that victim.hit_points() becomes transformed to the equivalent of (using UFCS notation) Mob::hit_points(&*victim).

To understand case 3, let’s look at a call to take_damage:

fn attack(victim: &mut Monster) {
    let hp = victim.hit_points(); // ...this is what we saw before
    let damage = hp / 10;         // 1/10 of current HP in damage
    victim.take_damage(damage);
    ...
}

As before, we would auto-deref once to find the type Monster. This time, though, the expected self type is &mut Monster. This means that both cases 1 and 2 fail and we wind up at case 3, the test for which succeeds. Now we get to this statement: “adjust R for mutable borrow”.

At issue here is the overloading of the deref operator that was discussed earlier. In this case, the end result we want is Mob::hit_points(&mut *victim), which means that * is being used for a mutable borrow, which is indicated by the DerefMut trait. However, while doing the autoderef loop, we always searched for impls of the Deref trait, since we did not yet know which trait we wanted. [2] We need to patch this up. So this loop will check whether the type &mut Monster implements DerefMut, in addition to just Deref (it does).

This check for case 3 could fail if, e.g., victim had a type like Gc<Monster> or Rc<Monster>. You’d get a nice error message like “the type Rc does not support mutable borrows, and the method take_damage() requires a mutable receiver”.

We still have not seen an example of cases 1 or 4. Let’s use a slightly modified example:

fn flee_if_possible(victim: Gc<Monster>, room: &mut Room) {
  match room.find_random_exit() {
    None => { }
    Some(exit) => {
      victim.move_to_room(exit);
    }
  }
}

As before, we’ll start out with a type of Monster, but this type the method move_to_room() has a receiver type of Gc<Monster>. This doesn’t match cases 1, 2, or 3, so we proceed to case 4 and unwind by one adjustment. Since the most recent adjustment was to deref from Gc<Monster> to Monster, we are left with a type of Gc<Monster>. We now search again. This time, we match case 1. So the final result is Mob::move_to_room(victim, room). This last case is sort of interesting because we had to use the autoderef to find the method, but once resolution is complete we do not wind up dereferencing victim at all.

Finally, let’s see an error involving case 4. Imagine we modified the type of victim in our previous example to be &Monster and not Gc<Monster>:

fn flee_if_possible(victim: &Monster, room: &mut Room) {
  match room.find_random_exit() {
    None => { }
    Some(exit) => {
      victim.move_to_room(exit);
    }
  }
}

In this case, we would again unwind an adjustment, going from Monster to &Monster, but at that point we’d be stuck. There are no more adjustments to unwind and we never found a type Gc<Monster>. Therefore, we report an error like “the method move_to_room() expects a Gc<Monster> but was invoked with an &Monster”.

Inherent methods

Inherent methods can be “desugared” into traits by assuming a trait per struct or enum. Each impl like impl Foo is effectively an implementation of that trait, and all those traits are assumed to be imported and in scope.

Differences from today

Today’s algorithm isn’t really formally defined, but it works very differently from this one. For one thing, it is based purely on subtyping checks, and does not rely on the generic trait matching. This is a crucial limitation that prevents cases like those described in lack of backtracking from working. It also results in a lot of code duplication and a general mess.

Interaction with vtables and type inference

One of the goals of this proposal is to remove the hokey distinction between early and late resolution. The way that this will work now is that, as we execute, we’ll accumulate a list of pending trait obligations. Each obligation is the combination of a trait and set of types. It is called an obligation because, for the method to be correctly typed, we must eventually find an implementation of that trait for those types. Due to type inference, though, it may not be possible to do this right away, since some of the types may not yet be fully known.

The semantics of trait resolution mean that, at any point in time, the type checker is free to stop what it’s doing and try to resolve these pending obligations, so long as none of the input type parameters are unresolved (see below). If it is able to definitely match an impl, this may in turn affect some type variables which are output type parameters. The basic idea then is to always defer doing resolution until we either (a) encounter a point where we need more type information to proceed or (b) have finished checking the function. At those times, we can go ahead and try to do resolution. If, after type checking the function in its entirety, there are still obligations that cannot be definitely resolved, that’s an error.

Ensuring crate concatenation

To ensure crate concentanability, we must only consider the Self type parameter when deciding when a trait has been implemented (more generally, we must know the precise set of input type parameters; I will cover an expanded set of rules for this in a subsequent RFC).

To see why this matters, imagine a scenario like this one:

trait Produce<R> {
    fn produce(&self: Self) -> R;
}

Now imagine I have two crates, C and D. Crate C defines two types, Vector and Real, and specifies a way to combine them:

struct Vector;
impl Produce<int> for Vector { ... }

Now imagine crate C has some code like:

fn foo() {
    let mut v = None;
    loop {
        if v.is_some() {
            let x = v.get().produce(); // (*)
            ...
        } else {
            v = Some(Vector);
        }
    }
}

At the point (*) of the call to produce() we do not yet know the type of the receiver. But the inferencer might conclude that, since it can only see one impl of Produce for Vector, v must have type Vector and hence x must have the type int.

However, then we might find another crate D that adds a new impl:

struct Other;
struct Real;
impl Combine<Real> for Other { ... }

This definition passes the orphan check because at least one of the types (Real, in this case) in the impl is local to the current crate. But what does this mean for our previous inference result? In general, it looks risky to decide types based on the impls we can see, since there could always be more impls we can’t actually see.

It seems clear that this aggressive inference breaks the crate concatenation property. If we combined crates C and D into one crate, then inference would fail where it worked before.

If x were never used in any way that forces it to be an int, then it’s even plausible that the type Real would have been valid in some sense. So the inferencer is influencing program execution to some extent.

Implementation details

The “resolve” algorithm

The basis for the coherence check, method lookup, and vtable lookup algorithms is the same function, called RESOLVE. The basic idea is that it takes a set of obligations and tries to resolve them. The result is four sets:

• CONFIRMED: Obligations for which we were able to definitely select a specific impl.
• NO-IMPL: Obligations which we know can NEVER be satisfied, because there is no specific impl. The only reason that we can ever say this for certain is due to the orphan check.
• DEFERRED: Obligations that we could not definitely link to an impl, perhaps because of insufficient type information.
• UNDECIDABLE: Obligations that were not decidable due to excessive recursion.

In general, if we ever encounter a NO-IMPL or UNDECIDABLE, it’s probably an error. DEFERRED obligations are ok until we reach the end of the function. For details, please refer to the prototype.

Alternatives and downsides

Autoderef and ambiguity

The addition of a Deref trait makes autoderef complicated, because we may encounter situations where the smart pointer and its reference both implement a trait, and we cannot know what the user wanted.

The current rule just decides in favor of the smart pointer; this is somewhat unfortunate because it is likely to not be what the user wanted. It also means that adding methods to smart pointer types is a potentially breaking change. This is particularly problematic because we may want the smart pointer to implement a trait that requires the method in question!

An interesting thought would be to change this rule and say that we always autoderef first and only resolve the method against the innermost reference. Note that UFCS provides an explicit “opt-out” if this is not what was desired. This should also have the (beneficial, in my mind) effect of quelling the over-eager use of Deref for types that are not smart pointers.

This idea appeals to me but I think belongs in a separate RFC. It needs to be evaluated.

Footnotes

Note 3: The restriction that inputs must precede outputs is not strictly necessary. I added it to keep options open concerning associated types and so forth. See the Alternatives section, specifically the section on associated types.

Note 6: In the section on autoderef and ambiguity, I discuss alternate rules that might allow us to lift the requirement that the receiver be named self.

• Start Date: 2014-03-20
• RFC PR: rust-lang/rfcs#49
• Rust Issue: rust-lang/rust#12812

Summary

Allow attributes on match arms.

Motivation

One sometimes wishes to annotate the arms of match statements with attributes, for example with conditional compilation #[cfg]s or with branch weights (the latter is the most important use).

For the conditional compilation, the work-around is duplicating the whole containing function with a #[cfg]. A case study is sfackler’s bindings to OpenSSL, where many distributions remove SSLv2 support, and so that portion of Rust bindings needs to be conditionally disabled. The obvious way to support the various different SSL versions is an enum

pub enum SslMethod {
    #[cfg(sslv2)]
    /// Only support the SSLv2 protocol
    Sslv2,
    /// Only support the SSLv3 protocol
    Sslv3,
    /// Only support the TLSv1 protocol
    Tlsv1,
    /// Support the SSLv2, SSLv3 and TLSv1 protocols
    Sslv23,
}

However, all matchs can only mention Sslv2 when the cfg is active, i.e. the following is invalid:

fn name(method: SslMethod) -> &'static str {
    match method {
        Sslv2 => "SSLv2",
        Sslv3 => "SSLv3",
        _ => "..."
    }
}

A valid method would be to have two definitions: #[cfg(sslv2)] fn name(...) and #[cfg(not(sslv2)] fn name(...). The former has the Sslv2 arm, the latter does not. Clearly, this explodes exponentially for each additional cfg’d variant in an enum.

Branch weights would allow the careful micro-optimiser to inform the compiler that, for example, a certain match arm is rarely taken:

match foo {
    Common => {}
    #[cold]
    Rare => {}
}

Detailed design

Normal attribute syntax, applied to a whole match arm.

match x {
    #[attr]
    Thing => {}

    #[attr]
    Foo | Bar => {}

    #[attr]
    _ => {}
}

Alternatives

There aren’t really any general alternatives; one could probably hack around matching on conditional enum variants with some macros and helper functions to share as much code as possible; but in general this won’t work.

Unresolved questions

Nothing particularly.

• Start Date: 2014-04-18
• RFC PR: rust-lang/rfcs#50
• Rust Issue: rust-lang/rust#13789

Summary

Asserts are too expensive for release builds and mess up inlining. There must be a way to turn them off. I propose macros debug_assert! and assert!. For test cases, assert! should be used.

Motivation

Asserts are too expensive in release builds.

Detailed design

There should be two macros, debug_assert!(EXPR) and assert!(EXPR). In debug builds (without --cfg ndebug), debug_assert!() is the same as assert!(). In release builds (with --cfg ndebug), debug_assert!() compiles away to nothing. The definition of assert!() is if (!EXPR) { fail!("assertion failed ({}, {}): {}", file!(), line!(), stringify!(expr) }

Alternatives

Other designs that have been considered are using debug_assert! in test cases and not providing assert!, but this doesn’t work with separate compilation.

The impact of not doing this is that assert! will be expensive, prompting people will write their own local debug_assert! macros, duplicating functionality that should have been in the standard library.

Unresolved questions

None.

• Start Date: 2014-04-30
• RFC PR: rust-lang/rfcs#59
• Rust Issue: rust-lang/rust#13885

Summary

The tilde (~) operator and type construction do not support allocators and therefore should be removed in favor of the box keyword and a language item for the type.

Motivation

• There will be a unique pointer type in the standard library, Box<T,A> where A is an allocator. The ~T type syntax does not allow for custom allocators. Therefore, in order to keep ~T around while still supporting allocators, we would need to make it an alias for Box<T,Heap>. In the spirit of having one way to do things, it seems better to remove ~ entirely as a type notation.

• ~EXPR and box EXPR are duplicate functionality; the former does not support allocators. Again in the spirit of having one and only one way to do things, I would like to remove ~EXPR.

• Some people think ~ is confusing, as it is less self-documenting than Box.

• ~ can encourage people to blindly add sigils attempting to get their code to compile instead of consulting the library documentation.

Drawbacks

~T may be seen as convenient sugar for a common pattern in some situations.

Detailed design

The ~EXPR production is removed from the language, and all such uses are converted into box.

Add a lang item, box. That lang item will be defined in liballoc (NB: not libmetal/libmini, for bare-metal programming) as follows:

#[lang="box"]
pub struct Box<T,A=Heap>(*T);

All parts of the compiler treat instances of Box<T> identically to the way it treats ~T today.

The destructuring form for Box<T> will be box PAT, as follows:

let box(x) = box(10);
println!("{}", x); // prints 10

Alternatives

The other possible design here is to keep ~T as sugar. The impact of doing this would be that a common pattern would be terser, but I would like to not do this for the reasons stated in “Motivation” above.

Unresolved questions

The allocator design is not yet fully worked out.

It may be possible that unforeseen interactions will appear between the struct nature of Box<T> and the built-in nature of ~T when merged.

• Start Date: 2014-04-30
• RFC PR: rust-lang/rfcs#60
• Rust Issue: rust-lang/rust#14312

Summary

StrBuf should be renamed to String.

Motivation

Since StrBuf is so common, it would benefit from a more traditional name.

Drawbacks

It may be that StrBuf is a better name because it mirrors Java StringBuilder or C# StringBuffer. It may also be that String is confusing because of its similarity to &str.

Detailed design

Rename StrBuf to String.

Alternatives

The impact of not doing this would be that StrBuf would remain StrBuf.

Unresolved questions

None.

• Start Date: 2014-05-02
• RFC PR: rust-lang/rfcs#63
• Rust Issue: rust-lang/rust#14180

Summary

The rules about the places mod foo; can be used are tightened to only permit its use in a crate root and in mod.rs files, to ensure a more sane correspondence between module structure and file system hierarchy. Most notably, this prevents a common newbie error where a module is loaded multiple times, leading to surprising incompatibility between them. This proposal does not take away one’s ability to shoot oneself in the foot should one really desire to; it just removes almost all of the rope, leaving only mixed metaphors.

Motivation

It is a common newbie mistake to write things like this:

lib.rs:

mod foo;
pub mod bar;

foo.rs:

mod baz;

pub fn foo(_baz: baz::Baz) { }

bar.rs:

mod baz;
use foo::foo;

pub fn bar(baz: baz::Baz) {
    foo(baz)
}

baz.rs:

pub struct Baz;

This fails to compile because foo::foo() wants a foo::baz::Baz, while bar::bar() is giving it a bar::baz::Baz.

Such a situation, importing one file multiple times, is exceedingly rarely what the user actually wanted to do, but the present design allows it to occur without warning the user. The alterations contained herein ensure that there is no situation where such double loading can occur without deliberate intent via #[path = "….rs"].

Drawbacks

None known.

Detailed design

When a mod foo; statement is used, the compiler attempts to find a suitable file. At present, it just blindly seeks for foo.rs or foo/mod.rs (relative to the file under parsing).

The new behaviour will only permit mod foo; if at least one of the following conditions hold:

• The file under parsing is the crate root, or

• The file under parsing is a mod.rs, or

• #[path] is specified, e.g. #[path = "foo.rs"] mod foo;.

In layman’s terms, the file under parsing must “own” the directory, so to speak.

Alternatives

The rationale is covered in the summary. This is the simplest repair to the current lack of structure; all alternatives would be more complex and invasive.

One non-invasive alternative is a lint which would detect double loads. This is less desirable than the solution discussed in this RFC as it doesn’t fix the underlying problem which can, fortunately, be fairly easily fixed.

Unresolved questions

None.

• Start Date: 2014-05-04
• RFC PR: rust-lang/rfcs#66
• Rust Issue: rust-lang/rust#15023

Summary

Temporaries live for the enclosing block when found in a let-binding. This only holds when the reference to the temporary is taken directly. This logic should be extended to extend the cleanup scope of any temporary whose lifetime ends up in the let-binding.

For example, the following doesn’t work now, but should:

use std::os;

fn main() {
	let x = os::args().slice_from(1);
	println!("{}", x);
}

Motivation

Temporary lifetimes are a bit confusing right now. Sometimes you can keep references to them, and sometimes you get the dreaded “borrowed value does not live long enough” error. Sometimes one operation works but an equivalent operation errors, e.g. autoref of ~[T] to &[T] works but calling .as_slice() doesn’t. In general it feels as though the compiler is simply being overly restrictive when it decides the temporary doesn’t live long enough.

Drawbacks

I can’t think of any drawbacks.

Detailed design

When a reference to a temporary is passed to a function (either as a regular argument or as the self argument of a method), and the function returns a value with the same lifetime as the temporary reference, the lifetime of the temporary should be extended the same way it would if the function was not invoked.

For example, ~[T].as_slice() takes &'a self and returns &'a [T]. Calling as_slice() on a temporary of type ~[T] will implicitly take a reference &'a ~[T] and return a value &'a [T] This return value should be considered to extend the lifetime of the ~[T] temporary just as taking an explicit reference (and skipping the method call) would.

Alternatives

Don’t do this. We live with the surprising borrowck errors and the ugly workarounds that look like

let x = os::args();
let x = x.slice_from(1);

Unresolved questions

None that I know of.

• Start Date: 2014-06-11
• RFC PR: rust-lang/rfcs#68
• Rust Issue: rust-lang/rust#7362

Summary

Rename *T to *const T, retain all other semantics of unsafe pointers.

Motivation

Currently the T* type in C is equivalent to *mut T in Rust, and the const T* type in C is equivalent to the *T type in Rust. Noticeably, the two most similar types, T* and *T have different meanings in Rust and C, frequently causing confusion and often incorrect declarations of C functions.

If the compiler is ever to take advantage of the guarantees of declaring an FFI function as taking T* or const T* (in C), then it is crucial that the FFI declarations in Rust are faithful to the declaration in C.

The current difference in Rust unsafe pointers types with C pointers types is proving to be too error prone to realistically enable these optimizations at a future date. By renaming Rust’s unsafe pointers to closely match their C brethren, the likelihood for erroneously transcribing a signature is diminished.

Detailed design

This section will assume that the current unsafe pointer design is forgotten completely, and will explain the unsafe pointer design from scratch.

There are two unsafe pointers in rust, *mut T and *const T. These two types are primarily useful when interacting with foreign functions through a FFI. The *mut T type is equivalent to the T* type in C, and the *const T type is equivalent to the const T* type in C.

The type &mut T will automatically coerce to *mut T in the normal locations that coercion occurs today. It will also be possible to explicitly cast with an as expression. Additionally, the &T type will automatically coerce to *const T. Note that &mut T will not automatically coerce to *const T.

The two unsafe pointer types will be freely castable among one another via as expressions, but no coercion will occur between the two. Additionally, values of type uint can be casted to unsafe pointers.

When is a coercion valid?

When coercing from &'a T to *const T, Rust will guarantee that the memory will remain valid for the lifetime 'a and the memory will be immutable up to memory stored in Unsafe<U>. It is the responsibility of the code working with the *const T that the pointer is only dereferenced in the lifetime 'a.

When coercing from &'a mut T to *mut T, Rust will guarantee that the memory will stay valid during 'a and that the memory will not be accessed during 'a. Additionally, Rust will consume the &'a mut T during the coercion. It is the responsibility of the code working with the *mut T to guarantee that the unsafe pointer is only dereferenced in the lifetime 'a, and that the memory is “valid again” after 'a.

Note: Rust will consume &mut T coercions with both implicit and explicit coercions.

The term “valid again” is used to represent that some types in Rust require internal invariants, such as Box<T> never being NULL. This is often a per-type invariant, so it is the responsibility of the unsafe code to uphold these invariants.

When is a safe cast valid?

Unsafe code can convert an unsafe pointer to a safe pointer via dereferencing inside of an unsafe block. This section will discuss when this action is valid.

When converting *mut T to &'a mut T, it must be guaranteed that the memory is initialized to start out with and that nobody will access the memory during 'a except for the converted pointer.

When converting *const T to &'a T, it must be guaranteed that the memory is initialized to start out with and that nobody will write to the pointer during 'a except for memory within Unsafe<U>.

Drawbacks

Today’s unsafe pointers design is consistent with the borrowed pointers types in Rust, using the mut qualifier for a mutable pointer, and no qualifier for an “immutable” pointer. Renaming the pointers would be divergence from this consistency, and would also introduce a keyword that is not used elsewhere in the language, const.

Alternatives

• The current *mut T type could be removed entirely, leaving only one unsafe pointer type, *T. This will not allow FFI calls to take advantage of the const T* optimizations on the caller side of the function. Additionally, this may not accurately express to the programmer what a FFI API is intending to do. Note, however, that other variants of unsafe pointer types could likely be added in the future in a backwards-compatible way.

• More effort could be invested in auto-generating bindings, and hand-generating bindings could be greatly discouraged. This would maintain consistency with Rust pointer types, and it would allow APIs to usually being transcribed accurately by automating the process. It is unknown how realistic this solution is as it is currently not yet implemented. There may still be confusion as well that *T is not equivalent to C’s T*.

Unresolved questions

• How much can the compiler help out when coercing &mut T to *mut T? As previously stated, the source pointer &mut T is consumed during the coercion (it’s already a linear type), but this can lead to some unexpected results:

  extern {
      fn bar(a: *mut int, b: *mut int);
  }
  
  fn foo(a: &mut int) {
      unsafe {
          bar(&mut *a, &mut *a);
      }
  }
  

  This code is invalid because it is creating two copies of the same mutable pointer, and the external function is unaware that the two pointers alias. The rule that the programmer has violated is that the pointer *mut T is only dereferenced during the lifetime of the &'a mut T pointer. For example, here are the lifetimes spelled out:

  fn foo(a: &mut int) {
      unsafe {
          bar(&mut *a, &mut *a);
  //          |-----|  |-----|
  //             |        |
  //             |       Lifetime of second argument
  //            Lifetime of first argument
      }
  }
  

  Here it can be seen that it is impossible for the C code to safely dereference the pointers passed in because lifetimes don’t extend into the function call itself. The compiler could, in this case, extend the lifetime of a coerced pointer to follow the otherwise applied temporary rules for expressions.

  In the example above, the compiler’s temporary lifetime rules would cause the first coercion to last for the entire lifetime of the call to bar, thereby disallowing the second reborrow because it has an overlapping lifetime with the first.

  It is currently an open question how necessary this sort of treatment will be, and this lifetime treatment will likely require a new RFC.

• Will all pointer types in C need to have their own keyword in Rust for representation in the FFI?

• To what degree will the compiler emit metadata about FFI function calls in order to take advantage of optimizations on the caller side of a function call? Do the theoretical wins justify the scope of this redesign? There is currently no concrete data measuring what benefits could be gained from informing optimization passes about const vs non-const pointers.

• Start Date: 2014-05-05
• RFC PR: rust-lang/rfcs#69
• Rust Issue: rust-lang/rust#14646

Summary

Add ASCII byte literals and ASCII byte string literals to the language, similar to the existing (Unicode) character and string literals. Before the RFC process was in place, this was discussed in #4334.

Motivation

Programs dealing with text usually should use Unicode, represented in Rust by the str and char types. In some cases however, a program may be dealing with bytes that can not be interpreted as Unicode as a whole, but still contain ASCII compatible bits.

For example, the HTTP protocol was originally defined as Latin-1, but in practice different pieces of the same request or response can use different encodings. The PDF file format is mostly ASCII, but can contain UTF-16 strings and raw binary data.

There is a precedent at least in Python, which has both Unicode and byte strings.

Drawbacks

The language becomes slightly more complex, although that complexity should be limited to the parser.

Detailed design

Using terminology from the Reference Manual:

Extend the syntax of expressions and patterns to add byte literals of type u8 and byte string literals of type &'static [u8] (or [u8], post-DST). They are identical to the existing character and string literals, except that:

• They are prefixed with a b (for “binary”), to distinguish them. This is similar to the r prefix for raw strings.
• Unescaped code points in the body must be in the ASCII range: U+0000 to U+007F.
• '\x5c' 'u' hex_digit 4 and '\x5c' 'U' hex_digit 8 escapes are not allowed.
• '\x5c' 'x' hex_digit 2 escapes represent a single byte rather than a code point. (They are the only way to express a non-ASCII byte.)

Examples: b'A' == 65u8, b'\t' == 9u8, b'\xFF' == 0xFFu8, b"A\t\xFF" == [65u8, 9, 0xFF]

Assuming buffer of type &[u8]

match buffer[i] {
    b'a' .. b'z' => { /* ... */ }
    c => { /* ... */ }
}

Alternatives

Status quo: patterns must use numeric literals for ASCII values, or (for a single byte, not a byte string) cast to char

match buffer[i] {
    c @ 0x61 .. 0x7A => { /* ... */ }
    c => { /* ... */ }
}
match buffer[i] as char {
    // `c` is of the wrong type!
    c @ 'a' .. 'z' => { /* ... */ }
    c => { /* ... */ }
}

Another option is to change the syntax so that macros such as bytes!() can be used in patterns, and add a byte!() macro:

match buffer[i] {
    c @ byte!('a') .. byte!('z') => { /* ... */ }
    c => { /* ... */ }
}q

This RFC was written to align the syntax with Python, but there could be many variations such as using a different prefix (maybe a for ASCII), or using a suffix instead (maybe u8, as in integer literals).

The code points from syntax could be encoded as UTF-8 rather than being mapped to bytes of the same value, but assuming UTF-8 is not always appropriate when working with bytes.

See also previous discussion in #4334.

Unresolved questions

Should there be “raw byte string” literals? E.g. pdf_file.write(rb"<< /Title (FizzBuzz \(Part one\)) >>")

Should control characters (U+0000 to U+001F) be disallowed in syntax? This should be consistent across all kinds of literals.

Should the bytes!() macro be removed in favor of this?

• Start Date: 2014-05-07
• RFC PR: rust-lang/rfcs#71
• Rust Issue: rust-lang/rust#14181

Summary

Allow block expressions in statics, as long as they only contain items and a trailing const expression.

Example:

static FOO: uint = { 100 };
static BAR: fn() -> int = {
    fn hidden() -> int {
        42
    }
    hidden
};

Motivation

This change allows defining items as part of a const expression, and evaluating to a value using them. This is mainly useful for macros, as it allows hiding complex machinery behind something that expands to a value, but also enables using unsafe {} blocks in a static initializer.

Real life examples include the regex! macro, which currently expands to a block containing a function definition and a value, and would be usable in a static with this.

Another example would be to expose a static reference to a fixed memory address by dereferencing a raw pointer in a const expr, which is useful in embedded and kernel, but requires a unsafe block to do.

The outcome of this is that one additional expression type becomes valid as a const expression, with semantics that are a strict subset of its equivalent in a function.

Drawbacks

Block expressions in a function are usually just used to run arbitrary code before evaluating to a value. Allowing them in statics without allowing code execution might be confusing.

Detailed design

A branch implementing this feature can be found at https://github.com/Kimundi/rust/tree/const_block.

It mainly involves the following changes:

• const check now allows block expressions in statics:
  • All statements that are not item declarations lead to an compile error.
• trans and const eval are made aware of block expressions:
  • A trailing expression gets evaluated as a constant.
  • A missing trailing expressions is treated as a unit value.
• trans is made to recurse into static expressions to generate possible items.

Things like privacy/reachability of definitions inside a static block are already handled more generally at other places, as the situation is very similar to a regular function.

The branch also includes tests that show how this feature works in practice.

Alternatives

Because this feature is a straight forward extension of the valid const expressions, it already causes a very minimal impact on the language, with most alternative ways of enabling the same benefits being more complex.

For example, a expression AST node that can include items but is only usable from procedural macros could be added.

Not having this feature would not prevent anything interesting from getting implemented, but it would lead to less nice looking solutions.

For example, a comparison between static-supporting regex! with and without this feature:

// With this feature, you can just initialize a static:
static R: Regex = regex!("[0-9]");

// Without it, the static needs to be generated by the
// macro itself, alongside all generated items:
regex! {
    static R = "[0-9]";
}

Unresolved questions

None so far.

• Start Date: 2014-05-17
• RFC PR: rust-lang/rfcs#79
• Rust Issue: rust-lang/rust#14309

Summary

Leave structs with unspecified layout by default like enums, for optimisation purposes. Use something like #[repr(C)] to expose C compatible layout.

Motivation

The members of a struct are always laid in memory in the order in which they were specified, e.g.

struct A {
    x: u8,
    y: u64,
    z: i8,
    w: i64,
}

will put the u8 first in memory, then the u64, the i8 and lastly the i64. Due to the alignment requirements of various types padding is often required to ensure the members start at an appropriately aligned byte. Hence the above struct is not 1 + 8 + 1 + 8 == 18 bytes, but rather 1 + 7 + 8 + 1 + 7 + 8 == 32 bytes, since it is laid out like

#[packed] // no automatically inserted padding
struct AFull {
    x: u8,
    _padding1: [u8, .. 7],
    y: u64,
    z: i8,
    _padding2: [u8, .. 7],
    w: i64
}

If the fields were reordered to

struct B {
    y: u64,
    w: i64,

    x: u8,
    i: i8
}

then the struct is (strictly) only 18 bytes (but the alignment requirements of u64 forces it to take up 24).

Having an undefined layout does allow for possible security improvements, like randomising struct fields, but this can trivially be done with a syntax extension that can be attached to a struct to reorder the fields in the AST itself. That said, there may be benefits from being able to randomise all structs in a program automatically/for testing, effectively fuzzing code (especially unsafe code).

Notably, Rust’s enums already have undefined layout, and provide the #[repr] attribute to control layout more precisely (specifically, selecting the size of the discriminant).

Drawbacks

Forgetting to add #[repr(C)] for a struct intended for FFI use can cause surprising bugs and crashes. There is already a lint for FFI use of enums without a #[repr(...)] attribute, so this can be extended to include structs.

Having an unspecified (or otherwise non-C-compatible) layout by default makes interfacing with C slightly harder. A particularly bad case is passing to C a struct from an upstream library that doesn’t have a repr(C) attribute. This situation seems relatively similar to one where an upstream library type is missing an implementation of a core trait e.g. Hash if one wishes to use it as a hashmap key.

It is slightly better if structs had a specified-but-C-incompatible layout, and one has control over the C interface, because then one can manually arrange the fields in the C definition to match the Rust order.

That said, this scenario requires:

• Needing to pass a Rust struct into C/FFI code, where that FFI code actually needs to use things from the struct, rather than just pass it through, e.g., back into a Rust callback.
• The Rust struct is defined upstream & out of your control, and not intended for use with C code.
• The C/FFI code is designed by someone other than that vendor, or otherwise not designed for use with the Rust struct (or else it is a bug in the vendor’s library that the Rust struct can’t be sanely passed to C).

Detailed design

A struct declaration like

struct Foo {
    x: T,
    y: U,
    ...
}

has no fixed layout, that is, a compiler can choose whichever order of fields it prefers.

A fixed layout can be selected with the #[repr] attribute

#[repr(C)]
struct Foo {
    x: T,
    y: U,
    ...
}

This will force a struct to be laid out like the equivalent definition in C.

There would be a lint for the use of non-repr(C) structs in related FFI definitions, for example:

struct UnspecifiedLayout {
   // ...
}

#[repr(C)]
struct CLayout {
   // ...
}


extern {
    fn foo(x: UnspecifiedLayout); // warning: use of non-FFI-safe struct in extern declaration

    fn bar(x: CLayout); // no warning
}

extern "C" fn foo(x: UnspecifiedLayout) { } // warning: use of non-FFI-safe struct in function with C abi.

Alternatives

• Have non-C layouts opt-in, via #[repr(smallest)] and #[repr(random)] (or similar).
• Have layout defined, but not declaration order (like Java(?)), for example, from largest field to smallest, so u8 fields get placed last, and [u8, .. 1000000] fields get placed first. The #[repr] attributes would still allow for selecting declaration-order layout.

Unresolved questions

• How does this interact with binary compatibility of dynamic libraries?
• How does this interact with DST, where some fields have to be at the end of a struct? (Just always lay-out unsized fields last? (i.e. after monomorphisation if a field was originally marked Sized? then it needs to be last).)

• Start Date: 2014-05-21
• RFC PR: rust-lang/rfcs#85
• Rust Issue: rust-lang/rust#14473

Summary

Allow macro expansion in patterns, i.e.

match x {
    my_macro!() => 1,
    _ => 2,
}

Motivation

This is consistent with allowing macros in expressions etc. It’s also a year-old open issue.

I have implemented this feature already and I’m using it to condense some ubiquitous patterns in the HTML parser I’m writing. This makes the code more concise and easier to cross-reference with the spec.

Drawbacks / alternatives

A macro invocation in this position:

match x {
    my_macro!()

could potentially expand to any of three different syntactic elements:

• A pattern, i.e. Foo(x)
• The left side of a match arm, i.e. Foo(x) | Bar(x) if x > 5
• An entire match arm, i.e. Foo(x) | Bar(x) if x > 5 => 1

This RFC proposes only the first of these, but the others would be more useful in some cases. Supporting multiple of the above would be significantly more complex.

Another alternative is to use a macro for the entire match expression, e.g.

my_match!(x {
    my_new_syntax => 1,
    _ => 2,
})

This doesn’t involve any language changes, but requires writing a complicated procedural macro. (My sustained attempts to do things like this with MBE macros have all failed.) Perhaps I could alleviate some of the pain with a library for writing match-like macros, or better use of the existing parser in libsyntax.

The my_match! approach is also not very composable.

Another small drawback: rustdoc can’t document the name of a function argument which is produced by a pattern macro.

Unresolved questions

None, as far as I know.

• Start Date: 2014-05-22
• RFC PR: rust-lang/rfcs#86
• Rust Issue: rust-lang/rust#14637

Summary

Generalize the #[macro_registrar] feature so it can register other kinds of compiler plugins.

Motivation

I want to implement loadable lints and use them for project-specific static analysis passes in Servo. Landing this first will allow more evolution of the plugin system without breaking source compatibility for existing users.

Detailed design

To register a procedural macro in current Rust:

use syntax::ast::Name;
use syntax::parse::token;
use syntax::ext::base::{SyntaxExtension, BasicMacroExpander, NormalTT};

#[macro_registrar]
pub fn macro_registrar(register: |Name, SyntaxExtension|) {
    register(token::intern("named_entities"),
        NormalTT(box BasicMacroExpander {
            expander: named_entities::expand,
            span: None
        },
        None));
}

I propose an interface like

use syntax::parse::token;
use syntax::ext::base::{BasicMacroExpander, NormalTT};

use rustc::plugin::Registry;

#[plugin_registrar]
pub fn plugin_registrar(reg: &mut Registry) {
    reg.register_macro(token::intern("named_entities"),
        NormalTT(box BasicMacroExpander {
            expander: named_entities::expand,
            span: None
        },
        None));
}

Then the struct Registry could provide additional methods such as register_lint as those features are implemented.

It could also provide convenience methods:

use rustc::plugin::Registry;

#[plugin_registrar]
pub fn plugin_registrar(reg: &mut Registry) {
    reg.register_simple_macro("named_entities", named_entities::expand);
}

phase(syntax) becomes phase(plugin), with the former as a deprecated synonym that warns. This is to avoid silent breakage of the very common #[phase(syntax)] extern crate log.

We only need one phase of loading plugin crates, even though the plugins we load may be used at different points (or not at all).

Drawbacks

Breaking change for existing procedural macros.

More moving parts.

Registry is provided by librustc, because it will have methods for registering lints and other librustc things. This means that syntax extensions must link librustc, when before they only needed libsyntax (but could link librustc anyway if desired). This was discussed on the RFC PR and the Rust PR and on IRC.

#![feature(macro_registrar)] becomes unknown, contradicting a comment in feature_gate.rs:

This list can never shrink, it may only be expanded (in order to prevent old programs from failing to compile)

Since when do we ensure that old programs will compile? ;) The #[macro_registrar] attribute wouldn’t work anyway.

Alternatives

We could add #[lint_registrar] etc. alongside #[macro_registrar]. This seems like it will produce more duplicated effort all around. It doesn’t provide convenience methods, and it won’t support API evolution as well.

We could support the old #[macro_registrar] by injecting an adapter shim. This is significant extra work to support a feature with no stability guarantee.

Unresolved questions

Naming bikeshed.

What set of convenience methods should we provide?

• Start Date: 2014-05-22
• RFC PR: rust-lang/rfcs#87
• Rust Issue: rust-lang/rust#12778

Summary

Bounds on trait objects should be separated with +.

Motivation

With DST there is an ambiguity between the following two forms:

trait X {
    fn f(foo: b);
}

and

trait X {
    fn f(Trait: Share);
}

See Rust issue #12778 for details.

Also, since kinds are now just built-in traits, it makes sense to treat a bounded trait object as just a combination of traits. This could be extended in the future to allow objects consisting of arbitrary trait combinations.

Detailed design

Instead of : in trait bounds for first-class traits (e.g. &Trait:Share + Send), we use + (e.g. &Trait + Share + Send).

+ will not be permitted in as without parentheses. This will be done via a special restriction in the type grammar: the special TYPE production following as will be the same as the regular TYPE production, with the exception that it does not accept + as a binary operator.

Drawbacks

• It may be that + is ugly.

• Adding a restriction complicates the type grammar more than I would prefer, but the community backlash against the previous proposal was overwhelming.

Alternatives

The impact of not doing this is that the inconsistencies and ambiguities above remain.

Unresolved questions

Where does the 'static bound fit into all this?

• Start Date: 2014-05-23
• RFC PR: rust-lang/rfcs#89
• Rust Issue: rust-lang/rust#14067

Summary

Allow users to load custom lints into rustc, similar to loadable syntax extensions.

Motivation

There are many possibilities for user-defined static checking:

• Enforcing correct usage of Servo’s JS-managed pointers
• lilyball’s use case: checking that rust-lua functions which call longjmp never have destructors on stack variables
• Enforcing a company or project style guide
• Detecting common misuses of a library, e.g. expensive or non-idiomatic constructs
• In cryptographic code, annotating which variables contain secrets and then forbidding their use in variable-time operations or memory addressing

Existing project-specific static checkers include:

• A Clang plugin that detects misuse of GLib and GObject
• A GCC plugin (written in Python!) that detects misuse of the CPython extension API
• Sparse, which checks Linux kernel code for issues such as mixing up userspace and kernel pointers (often exploitable for privilege escalation)

We should make it easy to build such tools and integrate them with an existing Rust project.

Detailed design

In rustc::lint (which today is rustc::middle::lint):

pub struct Lint {
    /// An identifier for the lint, written with underscores,
    /// e.g. "unused_imports".
    pub name: &'static str,

    /// Default level for the lint.
    pub default_level: Level,

    /// Description of the lint or the issue it detects,
    /// e.g. "imports that are never used"
    pub desc: &'static str,
}

#[macro_export]
macro_rules! declare_lint ( ($name:ident, $level:ident, $desc:expr) => (
    static $name: &'static ::rustc::lint::Lint
        = &::rustc::lint::Lint {
            name: stringify!($name),
            default_level: ::rustc::lint::$level,
            desc: $desc,
        };
))

pub type LintArray = &'static [&'static Lint];

#[macro_export]
macro_rules! lint_array ( ($( $lint:expr ),*) => (
    {
        static array: LintArray = &[ $( $lint ),* ];
        array
    }
))

pub trait LintPass {
    fn get_lints(&self) -> LintArray;

    fn check_item(&mut self, cx: &Context, it: &ast::Item) { }
    fn check_expr(&mut self, cx: &Context, e: &ast::Expr) { }
    ...
}

pub type LintPassObject = Box<LintPass: 'static>;

To define a lint:

#![crate_id="lipogram"]
#![crate_type="dylib"]
#![feature(phase, plugin_registrar)]

extern crate syntax;

// Load rustc as a plugin to get macros
#[phase(plugin, link)]
extern crate rustc;

use syntax::ast;
use syntax::parse::token;
use rustc::lint::{Context, LintPass, LintPassObject, LintArray};
use rustc::plugin::Registry;

declare_lint!(letter_e, Warn, "forbid use of the letter 'e'")

struct Lipogram;

impl LintPass for Lipogram {
    fn get_lints(&self) -> LintArray {
        lint_array!(letter_e)
    }

    fn check_item(&mut self, cx: &Context, it: &ast::Item) {
        let name = token::get_ident(it.ident);
        if name.get().contains_char('e') || name.get().contains_char('E') {
            cx.span_lint(letter_e, it.span, "item name contains the letter 'e'");
        }
    }
}

#[plugin_registrar]
pub fn plugin_registrar(reg: &mut Registry) {
    reg.register_lint_pass(box Lipogram as LintPassObject);
}

A pass which defines multiple lints will have e.g. lint_array!(deprecated, experimental, unstable).

To use a lint when compiling another crate:

#![feature(phase)]

#[phase(plugin)]
extern crate lipogram;

fn hello() { }

fn main() { hello() }

And you will get

test.rs:6:1: 6:15 warning: item name contains the letter 'e', #[warn(letter_e)] on by default
test.rs:6 fn hello() { }
          ^~~~~~~~~~~~~~

Internally, lints are identified by the address of a static Lint. This has a number of benefits:

• The linker takes care of assigning unique IDs, even with dynamically loaded plugins.
• A typo writing a lint ID is usually a compiler error, unlike with string IDs.
• The ability to output a given lint is controlled by the usual visibility mechanism. Lints defined within rustc use the same infrastructure and will simply export their Lints if other parts of the compiler need to output those lints.
• IDs are small and easy to hash.
• It’s easy to go from an ID to name, description, etc.

User-defined lints are controlled through the usual mechanism of attributes and the -A -W -D -F flags to rustc. User-defined lints will show up in -W help if a crate filename is also provided; otherwise we append a message suggesting to re-run with a crate filename.

See also the full demo.

Drawbacks

This increases the amount of code in rustc to implement lints, although it makes each individual lint much easier to understand in isolation.

Loadable lints produce more coupling of user code to rustc internals (with no official stability guarantee, of course).

There’s no scoping / namespacing of the lint name strings used by attributes and compiler flags. Attempting to register a lint with a duplicate name is an error at registration time.

The use of &'static means that lint plugins can’t dynamically generate the set of lints based on some external resource.

Alternatives

We could provide a more generic mechanism for user-defined AST visitors. This could support other use cases like code transformation. But it would be harder to use, and harder to integrate with the lint infrastructure.

It would be nice to magically find all static Lints in a crate, so we don’t need get_lints. Is this worth adding another attribute and another crate metadata type? The plugin::Registry mechanism was meant to avoid such a proliferation of metadata types, but it’s not as declarative as I would like.

Unresolved questions

Do we provide guarantees about visit order for a lint, or the order of multiple lints defined in the same crate? Some lints may require multiple passes.

Should we enforce (while running lints) that each lint printed with span_lint was registered by the corresponding LintPass? Users who particularly care can already wrap lints in modules and use visibility to enforce this statically.

Should we separate registering a lint pass from initializing / constructing the value implementing LintPass? This would support a future where a single rustc invocation can compile multiple crates and needs to reset lint state.

• Start Date: 2014-05-23
• RFC PR: rust-lang/rfcs#90
• Rust Issue: rust-lang/rust#14504

Summary

Simplify Rust’s lexical syntax to make tooling easier to use and easier to define.

Motivation

Rust’s lexer does a lot of work. It un-escapes escape sequences in string and character literals, and parses numeric literals of 4 different bases. It also strips comments, which is sensible, but can be undesirable for pretty printing or syntax highlighting without hacks. Since many characters are allowed in strings both escaped and raw (tabs, newlines, and unicode characters come to mind), after lexing it is impossible to tell if a given character was escaped or unescaped in the source, making the lexer difficult to test against a model.

Detailed design

The following (antlr4) grammar completely describes the proposed lexical syntax:

lexer grammar RustLexer;

/* import Xidstart, Xidcont; */

/* Expression-operator symbols */

EQ      : '=' ;
LT      : '<' ;
LE      : '<=' ;
EQEQ    : '==' ;
NE      : '!=' ;
GE      : '>=' ;
GT      : '>' ;
ANDAND  : '&&' ;
OROR    : '||' ;
NOT     : '!' ;
TILDE   : '~' ;
PLUS    : '+' ;
MINUS   : '-' ;
STAR    : '*' ;
SLASH   : '/' ;
PERCENT : '%' ;
CARET   : '^' ;
AND     : '&' ;
OR      : '|' ;
SHL     : '<<' ;
SHR     : '>>' ;

BINOP
    : PLUS
    | MINUS
    | STAR
    | PERCENT
    | CARET
    | AND
    | OR
    | SHL
    | SHR
    ;

BINOPEQ : BINOP EQ ;

/* "Structural symbols" */

AT         : '@' ;
DOT        : '.' ;
DOTDOT     : '..' ;
DOTDOTDOT  : '...' ;
COMMA      : ',' ;
SEMI       : ';' ;
COLON      : ':' ;
MOD_SEP    : '::' ;
LARROW     : '->' ;
FAT_ARROW  : '=>' ;
LPAREN     : '(' ;
RPAREN     : ')' ;
LBRACKET   : '[' ;
RBRACKET   : ']' ;
LBRACE     : '{' ;
RBRACE     : '}' ;
POUND      : '#';
DOLLAR     : '$' ;
UNDERSCORE : '_' ;

KEYWORD : STRICT_KEYWORD | RESERVED_KEYWORD ;

fragment STRICT_KEYWORD
  : 'as'
  | 'box'
  | 'break'
  | 'continue'
  | 'crate'
  | 'else'
  | 'enum'
  | 'extern'
  | 'fn'
  | 'for'
  | 'if'
  | 'impl'
  | 'in'
  | 'let'
  | 'loop'
  | 'match'
  | 'mod'
  | 'mut'
  | 'once'
  | 'proc'
  | 'pub'
  | 'ref'
  | 'return'
  | 'self'
  | 'static'
  | 'struct'
  | 'super'
  | 'trait'
  | 'true'
  | 'type'
  | 'unsafe'
  | 'use'
  | 'virtual'
  | 'while'
  ;

fragment RESERVED_KEYWORD
  : 'alignof'
  | 'be'
  | 'const'
  | 'do'
  | 'offsetof'
  | 'priv'
  | 'pure'
  | 'sizeof'
  | 'typeof'
  | 'unsized'
  | 'yield'
  ;

// Literals

fragment HEXIT
  : [0-9a-fA-F]
  ;

fragment CHAR_ESCAPE
  : [nrt\\'"0]
  | [xX] HEXIT HEXIT
  | 'u' HEXIT HEXIT HEXIT HEXIT
  | 'U' HEXIT HEXIT HEXIT HEXIT HEXIT HEXIT HEXIT HEXIT
  ;

LIT_CHAR
  : '\'' ( '\\' CHAR_ESCAPE | ~[\\'\n\t\r] ) '\''
  ;

INT_SUFFIX
  : 'i'
  | 'i8'
  | 'i16'
  | 'i32'
  | 'i64'
  | 'u'
  | 'u8'
  | 'u16'
  | 'u32'
  | 'u64'
  ;

LIT_INTEGER
  : [0-9][0-9_]* INT_SUFFIX?
  | '0b' [01][01_]* INT_SUFFIX?
  | '0o' [0-7][0-7_]* INT_SUFFIX?
  | '0x' [0-9a-fA-F][0-9a-fA-F_]* INT_SUFFIX?
  ;

FLOAT_SUFFIX
  : 'f32'
  | 'f64'
  | 'f128'
  ;

LIT_FLOAT
  : [0-9][0-9_]* ('.' | ('.' [0-9][0-9_]*)? ([eE] [-+]? [0-9][0-9_]*)? FLOAT_SUFFIX?)
  ;

LIT_STR
  : '"' ('\\\n' | '\\\r\n' | '\\' CHAR_ESCAPE | .)*? '"'
  ;

/* this is a bit messy */

fragment LIT_STR_RAW_INNER
  : '"' .*? '"'
  | LIT_STR_RAW_INNER2
  ;

fragment LIT_STR_RAW_INNER2
  : POUND LIT_STR_RAW_INNER POUND
  ;

LIT_STR_RAW
  : 'r' LIT_STR_RAW_INNER
  ;

fragment BLOCK_COMMENT
  : '/*' (BLOCK_COMMENT | .)*? '*/'
  ;

COMMENT
  : '//' ~[\r\n]*
  | BLOCK_COMMENT
  ;

IDENT : XID_start XID_continue* ;

LIFETIME : '\'' IDENT ;

WHITESPACE : [ \r\n\t]+ ;

There are a few notable changes from today’s lexical syntax:

• Non-doc comments are not stripped. To compensate, when encountering a COMMENT token the parser can check itself whether or not it’s a doc comment. This can be done with a simple regex: (//(/[^/]|!)|/\*(\*[^*]|!)).
• Numeric literals are not differentiated based on presence of type suffix, nor are they converted from binary/octal/hexadecimal to decimal, nor are underscores stripped. This can be done trivially in the parser.
• Character escapes are not unescaped. That is, if you write ‘\x20’, this lexer will give you LIT_CHAR('\x20') rather than LIT_CHAR(' '). The same applies to string literals.

The output of the lexer then becomes annotated spans – which part of the document corresponds to which token type. Even whitespace is categorized.

Drawbacks

Including comments and whitespace in the token stream is very non-traditional and not strictly necessary.

• Start Date: 2014-06-10
• RFC PR: rust-lang/rfcs#92
• Rust Issue: rust-lang/rust#14803

Summary

Do not identify struct literals by searching for :. Instead define a sub- category of expressions which excludes struct literals and re-define for, if, and other expressions which take an expression followed by a block (or non-terminal which can be replaced by a block) to take this sub-category, instead of all expressions.

Motivation

Parsing by looking ahead is fragile - it could easily be broken if we allow : to appear elsewhere in types (e.g., type ascription) or if we change struct literals to not require the : (e.g., if we allow empty structs to be written with braces, or if we allow struct literals to unify field names to local variable names, as has been suggested in the past and which we currently do for struct literal patterns). We should also be able to give better error messages today if users make these mistakes. More worryingly, we might come up with some language feature in the future which is not predictable now and which breaks with the current system.

Hopefully, it is pretty rare to use struct literals in these positions, so there should not be much fallout. Any problems can be easily fixed by assigning the struct literal into a variable. However, this is a backwards incompatible change, so it should block 1.0.

Detailed design

Here is a simplified version of a subset of Rust’s abstract syntax:

e      ::= x
         | e `.` f
         | name `{` (x `:` e)+ `}`
         | block
         | `for` e `in` e block
         | `if` e block (`else` block)?
         | `|` pattern* `|` e
         | ...
block  ::=  `{` (e;)* e? `}`

Parsing this grammar is ambiguous since x cannot be distinguished from name, so e block in the for expression is ambiguous with the struct literal expression. We currently solve this by using lookahead to find a : token in the struct literal.

I propose the following adjustment:

e      ::= e'
         | name `{` (x `:` e)+ `}`
         | `|` pattern* `|` e
         | ...
e'     ::= x
         | e `.` f
         | block
         | `for` e `in` e' block
         | `if` e' block (`else` block)?
         | `|` pattern* `|` e'
         | ...
block  ::=  `{` (e;)* e? `}`

e' is just e without struct literal expressions. We use e' instead of e wherever e is followed directly by block or any other non-terminal which may have block as its first terminal (after any possible expansions).

For any expressions where a sub-expression is the final lexical element (closures in the subset above, but also unary and binary operations), we require two versions of the meta-expression - the normal one in e and a version with e' for the final element in e'.

Implementation would be simpler, we just add a flag to parser::restriction called RESTRICT_BLOCK or something, which puts us into a mode which reflects e'. We would drop in to this mode when parsing e' position expressions and drop out of it for all but the last sub-expression of an expression.

Drawbacks

It makes the formal grammar and parsing a little more complicated (although it is simpler in terms of needing less lookahead and avoiding a special case).

Alternatives

Don’t do this.

Allow all expressions but greedily parse non-terminals in these positions, e.g., for N {} {} would be parsed as for (N {}) {}. This seems worse because I believe it will be much rarer to have structs in these positions than to have an identifier in the first position, followed by two blocks (i.e., parse as (for N {}) {}).

Unresolved questions

Do we need to expose this distinction anywhere outside of the parser? E.g., macros?

• Start Date: 2014-06-10
• RFC PR: rust-lang/rfcs#93
• Rust Issue: rust-lang/rust#14812

Summary

Remove localization features from format!, and change the set of escapes accepted by format strings. The plural and select methods would be removed, # would no longer need to be escaped, and {{/}} would become escapes for { and }, respectively.

Motivation

Localization is difficult to implement correctly, and doing so will likely not be done in the standard library, but rather in an external library. After talking with others much more familiar with localization, it has come to light that our ad-hoc “localization support” in our format strings are woefully inadequate for most real use cases of support for localization.

Instead of having a half-baked unused system adding complexity to the compiler and libraries, the support for this functionality would be removed from format strings.

Detailed design

The primary localization features that format! supports today are the plural and select methods inside of format strings. These methods are choices made at format-time based on the input arguments of how to format a string. This functionality would be removed from the compiler entirely.

As fallout of this change, the # special character, a back reference to the argument being formatted, would no longer be necessary. In that case, this character no longer needs an escape sequence.

The new grammar for format strings would be as follows:

format_string := <text> [ format <text> ] *
format := '{' [ argument ] [ ':' format_spec ] '}'
argument := integer | identifier

format_spec := [[fill]align][sign]['#'][0][width]['.' precision][type]
fill := character
align := '<' | '>'
sign := '+' | '-'
width := count
precision := count | '*'
type := identifier | ''
count := parameter | integer
parameter := integer '$'

The current syntax can be found at http://doc.rust-lang.org/std/fmt/#syntax to see the diff between the two

Choosing a new escape sequence

Upon landing, there was a significant amount of discussion about the escape sequence that would be used in format strings. Some context can be found on some old pull requests, and the current escape mechanism has been the source of much confusion. With the removal of localization methods, and namely nested format directives, it is possible to reconsider the choices of escaping again.

The only two characters that need escaping in format strings are { and }. One of the more appealing syntaxes for escaping was to double the character to represent the character itself. This would mean that {{ is an escape for a { character, while }} would be an escape for a } character.

Adopting this scheme would avoid clashing with Rust’s string literal escapes. There would be no “double escape” problem. More details on this can be found in the comments of an old PR.

Drawbacks

The localization methods of select/plural are genuinely used for applications that do not involve localization. For example, the compiler and rustdoc often use plural to easily create plural messages. Removing this functionality from format strings would impose a burden of likely dynamically allocating a string at runtime or defining two separate format strings.

Additionally, changing the syntax of format strings is quite an invasive change. Raw string literals serve as a good use case for format strings that must escape the { and } characters. The current system is arguably good enough to pass with for today.

Alternatives

The major localization approach explored has been l20n, which has shown itself to be fairly incompatible with the way format strings work today. Different localization systems, however, have not been explored. Systems such as gettext would be able to leverage format strings quite well, but it was claimed that gettext for localization is inadequate for modern use-cases.

It is also an unexplored possibility whether the current format string syntax could be leveraged by l20n. It is unlikely that time will be allocated to polish off an localization library before 1.0, and it is currently seen as undesirable to have a half-baked system in the libraries rather than a first-class well designed system.

Unresolved questions

• Should localization support be left in std::fmt as a “poor man’s” implementation for those to use as they see fit?

• Start Date: 2014-06-01
• RFC PR: rust-lang/rfcs#100
• Rust Issue: rust-lang/rust#14987

Summary

Add a partial_cmp method to PartialOrd, analogous to cmp in Ord.

Motivation

The Ord::cmp method is useful when working with ordered values. When the exact ordering relationship between two values is required, cmp is both potentially more efficient than computing both a > b and then a < b and makes the code clearer as well.

I feel that in the case of partial orderings, an equivalent to cmp is even more important. I’ve found that it’s very easy to accidentally make assumptions that only hold true in the total order case (for example !(a < b) => a >= b). Explicitly matching against the possible results of the comparison helps keep these assumptions from creeping in.

In addition, the current default implementation setup is a bit strange, as implementations in the partial equality trait assume total equality. This currently makes it easier to incorrectly implement PartialOrd for types that do not have a total ordering, and if PartialOrd is separated from Ord in a way similar to this proposal, the default implementations for PartialOrd will need to be removed and an implementation of the trait will require four repetitive implementations of the required methods.

Detailed design

Add a method to PartialOrd, changing the default implementations of the other methods:

pub trait PartialOrd {
    fn partial_cmp(&self, other: &Self) -> Option<Ordering>;

    fn lt(&self, other: &Self) -> bool {
        match self.partial_cmp(other) {
            Some(Less) => true,
            _ => false,
        }
    }

    fn le(&self, other: &Self) -> bool {
        match self.partial_cmp(other) {
            Some(Less) | Some(Equal) => true,
            _ => false,
        }
    }

    fn gt(&self, other: &Self) -> bool {
        match self.partial_cmp(other) {
            Some(Greater) => true,
            _ => false,
        }
    }

    fn ge(&self, other: &Self) -> bool {
        match self.partial_cmp(other) {
            Some(Greater) | Some(Equal) => true,
            _ => false,
        }
    }
}

Since almost all ordered types have a total ordering, the implementation of partial_cmp is trivial in most cases:

impl PartialOrd for Foo {
    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
        Some(self.cmp(other))
    }
}

This can be done automatically if/when RFC #48 or something like it is accepted and implemented.

Drawbacks

This does add some complexity to PartialOrd. In addition, the more commonly used methods (lt, etc) may become more expensive than they would normally be if their implementations call into partial_ord.

Alternatives

We could invert the default implementations and have a default implementation of partial_cmp in terms of lt and gt. This may slightly simplify things in current Rust, but it makes the default implementation less efficient than it should be. It would also require more work to implement PartialOrd once the currently planned cmp reform has finished as noted above.

partial_cmp could just be called cmp, but it seems like UFCS would need to be implemented first for that to be workable.

Unresolved questions

We may want to add something similar to PartialEq as well. I don’t know what it would be called, though (maybe partial_eq?):

// I don't feel great about these variant names, but `Equal` is already taken
// by `Ordering` which is in the same module.
pub enum Equality {
    AreEqual,
    AreUnequal,
}

pub trait PartialEq {
    fn partial_eq(&self, other: &Self) -> Option<Equality>;

    fn eq(&self, other: &Self) -> bool {
        match self.partial_eq(other) {
            Some(AreEqual) => true,
            _ => false,
        }
    }

    fn neq(&self, other: &Self) -> bool {
        match self.partial_eq(other) {
            Some(AreUnequal) => true,
            _ => false,
        }
    }
}

• Start Date: 2014-06-05
• RFC PR: rust-lang/rfcs#107
• Rust Issue: rust-lang/rust#15287

Summary

Rust currently forbids pattern guards on match arms with move-bound variables. Allowing them would increase the applicability of pattern guards.

Motivation

Currently, if you attempt to use guards on a match arm with a move-bound variable, e.g.

struct A { a: Box<int> }

fn foo(n: int) {
    let x = A { a: box n };
    let y = match x {
        A { a: v } if *v == 42 => v,
        _ => box 0
    };
}

you get an error:

test.rs:6:16: 6:17 error: cannot bind by-move into a pattern guard
test.rs:6         A { a: v } if *v == 42 => v,
                         ^

This should be permitted in cases where the guard only accesses the moved value by reference or copies out of derived paths.

This allows for succinct code with less pattern matching duplication and a minimum number of copies at runtime. The lack of this feature was encountered by @kmcallister when developing Servo’s new HTML 5 parser.

Detailed design

This change requires all occurrences of move-bound pattern variables in the guard to be treated as paths to the values being matched before they are moved, rather than the moved values themselves. Any moves of matched values into the bound variables would occur on the control flow edge between the guard and the arm’s expression. There would be no changes to the handling of reference-bound pattern variables.

The arm would be treated as its own nested scope with respect to borrows, so that pattern-bound variables would be able to be borrowed and dereferenced freely in the guard, but these borrows would not be in scope in the arm’s expression. Since the guard dominates the expression and the move into the pattern-bound variable, moves of either the match’s head expression or any pattern-bound variables in the guard would trigger an error.

The following examples would be accepted:

struct A { a: Box<int> }

impl A {
    fn get(&self) -> int { *self.a }
}

fn foo(n: int) {
    let x = A { a: box n };
    let y = match x {
        A { a: v } if *v == 42 => v,
        _ => box 0
    };
}

fn bar(n: int) {
    let x = A { a: box n };
    let y = match x {
        A { a: v } if x.get() == 42 => v,
        _ => box 0
    };
}

fn baz(n: int) {
    let x = A { a: box n };
    let y = match x {
        A { a: v } if *v.clone() == 42 => v,
        _ => box 0
    };
}

This example would be rejected, due to a double move of v:

struct A { a: Box<int> }

fn foo(n: int) {
    let x = A { a: box n };
    let y = match x {
        A { a: v } if { drop(v); true } => v,
        _ => box 0
    };
}

This example would also be rejected, even though there is no use of the move-bound variable in the first arm’s expression, since the move into the bound variable would be moving the same value a second time:

enum VecWrapper { A(Vec<int>) }

fn foo(x: VecWrapper) -> uint {
    match x {
        A(v) if { drop(v); false } => 1,
        A(v) => v.len()
    }
}

There are issues with mutation of the bound values, but that is true without the changes proposed by this RFC, e.g. Rust issue #14684. The general approach to resolving that issue should also work with these proposed changes.

This would be implemented behind a feature(bind_by_move_pattern_guards) gate until we have enough experience with the feature to remove the feature gate.

Drawbacks

The current error message makes it more clear what the user is doing wrong, but if this change is made the error message for an invalid use of this feature (even if it were accidental) would indicate a use of a moved value, which might be more confusing.

This might be moderately difficult to implement in rustc.

Alternatives

As far as I am aware, the only workarounds for the lack of this feature are to manually expand the control flow of the guard (which can quickly get messy) or use unnecessary copies.

Unresolved questions

This has nontrivial interaction with guards in arbitrary patterns as proposed in #99.

• Start Date: 2014-06-24
• RFC PR: rust-lang/rfcs#109
• Rust Issue: rust-lang/rust#14470

Summary

• Remove the crate_id attribute and knowledge of versions from rustc.
• Add a #[crate_name] attribute similar to the old #[crate_id] attribute
• Filenames will no longer have versions, nor will symbols
• A new flag, --extern, will be used to override searching for external crates
• A new flag, -C metadata=foo, used when hashing symbols

Motivation

The intent of CrateId and its support has become unclear over time as the initial impetus, rustpkg, has faded over time. With cargo on the horizon, doubts have been cast on the compiler’s support for dealing with crate versions and friends. The goal of this RFC is to simplify the compiler’s knowledge about the identity of a crate to allow cargo to do all the necessary heavy lifting.

This new crate identification is designed to not compromise on the usability of the compiler independent of cargo. Additionally, all use cases support today with a CrateId should still be supported.

Detailed design

A new #[crate_name] attribute will be accepted by the compiler, which is the equivalent of the old #[crate_id] attribute, except without the “crate id” support. This new attribute can have a string value describe a valid crate name.

A crate name must be a valid rust identifier with the exception of allowing the - character after the first character.

#![crate_name = "foo"]
#![crate_type = "lib"]

pub fn foo() { /* ... */ }

Naming library filenames

Currently, rustc creates filenames for library following this pattern:

lib<name>-<version>-<hash>.rlib

The current scheme defines <hash> to be the hash of the CrateId value. This naming scheme achieves a number of goals:

• Libraries of the same name can exist next to one another if they have different versions.
• Libraries of the same name and version, but from different sources, can exist next to one another due to having different hashes.
• Rust libraries can have very privileged names such as core and std without worrying about polluting the global namespace of other system libraries.

One drawback of this scheme is that the output filename of the compiler is unknown due to the <hash> component. One must query rustc itself to determine the name of the library output.

Under this new scheme, the new output filenames by the compiler would be:

lib<name>.rlib

Note that both the <version> and the <hash> are missing by default. The <version> was removed because the compiler no longer knows about the version, and the <hash> was removed to make the output filename predictable.

The three original goals can still be satisfied with this simplified naming scheme. As explained in the next section, the compiler’s “glob pattern” when searching for a crate named foo will be libfoo*.rlib, which will help rationalize some of these conclusions.

• Libraries of the same name can exist next to one another because they can be manually renamed to have extra data after the libfoo, such as the version.
• Libraries of the same name and version, but different source, can also exist by modifying what comes after libfoo, such as including a hash.
• Rust does not need to occupy a privileged namespace as the default rust installation would include hashes in all the filenames as necessary. More on this later.

Additionally, with a predictable filename output external tooling should be easier to write.

Loading crates

The goal of the crate loading phase of the compiler is to map a set of extern crate statements to (dylib,rlib) pairs that are present on the filesystem. To do this, the current system matches dependencies via the CrateId syntax:

extern crate json = "super-fast-json#0.1.0";

In today’s compiler, this directive indicates that the a filename of the form libsuper-fast-json-0.1.0-<hash>.rlib must be found to be a candidate. Further checking happens once a candidate is found to ensure that it is indeed a rust library.

Concerns have been raised that this key point of dependency management is where the compiler is doing work that is not necessarily its prerogative. In a cargo-driven world, versions are primarily managed in an external manifest, in addition to doing other various actions such as renaming packages at compile time.

One solution would be to add more version management to the compiler, but this is seen as the compiler delving too far outside what it was initially tasked to do. With this in mind, this is the new proposal for the extern crate syntax:

extern crate json = "super-fast-json";

Notably, the CrateId is removed entirely, along with the version and path associated with it. The string value of the extern crate directive is still optional (defaulting to the identifier), and the string must be a valid crate name (as defined above).

The compiler’s searching and file matching logic would be altered to only match crates based on name. If two versions of a crate are found, the compiler will unconditionally emit an error. It will be up to the user to move the two libraries on the filesystem and control the -L flags to the compiler to enable disambiguation.

This imples that when the compiler is searching for the crate named foo, it will search all of the lookup paths for files which match the pattern libfoo*.{so,rlib}. This is likely to return many false positives, but they will be easily weeded out once the compiler realizes that there is no metadata in the library.

This scheme is strictly less powerful than the previous, but it moves a good deal of logic from the compiler to cargo.

Manually specifying dependencies

Cargo is often seen as “expert mode” in its usage of the compiler. Cargo will always have prior knowledge about what exact versions of a library will be used for any particular dependency, as well as where the outputs are located.

If the compiler provided no support for loading crates beyond matching filenames, it would limit many of cargo’s use cases. For example, cargo could not compile a crate with two different versions of an upstream crate. Additionally, cargo could not substitute libfast-json for libslow-json at compile time (assuming they have the same API).

To accommodate an “expert mode” in rustc, the compiler will grow a new command line flag of the form:

--extern json=path/to/libjson

This directive will indicate that the library json can be found at path/to/libjson. The file extension is not specified, and it is assume that the rlib/dylib pair are located next to one another at this location (libjson is the file stem).

This will enable cargo to drive how the compiler loads crates by manually specifying where files are located and exactly what corresponds to what.

Symbol mangling

Today, mangled symbols contain the version number at the end of the symbol itself. This was originally intended to tie into Linux’s ability to version symbols, but in retrospect this is generally viewed as over-ambitious as the support is not currently there, nor does it work on windows or OSX.

Symbols would no longer contain the version number anywhere within them. The hash at the end of each symbol would only include the crate name and metadata from the command line. Metadata from the command line will be passed via a new command line flag, -C metadata=foo, which specifies a string to hash.

The standard rust distribution

The standard distribution would continue to put hashes in filenames manually because the libraries are intended to occupy a privileged space on the system. The build system would manually move a file after it was compiled to the correct destination filename.

Drawbacks

• The compiler is able to operate fairly well independently of cargo today, and this scheme would hamstring the compiler by limiting the number of “it just works” use cases. If cargo is not being used, build systems will likely have to start using --extern to specify dependencies if name conflicts or version conflicts arise between crates.

• This scheme still has redundancy in the list of dependencies with the external cargo manifest. The source code would no longer list versions, but the cargo manifest will contain the same identifier for each dependency that the source code will contain.

Alternatives

• The compiler could go in the opposite direction of this proposal, enhancing extern crate instead of simplifying it. The compiler could learn about things like version ranges and friends, while still maintaining flags to fine tune its behavior. It is unclear whether this increase in complexity will be paired with a large enough gain in usability of the compiler independent of cargo.

Unresolved questions

• An implementation for the more advanced features of cargo does not currently exist, to it is unknown whether --extern will be powerful enough for cargo to satisfy all its use cases with.

• Are the string literal parts of extern crate justified? Allowing a string literal just for the - character may be overkill.

• Start Date: 2014-06-09
• RFC PR: rust-lang/rfcs#111
• Rust Issue: rust-lang/rust#6515

Summary

Index should be split into Index and IndexMut.

Motivation

Currently, the Index trait is not suitable for most array indexing tasks. The slice functionality cannot be replicated using it, and as a result the new Vec has to use .get() and .get_mut() methods.

Additionally, this simply follows the Deref/DerefMut split that has been implemented for a while.

Detailed design

We split Index into two traits (borrowed from @nikomatsakis):

// self[element] -- if used as rvalue, implicitly a deref of the result
trait Index<E,R> {
    fn index<'a>(&'a self, element: &E) -> &'a R;
}

// &mut self[element] -- when used as a mutable lvalue
trait IndexMut<E,R> {
    fn index_mut<'a>(&'a mut self, element: &E) -> &'a mut R;
}

Drawbacks

• The number of lang. items increases.

• This design doesn’t support moving out of a vector-like object. This can be added backwards compatibly.

• This design doesn’t support hash tables because there is no assignment operator. This can be added backwards compatibly.

Alternatives

The impact of not doing this is that the [] notation will not be available to Vec.

Unresolved questions

None that I’m aware of.

• Start Date: 2014-06-09
• RFC PR: rust-lang/rfcs#112
• Rust Issue: rust-lang/rust#10504

Summary

Remove the coercion from Box<T> to &mut T from the language.

Motivation

Currently, the coercion between Box<T> to &mut T can be a hazard because it can lead to surprising mutation where it was not expected.

Detailed design

The coercion between Box<T> and &mut T should be removed.

Note that methods that take &mut self can still be called on values of type Box<T> without any special referencing or dereferencing. That is because the semantics of auto-deref and auto-ref conspire to make it work: the types unify after one autoderef followed by one autoref.

Drawbacks

Borrowing from Box<T> to &mut T may be convenient.

Alternatives

An alternative is to remove &T coercions as well, but this was decided against as they are convenient.

The impact of not doing this is that the coercion will remain.

Unresolved questions

None.

• Start Date: 2014-07-29
• RFC PR: rust-lang/rfcs#114
• Rust Issue: rust-lang/rust#16095

Summary

• Convert function call a(b, ..., z) into an overloadable operator via the traits Fn<A,R>, FnShare<A,R>, and FnOnce<A,R>, where A is a tuple (B, ..., Z) of the types B...Z of the arguments b...z, and R is the return type. The three traits differ in their self argument (&mut self vs &self vs self).
• Remove the proc expression form and type.
• Remove the closure types (though the form lives on as syntactic sugar, see below).
• Modify closure expressions to permit specifying by-reference vs by-value capture and the receiver type:
  • Specifying by-reference vs by-value closures:
    • ref |...| expr indicates a closure that captures upvars from the environment by reference. This is what closures do today and the behavior will remain unchanged, other than requiring an explicit keyword.
    • |...| expr will therefore indicate a closure that captures upvars from the environment by value. As usual, this is either a copy or move depending on whether the type of the upvar implements Copy.
  • Specifying receiver mode (orthogonal to capture mode above):
    • |a, b, c| expr is equivalent to |&mut: a, b, c| expr
    • |&mut: ...| expr indicates that the closure implements Fn
    • |&: ...| expr indicates that the closure implements FnShare
    • |: a, b, c| expr indicates that the closure implements FnOnce.
• Add syntactic sugar where |T1, T2| -> R1 is translated to a reference to one of the fn traits as follows:
  • |T1, ..., Tn| -> R is translated to Fn<(T1, ..., Tn), R>
  • |&mut: T1, ..., Tn| -> R is translated to Fn<(T1, ..., Tn), R>
  • |&: T1, ..., Tn| -> R is translated to FnShare<(T1, ..., Tn), R>
  • |: T1, ..., Tn| -> R is translated to FnOnce<(T1, ..., Tn), R>

One aspect of closures that this RFC does not describe is that we must permit trait references to be universally quantified over regions as closures are today. A description of this change is described below under Unresolved questions and the details will come in a forthcoming RFC.

Motivation

Over time we have observed a very large number of possible use cases for closures. The goal of this RFC is to create a unified closure model that encompasses all of these use cases.

Specific goals (explained in more detail below):

1. Give control over inlining to users.
2. Support closures that bind by reference and closures that bind by value.
3. Support different means of accessing the closure environment, corresponding to self, &self, and &mut self methods.

As a side benefit, though not a direct goal, the RFC reduces the size/complexity of the language’s core type system by unifying closures and traits.

The core idea: unifying closures and traits

The core idea of the RFC is to unify closures, procs, and traits. There are a number of reasons to do this. First, it simplifies the language, because closures, procs, and traits already served similar roles and there was sometimes a lack of clarity about which would be the appropriate choice. However, in addition, the unification offers increased expressiveness and power, because traits are a more generic model that gives users more control over optimization.

The basic idea is that function calls become an overridable operator. Therefore, an expression like a(...) will be desugar into an invocation of one of the following traits:

trait Fn<A,R> {
    fn call(&mut self, args: A) -> R;
}

trait FnShare<A,R> {
    fn call_share(&self, args: A) -> R;
}

trait FnOnce<A,R> {
    fn call_once(self, args: A) -> R;
}

Essentially, a(b, c, d) becomes sugar for one of the following:

Fn::call(&mut a, (b, c, d))
FnShare::call_share(&a, (b, c, d))
FnOnce::call_once(a, (b, c, d))

To integrate with this, closure expressions are then translated into a fresh struct that implements one of those three traits. The precise trait is currently indicated using explicit syntax but may eventually be inferred.

This change gives user control over virtual vs static dispatch. This works in the same way as generic types today:

fn foo(x: &mut Fn<(int,),int>) -> int {
    x(2) // virtual dispatch
}

fn foo<F:Fn<(int,),int>>(x: &mut F) -> int {
    x(2) // static dispatch
}

The change also permits returning closures, which is not currently possible (the example relies on the proposed impl syntax from rust-lang/rfcs#105):

fn foo(x: impl Fn<(int,),int>) -> impl Fn<(int,),int> {
    |v| x(v * 2)
}

Basically, in this design there is nothing special about a closure. Closure expressions are simply a convenient way to generate a struct that implements a suitable Fn trait.

Bind by reference vs bind by value

When creating a closure, it is now possible to specify whether the closure should capture variables from its environment (“upvars”) by reference or by value. The distinction is indicated using the leading keyword ref:

|| foo(a, b)      // captures `a` and `b` by value

ref || foo(a, b)  // captures `a` and `b` by reference, as today

Reasons to bind by value

Bind by value is useful when creating closures that will escape from the stack frame that created them, such as task bodies (spawn(|| ...)) or combinators. It is also useful for moving values out of a closure, though it should be possible to enable that with bind by reference as well in the future.

Reasons to bind by reference

Bind by reference is useful for any case where the closure is known not to escape the creating stack frame. This frequently occurs when using closures to encapsulate common control-flow patterns:

map.insert_or_update_with(key, value, || ...)
opt_val.unwrap_or_else(|| ...)

In such cases, the closure frequently wishes to read or modify local variables on the enclosing stack frame. Generally speaking, then, such closures should capture variables by-reference – that is, they should store a reference to the variable in the creating stack frame, rather than copying the value out. Using a reference allows the closure to mutate the variables in place and also avoids moving values that are simply read temporarily.

The vast majority of closures in use today are should be “by reference” closures. The only exceptions are those closures that wish to “move out” from an upvar (where we commonly use the so-called “option dance” today). In fact, even those closures could be “by reference” closures, but we will have to extend the inference to selectively identify those variables that must be moved and take those “by value”.

Detailed design

Closure expression syntax

Closure expressions will have the following form (using EBNF notation, where [] denotes optional things and {} denotes a comma-separated list):

CLOSURE = ['ref'] '|' [SELF] {ARG} '|' ['->' TYPE] EXPR
SELF    =  ':' | '&' ':' | '&' 'mut' ':'
ARG     = ID [ ':' TYPE ]

The optional keyword ref is used to indicate whether this closure captures by reference or by value.

Closures are always translated into a fresh struct type with one field per upvar. In a by-value closure, the types of these fields will be the same as the types of the corresponding upvars (modulo &mut reborrows, see below). In a by-reference closure, the types of these fields will be a suitable reference (&, &mut, etc) to the variables being borrowed.

By-value closures

The default form for a closure is by-value. This implies that all upvars which are referenced are copied/moved into the closure as appropriate. There is one special case: if the type of the value to be moved is &mut, we will “reborrow” the value when it is copied into the closure. That is, given an upvar x of type &'a mut T, the value which is actually captured will have type &'b mut T where 'b <= 'a. This rule is consistent with our general treatment of &mut, which is to aggressively reborrow wherever possible; moreover, this rule cannot introduce additional compilation errors, it can only make more programs successfully typecheck.

By-reference closures

A by-reference closure is a convenience form in which values used in the closure are converted into references before being captured. By-reference closures are always rewritable into by-value closures if desired, but the rewrite can often be cumbersome and annoying.

Here is a (rather artificial) example of a by-reference closure in use:

let in_vec: Vec<int> = ...;
let mut out_vec: Vec<int> = Vec::new();
let opt_int: Option<int> = ...;

opt_int.map(ref |v| {
    out_vec.push(v);
    in_vec.fold(v, |a, &b| a + b)
});

This could be rewritten into a by-value closure as follows:

let in_vec: Vec<int> = ...;
let mut out_vec: Vec<int> = Vec::new();
let opt_int: Option<int> = ...;

opt_int.map({
    let in_vec = &in_vec;
    let out_vec = &mut in_vec;
    |v| {
        out_vec.push(v);
        in_vec.fold(v, |a, &b| a + b)
    }
})

In this case, the capture closed over two variables, in_vec and out_vec. As you can see, the compiler automatically infers, for each variable, how it should be borrowed and inserts the appropriate capture.

In the body of a ref closure, the upvars continue to have the same type as they did in the outer environment. For example, the type of a reference to in_vec in the above example is always Vec<int>, whether or not it appears as part of a ref closure. This is not only convenient, it is required to make it possible to infer whether each variable is borrowed as an &T or &mut T borrow.

Note that there are some cases where the compiler internally employs a form of borrow that is not available in the core language, &uniq. This borrow does not permit aliasing (like &mut) but does not require mutability (like &). This is required to allow transparent closing over of &mut pointers as described in this blog post.

Evolutionary note: It is possible to evolve by-reference closures in the future in a backwards compatible way. The goal would be to cause more programs to type-check by default. Two possible extensions follow:

• Detect when values are moved and hence should be taken by value rather than by reference. (This is only applicable to once closures.)
• Detect when it is only necessary to borrow a sub-path. Imagine a closure like ref || use(&context.variable_map). Currently, this closure will borrow context, even though it only uses the field variable_map. As a result, it is sometimes necessary to rewrite the closure to have the form {let v = &context.variable_map; || use(v)}. In the future, however, we could extend the inference so that rather than borrowing context to create the closure, we would borrow context.variable_map directly.

Closure sugar in trait references

The current type for closures, |T1, T2| -> R, will be repurposed as syntactic sugar for a reference to the appropriate Fn trait. This shorthand be used any place that a trait reference is appropriate. The full type will be written as one of the following:

<'a...'z> |T1...Tn|: K -> R
<'a...'z> |&mut: T1...Tn|: K -> R
<'a...'z> |&: T1...Tn|: K -> R
<'a...'z> |: T1...Tn|: K -> R

Each of which would then be translated into the following trait references, respectively:

<'a...'z> Fn<(T1...Tn), R> + K
<'a...'z> Fn<(T1...Tn), R> + K
<'a...'z> FnShare<(T1...Tn), R> + K
<'a...'z> FnOnce<(T1...Tn), R> + K

Note that the bound lifetimes 'a...'z are not in scope for the bound K.

Drawbacks

This model is more complex than the existing model in some respects (but the existing model does not serve the full set of desired use cases).

Alternatives

There is one aspect of the design that is still under active discussion:

Introduce a more generic sugar. It was proposed that we could introduce Trait(A, B) -> C as syntactic sugar for Trait<(A,B),C> rather than retaining the form |A,B| -> C. This is appealing but removes the correspondence between the expression form and the corresponding type. One (somewhat open) question is whether there will be additional traits that mirror fn types that might benefit from this more general sugar.

Tweak trait names. In conjunction with the above, there is some concern that the type name fn(A) -> B for a bare function with no environment is too similar to Fn(A) -> B for a closure. To remedy that, we could change the name of the trait to something like Closure(A) -> B (naturally the other traits would be renamed to match).

Then there are a large number of permutations and options that were largely rejected:

Only offer by-value closures. We tried this and found it required a lot of painful rewrites of perfectly reasonable code.

Make by-reference closures the default. We felt this was inconsistent with the language as a whole, which tends to make “by value” the default (e.g., x vs ref x in patterns, x vs &x in expressions, etc.).

Use a capture clause syntax that borrows individual variables. “By value” closures combined with let statements already serve this role. Simply specifying “by-reference closure” also gives us room to continue improving inference in the future in a backwards compatible way. Moreover, the syntactic space around closures expressions is extremely constrained and we were unable to find a satisfactory syntax, particularly when combined with self-type annotations. Finally, if we decide we do want the ability to have “mostly by-value” closures, we can easily extend the current syntax by writing something like (ref x, ref mut y) || ... etc.

Retain the proc expression form. It was proposed that we could retain the proc expression form to specify a by-value closure and have || expressions be by-reference. Frankly, the main objection to this is that nobody likes the proc keyword.

Use variadic generics in place of tuple arguments. While variadic generics are an interesting addition in their own right, we’d prefer not to introduce a dependency between closures and variadic generics. Having all arguments be placed into a tuple is also a simpler model overall. Moreover, native ABIs on platforms of interest treat a structure passed by value identically to distinct arguments. Finally, given that trait calls have the “Rust” ABI, which is not specified, we can always tweak the rules if necessary (though there are advantages for tooling when the Rust ABI closely matches the native ABI).

Use inference to determine the self type of a closure rather than an annotation. We retain this option for future expansion, but it is not clear whether we can always infer the self type of a closure. Moreover, using inference rather a default raises the question of what to do for a type like |int| -> uint, where inference is not possible.

Default to something other than &mut self. It is our belief that this is the most common use case for closures.

Transition plan

TBD. pcwalton is working furiously as we speak.

Unresolved questions

What relationship should there be between the closure traits? On the one hand, there is clearly a relationship between the traits. For example, given a FnShare, one can easily implement Fn:

impl<A,R,T:FnShare<A,R>> Fn<A,R> for T {
    fn call(&mut self, args: A) -> R {
        (&*self).call_share(args)
    }
}

Similarly, given a Fn or FnShare, you can implement FnOnce. From this, one might derive a subtrait relationship:

trait FnOnce { ... }
trait Fn : FnOnce { ... }
trait FnShare : Fn { ... }

Employing this relationship, however, would require that any manual implement of FnShare or Fn must implement adapters for the other two traits, since a subtrait cannot provide a specialized default of supertrait methods (yet?). On the other hand, having no relationship between the traits limits reuse, at least without employing explicit adapters.

Other alternatives that have been proposed to address the problem:

• Use impls to implement the fn traits in terms of one another, similar to what is shown above. The problem is that we would need to implement FnOnce both for all T where T:Fn and for all T where T:FnShare. This will yield coherence errors unless we extend the language with a means to declare traits as mutually exclusive (which might be valuable, but no such system has currently been proposed nor agreed upon).

• Have the compiler implement multiple traits for a single closure. As with supertraits, this would require manual implements to implement multiple traits. It would also require generic users to write T:Fn+FnMut or else employ an explicit adapter. On the other hand, it preserves the “one method per trait” rule described below.

Can we optimize away the trait vtable? The runtime representation of a reference &Trait to a trait object (and hence, under this proposal, closures as well) is a pair of pointers (data, vtable). It has been proposed that we might be able to optimize this representation to (data, fnptr) so long as Trait has a single function. This slightly improves the performance of invoking the function as one need not indirect through the vtable. The actual implications of this on performance are unclear, but it might be a reason to keep the closure traits to a single method.

Closures that are quantified over lifetimes

A separate RFC is needed to describe bound lifetimes in trait references. For example, today one can write a type like <'a> |&'a A| -> &'a B, which indicates a closure that takes and returns a reference with the same lifetime specified by the caller at each call-site. Note that a trait reference like Fn<(&'a A), &'a B>, while syntactically similar, does not have the same meaning because it lacks the universal quantifier <'a>. Therefore, in the second case, 'a refers to some specific lifetime 'a, rather than being a lifetime parameter that is specified at each callsite. The high-level summary of the change therefore is to permit trait references like <'a> Fn<(&'a A), &'a B>; in this case, the value of <'a> will be specified each time a method or other member of the trait is accessed.

• Start Date: 2014-06-11
• RFC PR: rust-lang/rfcs#115
• Rust Issue: rust-lang/rust#6023

Summary

Currently we use inference to find the current type of otherwise-unannotated integer literals, and when that fails the type defaults to int. This is often felt to be potentially error-prone behavior.

This proposal removes the integer inference fallback and strengthens the types required for several language features that interact with integer inference.

Motivation

With the integer fallback, small changes to code can change the inferred type in unexpected ways. It’s not clear how big a problem this is, but previous experiments1 indicate that removing the fallback has a relatively small impact on existing code, so it’s reasonable to back off of this feature in favor of more strict typing.

See also https://github.com/mozilla/rust/issues/6023.

Detailed design

The primary change here is that, when integer type inference fails, the compiler will emit an error instead of assigning the value the type int.

This change alone will cause a fair bit of existing code to be unable to type check because of lack of constraints. To add more constraints and increase likelihood of unification, we ‘tighten’ up what kinds of integers are required in some situations:

• Array repeat counts must be uint ([expr, .. count])
• << and >> require uint when shifting integral types

Finally, inference for as will be modified to track the types a value is being cast to for cases where the value being cast is unconstrained, like 0 as u8.

Treatment of enum discriminants will need to change:

enum Color { Red = 0, Green = 1, Blue = 2 }

Currently, an unsuffixed integer defaults to int. Instead, we will only require enum discriminants primitive integers of unspecified type; assigning an integer to an enum will behave as if casting from from the type of the integer to an unsigned integer with the size of the enum discriminant.

Drawbacks

This will force users to type hint somewhat more often. In particular, ranges of unsigned ints may need to be type-hinted:

for _ in range(0u, 10) { }

Alternatives

Do none of this.

Unresolved questions

• If we’re putting new restrictions on shift operators, should we change the traits, or just make the primitives special?

• Start Date: 2014-06-12
• RFC PR #: https://github.com/rust-lang/rfcs/pull/116
• Rust Issue #: https://github.com/rust-lang/rust/issues/16464

Summary

Remove or feature gate the shadowing of view items on the same scope level, in order to have less complicated semantic and be more future proof for module system changes or experiments.

This means the names brought in scope by extern crate and use may never collide with each other, nor with any other item (unless they live in different namespaces). Eg, this will no longer work:

extern crate foo;
use foo::bar::foo; // ERROR: There is already a module `foo` in scope

Shadowing would still be allowed in case of lexical scoping, so this continues to work:

extern crate foo;

fn bar() {
    use foo::bar::foo; // Shadows the outer foo

    foo::baz();
}

Definitions

Due to a certain lack of official, clearly defined semantics and terminology, a list of relevant definitions is included:

• Scope A scope in Rust is basically defined by a block, following the rules of lexical scoping:

  scope 1 (visible: scope 1)
  {
        scope 1-1 (visible: scope 1, scope 1-1)
        {
            scope 1-1-1 (visible: scope 1, scope 1-1, scope 1-1-1)
        }
        scope 1-1
        {
            scope 1-1-2
        }
        scope 1-1
  }
  scope 1
  

  Blocks include block expressions, fn items and mod items, but not things like extern, enum or struct. Additionally, mod is special in that it isolates itself from parent scopes.

• Scope Level Anything with the same name in the example above is on the same scope level. In a scope level, all names defined in parent scopes are visible, but can be shadowed by a new definition with the same name, which will be in scope for that scope itself and all its child scopes.

• Namespace Rust has different namespaces, and the scoping rules apply to each one separately. The exact number of different namespaces is not well defined, but they are roughly

  • types (enum Foo {})
  • modules (mod foo {})
  • item values (static FOO: uint = 0;)
  • local values (let foo = 0;)
  • lifetimes (impl<'a> ...)
  • macros (macro_rules! foo {...})

• Definition Item Declarations that create new entities in a crate are called (by the author) definition items. They include struct, enum, mod, fn, etc. Each of them creates a name in the type, module, item value or macro namespace in the same scope level they are written in.

• View Item Declarations that just create aliases to existing declarations in a crate are called view items. They include use and extern crate, and also create a name in the type, module, item value or macro namespace in the same scope level they are written in.

• Item Both definition items and view items together are collectively called items.

• Shadowing While the principle of shadowing exists in all namespaces, there are different forms of it:

  • item-style: Declarations shadow names from outer scopes, and are visible everywhere in their own, including lexically before their own definition. This requires there to be only one definition with the same name and namespace per scope level. Types, modules, item values and lifetimes fall under these rules.
  • sequentially: Declarations shadow names that are lexically before them, both in parent scopes and their own. This means you can reuse the same name in the same scope, but a definition will not be visibly before itself. This is how local values and macros work. (Due to sequential code execution and parsing, respectively)
  • view item: A special case exists with view items; In the same scope level, extern crate creates entries in the module namespace, which are shadowable by names created with use, which are shadowable with any definition item. The singular goal of this RFC is to remove this shadowing behavior of view items

Motivation

As explained above, what is currently visible under which namespace in a given scope is determined by a somewhat complicated three step process:

1. First, every extern crate item creates a name in the module namespace.
2. Then, every use can create a name in any namespace, shadowing the extern crate ones.
3. Lastly, any definition item can shadow any name brought in scope by both extern crate and use.

These rules have developed mostly in response to the older, more complicated import system, and the existence of wildcard imports (use foo::*). In the case of wildcard imports, this shadowing behavior prevents local code from breaking if the source module gets updated to include new names that happen to be defined locally.

However, wildcard imports are now feature gated, and name conflicts in general can be resolved by using the renaming feature of extern crate and use, so in the current non-gated state of the language there is no need for this shadowing behavior.

Gating it off opens the door to remove it altogether in a backwards compatible way, or to re-enable it in case wildcard imports are officially supported again.

It also makes the mental model around items simpler: Any shadowing of items happens through lexical scoping only, and every item can be considered unordered and mutually recursive.

If this RFC gets accepted, a possible next step would be a RFC to lift the ordering restriction between extern crate, use and definition items, which would make them truly behave the same in regard to shadowing and the ability to be reordered. It would also lift the weirdness of use foo::bar; mod foo;.

Implementing this RFC would also not change anything about how name resolution works, as its just a tightening of the existing rules.

Drawbacks

• Feature gating import shadowing might break some code using #[feature(globs)].
• The behavior of libstds prelude becomes more magical if it still allows shadowing, but this could be de-magified again by a new feature, see below in unresolved questions.
• Or the utility of libstds prelude becomes more restricted if it doesn’t allow shadowing.

Detailed design

A new feature gate import_shadowing gets created.

During the name resolution phase of compilation, every time the compiler detects a shadowing between extern crate, use and definition items in the same scope level, it bails out unless the feature gate got enabled. This amounts to two rules:

• Items in the same scope level and either the type, module, item value or lifetime namespace may not shadow each other in the respective namespace.
• Items may shadow names from outer scopes in any namespace.

Just like for the globs feature, the libstd prelude import would be preempt from this, and still be allowed to be shadowed.

Alternatives

The alternative is to do nothing, and risk running into a backwards compatibility hazard, or committing to make a final design decision around the whole module system before 1.0 gets released.

Unresolved questions

• It is unclear how the libstd preludes fits into this.

  On the one hand, it basically acts like a hidden use std::prelude::*; import which ignores the globs feature, so it could simply also ignore the import_shadowing feature as well, and the rule becomes that the prelude is a magic compiler feature that injects imports into every module but doesn’t prevent the user from taking the same names.

  On the other hand, it is also thinkable to simply forbid shadowing of prelude items as well, as defining things with the same name as std exports is not recommended anyway, and this would nicely enforce that. It would however mean that the prelude can not change without breaking backwards compatibility, which might be too restricting.

  A compromise would be to specialize wildcard imports into a new prelude use feature, which has the explicit properties of being shadow-able and using a wildcard import. libstds prelude could then simply use that, and users could define and use their own preludes as well. But that’s a somewhat orthogonal feature, and should be discussed in its own RFC.

• Interaction with overlapping imports.

  Right now its legal to write this:

fn main() { use Bar = std::result::Result; use Bar = std::option::Option; let x: Bar = None; }

where the latter `use` shadows the former. This would have to be forbidden as well,
however the current semantic seems like a accident anyway.

• Start Date: 2014-06-15
• RFC PR #: rust-lang/rfcs#123
• Rust Issue #: rust-lang/rust#16281

Summary

Rename the Share trait to Sync

Motivation

With interior mutability, the name “immutable pointer” for a value of type &T is not quite accurate. Instead, the term “shared reference” is becoming popular to reference values of type &T. The usage of the term “shared” is in conflict with the Share trait, which is intended for types which can be safely shared concurrently with a shared reference.

Detailed design

Rename the Share trait in std::kinds to Sync. Documentation would refer to &T as a shared reference and the notion of “shared” would simply mean “many references” while Sync implies that it is safe to share among many threads.

Drawbacks

The name Sync may invoke conceptions of “synchronized” from languages such as Java where locks are used, rather than meaning “safe to access in a shared fashion across tasks”.

Alternatives

As any bikeshed, there are a number of other names which could be possible for this trait:

• Concurrent
• Synchronized
• Threadsafe
• Parallel
• Threaded
• Atomic
• DataRaceFree
• ConcurrentlySharable

Unresolved questions

None.

• Start Date: 2014-07-29
• RFC PR: rust-lang/rfcs#130
• Rust Issue: rust-lang/rust#16094

Summary

Remove special treatment of Box<T> from the borrow checker.

Motivation

Currently the Box<T> type is special-cased and converted to the old ~T internally. This is mostly invisible to the user, but it shows up in some places that give special treatment to Box<T>. This RFC is specifically concerned with the fact that the borrow checker has greater precision when dereferencing Box<T> vs other smart pointers that rely on the Deref traits. Unlike the other kinds of special treatment, we do not currently have a plan for how to extend this behavior to all smart pointer types, and hence we would like to remove it.

Here is an example that illustrates the extra precision afforded to Box<T> vs other types that implement the Deref traits. The following program, written using the Box type, compiles successfully:

struct Pair {
    a: uint,
    b: uint
}

fn example1(mut smaht: Box<Pair>) {
    let a = &mut smaht.a;
    let b = &mut smaht.b;
    ...
}

This program compiles because the type checker can see that (*smaht).a and (*smaht).b are always distinct paths. In contrast, if I use a smart pointer, I get compilation errors:

fn example2(cell: RefCell<Pair>) {
    let mut smaht: RefMut<Pair> = cell.borrow_mut();
    let a = &mut smaht.a;
    
    // Error: cannot borrow `smaht` as mutable more than once at a time
    let b = &mut smaht.b;
}

To see why this, consider the desugaring:

fn example2(smaht: RefCell<Pair>) {
    let mut smaht = smaht.borrow_mut();
    
    let tmp1: &mut Pair = smaht.deref_mut(); // borrows `smaht`
    let a = &mut tmp1.a;
    
    let tmp2: &mut Pair = smaht.deref_mut(); // borrows `smaht` again!
    let b = &mut tmp2.b;
}

It is a violation of the Rust type system to invoke deref_mut when the reference to a is valid and usable, since deref_mut requires &mut self, which in turn implies no alias to self or anything owned by self.

This desugaring suggests how the problem can be worked around in user code. The idea is to pull the result of the deref into a new temporary:

fn example3(smaht: RefCell<Pair>) {
    let mut smaht: RefMut<Pair> = smaht.borrow_mut();
    let temp: &mut Pair = &mut *smaht;
    let a = &mut temp.a;
    let b = &mut temp.b;
}

Detailed design

Removing this treatment from the borrow checker basically means changing the construction of loan paths for unique pointers.

I don’t actually know how best to implement this in the borrow checker, particularly concerning the desire to keep the ability to move out of boxes and use them in patterns. This requires some investigation. The easiest and best way may be to “do it right” and is probably to handle derefs of Box<T> in a similar way to how overloaded derefs are handled, but somewhat differently to account for the possibility of moving out of them. Some investigation is needed.

Drawbacks

The borrow checker rules are that much more restrictive.

Alternatives

We have ruled out inconsistent behavior between Box and other smart pointer types. We considered a number of ways to extend the current treatment of box to other smart pointer types:

1. Require compiler to introduce deref temporaries automatically where possible. This is plausible as a future extension but requires some thought to work through all cases. It may be surprising. Note that this would be a required optimization because if the optimization is not performed it affects what programs can successfully type check. (Naturally it is also observable.)

2. Some sort of unsafe deref trait that acknowledges possibility of other pointers into the referent. Unappealing because the problem is not that bad as to require unsafety.

3. Determining conditions (perhaps based on parametricity?) where it is provably safe to call deref. It is dubious and unknown if such conditions exist or what that even means. Rust also does not really enjoy parametricity properties due to presence of reflection and unsafe code.

Unresolved questions

Best implementation strategy.

• Start Date: 2014-06-18
• RFC PR: rust-lang/rfcs#131
• Rust Issue: rust-lang/rust#16093

Summary

Note: This RFC discusses the behavior of rustc, and not any changes to the language.

Change how target specification is done to be more flexible for unexpected usecases. Additionally, add support for the “unknown” OS in target triples, providing a minimum set of target specifications that is valid for bare-metal situations.

Motivation

One of Rust’s important use cases is embedded, OS, or otherwise “bare metal” software. At the moment, we still depend on LLVM’s split-stack prologue for stack safety. In certain situations, it is impossible or undesirable to support what LLVM requires to enable this (on x86, a certain thread-local storage setup). Additionally, porting rustc to a new platform requires modifying the compiler, adding a new OS manually.

Detailed design

A target triple consists of three strings separated by a hyphen, with a possible fourth string at the end preceded by a hyphen. The first is the architecture, the second is the “vendor”, the third is the OS type, and the optional fourth is environment type. In theory, this specifies precisely what platform the generated binary will be able to run on. All of this is determined not by us but by LLVM and other tools. When on bare metal or a similar environment, there essentially is no OS, and to handle this there is the concept of “unknown” in the target triple. When the OS is “unknown”, no runtime environment is assumed to be present (including things such as dynamic linking, threads/thread-local storage, IO, etc).

Rather than listing specific targets for special treatment, introduce a general mechanism for specifying certain characteristics of a target triple. Redesign how targets are handled around this specification, including for the built-in targets. Extend the --target flag to accept a file name of a target specification. A table of the target specification flags and their meaning:

• data-layout: The LLVM data layout to use. Mostly included for completeness; changing this is unlikely to be used.
• link-args: Arguments to pass to the linker, unconditionally.
• cpu: Default CPU to use for the target, overridable with -C target-cpu
• features: Default target features to enable, augmentable with -C target-features.
• dynamic-linking-available: Whether the dylib crate type is allowed.
• split-stacks-supported: Whether there is runtime support that will allow LLVM’s split stack prologue to function as intended.
• llvm-target: What target to pass to LLVM.
• relocation-model: What relocation model to use by default.
• target_endian, target_word_size: Specify the strings used for the corresponding cfg variables.
• code-model: Code model to pass to LLVM, overridable with -C code-model.
• no-redzone: Disable use of any stack redzone, overridable with -C no-redzone

Rather than hardcoding a specific set of behaviors per-target, with no recourse for escaping them, the compiler would also use this mechanism when deciding how to build for a given target. The process would look like:

1. Look up the target triple in an internal map, and load that configuration if it exists. If that fails, check if the target name exists as a file, and try loading that. If the file does not exist, look up <target>.json in the RUST_TARGET_PATH, which is a colon-separated list of directories.
2. If -C linker is specified, use that instead of the target-specified linker.
3. If -C link-args is given, add those to the ones specified by the target.
4. If -C target-cpu is specified, replace the target cpu with it.
5. If -C target-feature is specified, add those to the ones specified by the target.
6. If -C relocation-model is specified, replace the target relocation-model with it.
7. If -C code-model is specified, replace the target code-model with it.
8. If -C no-redzone is specified, replace the target no-redzone with true.

Then during compilation, this information is used at the proper places rather than matching against an enum listing the OSes we recognize. The target_os, target_family, and target_arch cfg variables would be extracted from the --target passed to rustc.

Drawbacks

More complexity. However, this is very flexible and allows one to use Rust on a new or non-standard target incredibly easy, without having to modify the compiler. rustc is the only compiler I know of that would allow that.

Alternatives

A less holistic approach would be to just allow disabling split stacks on a per-crate basis. Another solution could be adding a family of targets, <arch>-unknown-unknown, which omits all of the above complexity but does not allow extending to new targets easily.

• Start Date: 2014-03-17
• RFC PR #: #132
• Rust Issue #: #16293

Summary

This RFC describes a variety of extensions to allow any method to be used as first-class functions. The same extensions also allow for trait methods without receivers to be invoked in a more natural fashion.

First, at present, the notation path::method() can be used to invoke inherent methods on types. For example, Vec::new() is used to create an instance of a vector. This RFC extends that notion to also cover trait methods, so that something like T::size_of() or T::default() is legal.

Second, currently it is permitted to reference so-called “static methods” from traits using a function-like syntax. For example, one can write Default::default(). This RFC extends that notation so it can be used with any methods, whether or not they are defined with a receiver. (In fact, the distinction between static methods and other methods is completely erased, as per the method lookup of RFC PR #48.)

Third, we introduce an unambiguous if verbose notation that permits one to precisely specify a trait method and its receiver type in one form. Specifically, the notation <T as TraitRef>::item can be used to designate an item item, defined in a trait TraitRef, as implemented by the type T.

Motivation

There are several motivations:

• There is a need for an unambiguous way to invoke methods. This is typically a fallback for when the more convenient invocation forms fail:
  • For example, when multiple traits are in scope that all define the same method for the same types, there must be a way to disambiguate which method you mean.
  • It is sometimes desirable not to have autoderef:
    • For methods like clone() that apply to almost all types, it is convenient to be more specific about which precise type you want to clone. To get this right with autoderef, one must know the precise rules being used, which is contrary to the “DWIM” intention.
    • For types that implement Deref<T>, UFCS can be used to unambiguously differentiate between methods invoked on the smart pointer itself and methods invoked on its referent.
• There are many methods, such as SizeOf::size_of(), that return properties of the type alone and do not naturally take any argument that can be used to decide which trait impl you are referring to.
  • This proposal introduces a variety of ways to invoke such methods, varying in the amount of explicit information one includes:
    • T::size_of() – shorthand, but only works if T is a path
    • <T>::size_of() – infers the trait SizeOf based on the traits in scope, just as with a method call
    • <T as SizeOf>::size_of() – completely unambiguous

Detailed design

Path syntax

The syntax of paths is extended as follows:

PATH = ID_SEGMENT { '::' ID_SEGMENT }
     | TYPE_SEGMENT { '::' ID_SEGMENT }
     | ASSOC_SEGMENT '::' ID_SEGMENT { '::' ID_SEGMENT }
ID_SEGMENT   = ID [ '::' '<' { TYPE ',' TYPE } '>' ]
TYPE_SEGMENT = '<' TYPE '>'
ASSOC_SEGMENT = '<' TYPE 'as' TRAIT_REFERENCE '>'

Examples of valid paths. In these examples, capitalized names refer to types (though this doesn’t affect the grammar).

a::b::c
a::<T1,T2>::b::c
T::size_of
<T>::size_of
<T as SizeOf>::size_of
Eq::eq
Eq::<T>::eq
Zero::zero

Normalization of path that reference types

Whenever a path like ...::a::... resolves to a type (but not a trait), it is rewritten (internally) to <...::a>::....

Note that there is a subtle distinction between the following paths:

ToStr::to_str
<ToStr>::to_str

In the former, we are selecting the member to_str from the trait ToStr. The result is a function whose type is basically equivalent to:

fn to_str<Self:ToStr>(self: &Self) -> String

In the latter, we are selecting the member to_str from the type ToStr (i.e., an ToStr object). Resolving type members is different. In this case, it would yield a function roughly equivalent to:

fn to_str(self: &ToStr) -> String

This subtle distinction arises from the fact that we pun on the trait name to indicate both a type and a reference to the trait itself. In this case, depending on which interpretation we choose, the path resolution rules differ slightly.

Paths that begin with a TYPE_SEGMENT

When a path begins with a TYPE_SEGMENT, it is a type-relative path. If this is the complete path (e.g., <int>), then the path resolves to the specified type. If the path continues (e.g., <int>::size_of) then the next segment is resolved using the following procedure. The procedure is intended to mimic method lookup, and hence any changes to method lookup may also change the details of this lookup.

Given a path <T>::m::...:

1. Search for members of inherent impls defined on T (if any) with the name m. If any are found, the path resolves to that item.
2. Otherwise, let IN_SCOPE_TRAITS be the set of traits that are in scope and which contain a member named m:
   • Let IMPLEMENTED_TRAITS be those traits from IN_SCOPE_TRAITS for which an implementation exists that (may) apply to T.
     • There can be ambiguity in the case that T contains type inference variables.
   • If IMPLEMENTED_TRAITS is not a singleton set, report an ambiguity error. Otherwise, let TRAIT be the member of IMPLEMENTED_TRAITS.
   • If TRAIT is ambiguously implemented for T, report an ambiguity error and request further type information.
   • Otherwise, rewrite the path to <T as Trait>::m::... and continue.

Paths that begin with an ASSOC_SEGMENT

When a path begins with an ASSOC_SEGMENT, it is a reference to an associated item defined from a trait. Note that such paths must always have a follow-on member m (that is, <T as Trait> is not a complete path, but <T as Trait>::m is).

To resolve the path, first search for an applicable implementation of Trait for T. If no implementation can be found – or the result is ambiguous – then report an error.

Otherwise:

• Determine the types of output type parameters for Trait from the implementation.
• If output type parameters were specified in the path, ensure that they are compatible with those specified on the impl.
  • For example, if the path were <int as SomeTrait<uint>>, and the impl is declared as impl SomeTrait<char> for int, then an error would be reported because char and uint are not compatible.
• Resolve the path to the member of the trait with the substitution composed of the output type parameters from the impl and Self => T.

Alternatives

We have explored a number of syntactic alternatives. This has been selected as being the only one that is simultaneously:

• Tolerable to look at.
• Able to convey all necessary information along with auxiliary information the user may want to verify:
  • Self type, type of trait, name of member, type output parameters

Here are some leading candidates that were considered along with their equivalents in the syntax proposed by this RFC. The reasons for their rejection are listed:

module::type::(Trait::member)    <module::type as Trait>::member
--> semantics of parentheses considered too subtle
--> cannot accommodate types that are not paths, like `[int]`

(type: Trait)::member            <type as Trait>::member
--> complicated to parse
--> cannot accommodate types that are not paths, like `[int]`

... (I can't remember all the rest)

One variation that is definitely possible is that we could use the : rather than the keyword as:

<type: Trait>::member            <type as Trait>::member
--> no real objection. `as` was chosen because it mimics the
    syntax for constructing a trait object.

Unresolved questions

Is there a better way to disambiguate a reference to a trait item ToStr::to_str versus a reference to a member of the object type <ToStr>::to_str? I personally do not think so: so long as we pun on the name of the trait, the potential for confusion will remain. Therefore, the only two possibilities I could come up with are to try and change the question:

• One answer might be that we simply make the second form meaningless by prohibiting inherent impls on object types. But there remains a utility to being able to write something like <ToStr>::is_sized() (where is_sized() is an example of a trait fn that could apply to both sized and unsized types). Moreover, artificially restricting object types just for this reason doesn’t seem right.

• Another answer is to change the syntax of object types. I have sometimes considered that impl ToStr might be better suited as the object type and then ToStr could be used as syntactic sugar for a type parameter. But there exists a lot of precedent for the current approach and hence I think this is likely a bad idea (not to mention that it’s a drastic change).

• Start Date: 2014-09-30
• RFC PR #: https://github.com/rust-lang/rfcs/pull/135
• Rust Issue #: https://github.com/rust-lang/rust/issues/17657

Summary

Add where clauses, which provide a more expressive means of specifying trait parameter bounds. A where clause comes after a declaration of a generic item (e.g., an impl or struct definition) and specifies a list of bounds that must be proven once precise values are known for the type parameters in question. The existing bounds notation would remain as syntactic sugar for where clauses.

So, for example, the impl for HashMap could be changed from this:

impl<K:Hash+Eq,V> HashMap<K, V>
{
    ..
}

to the following:

impl<K,V> HashMap<K, V>
    where K : Hash + Eq
{
    ..
}

The full grammar can be found in the detailed design.

Motivation

The high-level bit is that the current bounds syntax does not scale to complex cases. Introducing where clauses is a simple extension that gives us a lot more expressive power. In particular, it will allow us to refactor the operator traits to be in a convenient, multidispatch form (e.g., so that user-defined mathematical types can be added to int and vice versa). (It’s also worth pointing out that, once #5527 lands at least, implementing where clauses will be very little work.)

Here is a list of limitations with the current bounds syntax that are overcome with the where syntax:

• It cannot express bounds on anything other than type parameters. Therefore, if you have a function generic in T, you can write T:MyTrait to declare that T must implement MyTrait, but you can’t write Option<T> : MyTrait or (int, T) : MyTrait. These forms are less commonly required but still important.

• It does not work well with associated types. This is because there is no space to specify the value of an associated type. Other languages use where clauses (or something analogous) for this purpose.

• It’s just plain hard to read. Experience has shown that as the number of bounds grows, the current syntax becomes hard to read and format.

Let’s examine each case in detail.

Bounds are insufficiently expressive

Currently bounds can only be declared on type parameters. But there are situations where one wants to declare bounds not on the type parameter itself but rather a type that includes the type parameter.

Partially generic types

One situation where this occurs is when you want to write functions where types are partially known and have those interact with other functions that are fully generic. To explain the situation, let’s examine some code adapted from rustc.

Imagine I have a table parameterized by a value type V and a key type K. There are also two traits, Value and Key, that describe the keys and values. Also, each type of key is linked to a specific value:

struct Table<V:Value, K:Key<V>> { ... }
trait Key<V:Value> { ... }
trait Value { ... }

Now, imagine I want to write some code that operates over all keys whose value is an Option<T> for some T:

fn example<T,K:Key<Option<T>>>(table: &Table<Option<T>, K>) { ... }

This seems reasonable, but this code will not compile. The problem is that the compiler needs to know that the value type implements Value, but here the value type is Option<T>. So we’d need to declare Option<T> : Value, which we cannot do.

There are workarounds. I might write a new trait OptionalValue:

trait OptionalValue<T> {
    fn as_option<'a>(&'a self) -> &'a Option<T>; // identity fn
}

and then I could write my example as:

fn example<T,O:OptionalValue<T>,K:Key<O>>(table: &Table<O, K>) { ... }

But this is making my example function, already a bit complicated, become quite obscure.

Multidispatch traits

Another situation where a similar problem is encountered is multidispatch traits (aka, multiparameter type classes in Haskell). The idea of a multidispatch trait is to be able to choose the impl based not just on one type, as is the most common case, but on multiple types (usually, but not always, two).

Multidispatch is rarely needed because the vast majority of traits are characterized by a single type. But when you need it, you really need it. One example that arises in the standard library is the traits for binary operators like +. Today, the Add trait is defined using only single-dispatch (like so):

pub trait Add<Rhs,Sum> {
    fn add(&self, rhs: &Rhs) -> Sum;
}

The expression a + b is thus sugar for Add::add(&a, &b). Because of how our trait system works, this means that only the type of the left-hand side (the Self parameter) will be used to select the impl. The type for the right-hand side (Rhs) along with the type of their sum (Sum) are defined as trait parameters, which are always outputs of the trait matching: that is, they are specified by the impl and are not used to select which impl is used.

This setup means that addition is not as extensible as we would like. For example, the standard library includes implementations of this trait for integers and other built-in types:

impl Add<int,int> for int { ... }
impl Add<f32,f32> for f32 { ... }

The limitations of this setup become apparent when we consider how a hypothetical user library might integrate. Imagine a library L that defines a type Complex representing complex numbers:

struct Complex { ... }

Naturally, it should be possible to add complex numbers and integers. Since complex number addition is commutative, it should be possible to write both 1 + c and c + 1. Thus one might try the following impls:

impl Add<int,Complex> for Complex { ... }     // 1. Complex + int
impl Add<Complex,Complex> for int { ... }     // 2. int + Complex
impl Add<Complex,Complex> for Complex { ... } // 3. Complex + Complex

Due to the coherence rules, however, this setup will not work. There are in fact three errors. The first is that there are two impls of Add defined for Complex (1 and 3). The second is that there are two impls of Add defined for int (the one from the standard library and 2). The final error is that impl 2 violates the orphan rule, since the type int is not defined in the current crate.

This is not a new problem. Object-oriented languages, with their focus on single dispatch, have long had trouble dealing with binary operators. One common solution is double dispatch, an awkward but effective pattern in which no type ever implements Add directly. Instead, we introduce “indirection” traits so that, e.g., int is addable to anything that implements AddToInt and so on. This is not my preferred solution so I will not describe it in detail, but rather refer readers to this blog post where I describe how it works.

An alternative to double dispatch is to define Add on tuple types (LHS, RHS) rather than on a single value. Imagine that the Add trait were defined as follows:

trait Add<Sum> {
    fn add(self) -> Sum;
}

impl Add<int> for (int, int) {
    fn add(self) -> int {
        let (x, y) = self;
        x + y
    }
}

Now the expression a + b would be sugar for Add::add((a, b)). This small change has several interesting ramifications. For one thing, the library L can easily extend Add to cover complex numbers:

impl Add<Complex> for (Complex, int)     { ... }
impl Add<Complex> for (int, Complex)     { ... }
impl Add<Complex> for (Complex, Complex) { ... }

These impls do not violate the coherence rules because they are all applied to distinct types. Moreover, none of them violate the orphan rule because each of them is a tuple involving at least one type local to the library.

One downside of this Add pattern is that there is no way within the trait definition to refer to the type of the left- or right-hand side individually; we can only use the type Self to refer to the tuple of both types. In the Discussion section below, I will introduce an extended “multi-dispatch” pattern that addresses this particular problem.

There is however another problem that where clauses help to address. Imagine that we wish to define a function to increment complex numbers:

fn increment(c: Complex) -> Complex {
    1 + c
}

This function is pretty generic, so perhaps we would like to generalize it to work over anything that can be added to an int. We’ll use our new version of the Add trait that is implemented over tuples:

fn increment<T:...>(c: T) -> T {
    1 + c
}

At this point we encounter the problem. What bound should we give for T? We’d like to write something like (int, T) : Add<T> – that is, Add is implemented for the tuple (int, T) with the sum type T. But we can’t write that, because the current bounds syntax is too limited.

Where clauses give us an answer. We can write a generic version of increment like so:

fn increment<T>(c: T) -> T
    where (int, T) : Add<T>
{
    1 + c
}

Associated types

It is unclear exactly what form associated types will have in Rust, but it is well documented that our current design, in which type parameters decorate traits, does not scale particularly well. (For curious readers, there are several blog posts exploring the design space of associated types with respect to Rust in particular.)

The high-level summary of associated types is that we can replace a generic trait like Iterator:

trait Iterator<E> {
    fn next(&mut self) -> Option<E>;
}

With a version where the type parameter is a “member” of the Iterator trait:

trait Iterator {
    type E;
    
    fn next(&mut self) -> Option<E>;
}

This syntactic change helps to highlight that, for any given type, the type E is fixed by the impl, and hence it can be considered a member (or output) of the trait. It also scales better as the number of associated types grows.

One challenge with this design is that it is not clear how to convert a function like the following:

fn sum<I:Iterator<int>>(i: I) -> int {
    ...    
}

With associated types, the reference Iterator<int> is no longer valid, since the trait Iterator doesn’t have type parameters.

The usual solution to this problem is to employ a where clause:

fn sum<I:Iterator>(i: I) -> int
  where I::E == int
{
    ...    
}

We can also employ where clauses with object types via a syntax like &Iterator<where E=int> (admittedly somewhat wordy)

Readability

When writing very generic code, it is common to have a large number of parameters with a large number of bounds. Here is some example function extracted from rustc:

fn set_var_to_merged_bounds<T:Clone + InferStr + LatticeValue,
                            V:Clone+Eq+ToStr+Vid+UnifyVid<Bounds<T>>>(
                            &self,
                            v_id: V,
                            a: &Bounds<T>,
                            b: &Bounds<T>,
                            rank: uint)
                            -> ures;

Definitions like this are very difficult to read (it’s hard to even know how to format such a definition).

Using a where clause allows the bounds to be separated from the list of type parameters:

fn set_var_to_merged_bounds<T,V>(&self,
                                 v_id: V,
                                 a: &Bounds<T>,
                                 b: &Bounds<T>,
                                 rank: uint)
                                 -> ures
    where T:Clone,         // it is legal to use individual clauses...
          T:InferStr,
          T:LatticeValue,
          V:Clone+Eq+ToStr+Vid+UnifyVid<Bounds<T>>, // ...or use `+`
{                                     
    ..
}

This helps to separate out the function signature from the extra requirements that the function places on its types.

If I may step aside from the “impersonal voice” of the RFC for a moment, I personally find that when writing generic code it is helpful to focus on the types and signatures, and come to the bounds later. Where clauses help to separate these distinctions. Naturally, your mileage may vary. - nmatsakis

Detailed design

Where can where clauses appear?

Where clauses can be added to anything that can be parameterized with type/lifetime parameters with the exception of trait method definitions: impl declarations, fn declarations, and trait and struct definitions. They appear as follows:

impl Foo<A,B>
    where ...
{ }

impl Foo<A,B> for C
    where ...
{ }

impl Foo<A,B> for C
{
    fn foo<A,B> -> C
        where ...
    { }
}

fn foo<A,B> -> C
    where ...
{ }

struct Foo<A,B>
    where ...
{ }

trait Foo<A,B> : C
    where ...
{ }

Where clauses cannot (yet) appear on trait methods

Note that trait method definitions were specifically excluded from the list above. The reason is that including where clauses on a trait method raises interesting questions for what it means to implement the trait. Using where clauses it becomes possible to define methods that do not necessarily apply to all implementations. We intend to enable this feature but it merits a second RFC to delve into the details.

Where clause grammar

The grammar for a where clause would be as follows (BNF):

WHERE = 'where' BOUND { ',' BOUND } [,]
BOUND = TYPE ':' TRAIT { '+' TRAIT } [+]
TRAIT = Id [ '<' [ TYPE { ',' TYPE } [,] ] '>' ]
TYPE  = ... (same type grammar as today)

Semantics

The meaning of a where clause is fairly straightforward. Each bound in the where clause must be proven by the caller after substitution of the parameter types.

One interesting case concerns trivial where clauses where the self-type does not refer to any of the type parameters, such as the following:

fn foo()
    where int : Eq
{ ... }

Where clauses like these are considered an error. They have no particular meaning, since the callee knows all types involved. This is a conservative choice: if we find that we do desire a particular interpretation for them, we can always make them legal later.

Drawbacks

This RFC introduces two ways to declare a bound.

Alternatives

Remove the existing trait bounds. I decided against this both to avoid breaking lots of existing code and because the existing syntax is convenient much of the time.

Embed where clauses in the type parameter list. One alternative syntax that was proposed is to embed a where-like clause in the type parameter list. Thus the increment() example

fn increment<T>(c: T) -> T
    where () : Add<int,T,T>
{
    1 + c
}

would become something like:

fn increment<T, ():Add<int,T,T>>(c: T) -> T
{
    1 + c
}

This is unfortunately somewhat ambiguous, since a bound like T:Eq could either be declared a type parameter T or as a condition that the (existing) type T implement Eq.

Use a colon instead of the keyword. There is some precedent for this from the type state days. Unfortunately, it doesn’t work with traits due to the supertrait list, and it also doesn’t look good with the use of : as a trait-bound separator:

fn increment<T>(c: T) -> T
    : () : Add<int,T,T>
{
    1 + c
}

• Start Date: 2014-06-24
• RFC PR #: #136
• Rust Issue #: #16463

Summary

Require a feature gate to expose private items in public APIs, until we grow the appropriate language features to be able to remove the feature gate and forbid it entirely.

Motivation

Privacy is central to guaranteeing the invariants necessary to write correct code that employs unsafe blocks. Although the current language rules prevent a private item from being directly named from outside the current module, they still permit direct access to private items in some cases. For example, a public function might return a value of private type. A caller from outside the module could then invoke this function and, thanks to type inference, gain access to the private type (though they still could not invoke public methods or access public fields). This access could undermine the reasoning of the author of the module. Fortunately, it is not hard to prevent.

Detailed design

Overview

The general idea is that:

• If an item is declared as public, items referred to in the public-facing parts of that item (e.g. its type) must themselves be declared as public.

Details follow.

The rules

These rules apply as long as the feature gate is not enabled. After the feature gate has been removed, they will apply always.

When is an item “public”?

Items that are explicitly declared as pub are always public. In addition, items in the impl of a trait (not an inherent impl) are considered public if all of the following conditions are met:

• The trait being implemented is public.
• All input types (currently, the self type) of the impl are public.
• Motivation: If any of the input types or the trait is public, it should be impossible for an outside to access the items defined in the impl. They cannot name the types nor they can get direct access to a value of those types.

What restrictions apply to public items?

The rules for various kinds of public items are as follows:

• If it is a static declaration, items referred to in its type must be public.

• If it is an fn declaration, items referred to in its trait bounds, argument types, and return type must be public.

• If it is a struct or enum declaration, items referred to in its trait bounds and in the types of its pub fields must be public.

• If it is a type declaration, items referred to in its definition must be public.

• If it is a trait declaration, items referred to in its super-traits, in the trait bounds of its type parameters, and in the signatures of its methods (see fn case above) must be public.

Examples

Here are some examples to demonstrate the rules.

Struct fields

// A private struct may refer to any type in any field.
struct Priv {
    a: Priv,
    b: Pub,
    pub c: Priv
}

enum Vapor<A> { X, Y, Z } // Note that A is not used

// Public fields of a public struct may only refer to public types.
pub struct Item {
    // Private field may reference a private type.
    a: Priv,
    
    // Public field must refer to a public type.
    pub b: Pub,

    // ERROR: Public field refers to a private type.
    pub c: Priv,
    
    // ERROR: Public field refers to a private type.
    // For the purposes of this test, we do not descend into the type,
    // but merely consider the names that appear in type parameters
    // on the type, regardless of usage (or lack thereof) within the type
    // definition itself.
    pub d: Vapor<Priv>,
}

pub struct Pub { ... }

Methods

struct Priv { .. }
pub struct Pub { .. }
pub struct Foo { .. }

impl Foo {
    // Illegal: public method with argument of private type.
    pub fn foo(&self, p: Priv) { .. }
}

Trait bounds

trait PrivTrait { ... }

// Error: type parameter on public item bounded by a private trait.
pub struct Foo<X: PrivTrait> { ... }

// OK: type parameter on private item.
struct Foo<X: PrivTrait> { ... }

Trait definitions

struct PrivStruct { ... }

pub trait PubTrait {
    // Error: private struct referenced from method in public trait
    fn method(x: PrivStruct) { ... }
}

trait PrivTrait {
    // OK: private struct referenced from method in private trait 
    fn method(x: PrivStruct) { ... }
}

Implementations

To some extent, implementations are prevented from exposing private types because their types must match the trait. However, that is not true with generics.

pub trait PubTrait<T> {
    fn method(t: T);
}

struct PubStruct { ... }

struct PrivStruct { ... }

impl PubTrait<PrivStruct> for PubStruct {
           // ^~~~~~~~~~ Error: Private type referenced from impl of
           //            public trait on a public type. [Note: this is
           //            an "associated type" here, not an input.]

    fn method(t: PrivStruct) {
              // ^~~~~~~~~~ Error: Private type in method signature.
              //
              // Implementation note. It may not be a good idea to report
              // an error here; I think private types can only appear in
              // an impl by having an associated type bound to a private
              // type.
    }
}

Type aliases

Note that the path to the public item does not have to be private.

mod impl {
    pub struct Foo { ... }
}
pub type Bar = self::impl::Foo;

Negative examples

The following examples should fail to compile under these rules.

Non-public items referenced by a pub use

These examples are illegal because they use a pub use to re-export a private item:

struct Item { ... }
pub mod module {
    // Error: Item is not declared as public, but is referenced from
    // a `pub use`.
    pub use Item;
}

struct Foo { ... }
// Error: Non-public item referenced by `pub use`.
pub use Item = Foo;

If it was desired to have a private name that is publicly “renamed” using a pub use, that can be achieved using a module:

mod impl {
    pub struct ItemPriv;
}
pub use Item = self::impl::ItemPriv;

Drawbacks

Adds a (temporary) feature gate.

Requires some existing code to opt-in to the feature gate before transitioning to a more explicit alternative.

Requires effort to implement.

Alternatives

If we stick with the status quo, we’ll have to resolve several bizarre questions and keep supporting its behavior indefinitely after 1.0.

Instead of a feature gate, we could just ban these things outright right away, at the cost of temporarily losing some convenience and a small amount of expressiveness before the more principled replacement features are implemented.

We could make an exception for private supertraits, as these are not quite as problematic as the other cases. However, especially given that a more principled alternative is known (private methods), I would rather not make any exceptions.

The original design of this RFC had a stronger notion of “public” which also considered whether a public path existed to the item. In other words, a module X could not refer to a public item Y from a submodule Z, unless X also exposed a public path to Y (whether that be because Z was public, or via a pub use). This definition strengthened the basic guarantee of “private things are only directly accessible from within the current module” to include the idea that public functions in outer modules cannot accidentally refer to public items from inner modules unless there is a public path from the outer to the inner module. Unfortunately, these rules were complex to state concisely and also hard to understand in practice; when an error occurred under these rules, it was very hard to evaluate whether the error was legitimate. The newer rules are simpler while still retaining the basic privacy guarantee.

One important advantage of the earlier approach, and a scenario not directly addressed in this RFC, is that there may be items which are declared as public by an inner module but still not intended to be exposed to the world at large (in other words, the items are only expected to be used within some subtree). A special case of this is crate-local data. In the older rules, the “intended scope” of privacy could be somewhat inferred from the existence (or non-existence) of pub use declarations. However, in the author’s opinion, this scenario would be best addressed by making pub declarations more expressive so that the intended scope can be stated directly.

• Start Date: 2014-06-25
• RFC PR: rust-lang/rfcs#139
• Rust Issue: rust-lang/rust#10504

Summary

Remove the coercion from Box<T> to &T from the language.

Motivation

The coercion between Box<T> to &T is not replicable by user-defined smart pointers and has been found to be rarely used 1. We already removed the coercion between Box<T> and &mut T in RFC 33.

Detailed design

The coercion between Box<T> and &T should be removed.

Note that methods that take &self can still be called on values of type Box<T> without any special referencing or dereferencing. That is because the semantics of auto-deref and auto-ref conspire to make it work: the types unify after one autoderef followed by one autoref.

Drawbacks

Borrowing from Box<T> to &T may be convenient.

Alternatives

The impact of not doing this is that the coercion will remain.

Unresolved questions

None.

• Start Date: 2014-06-24
• RFC PR: rust-lang/rfcs#141
• Rust Issue: rust-lang/rust#15552

Summary

This RFC proposes to

1. Expand the rules for eliding lifetimes in fn definitions, and
2. Follow the same rules in impl headers.

By doing so, we can avoid writing lifetime annotations ~87% of the time that they are currently required, based on a survey of the standard library.

Motivation

In today’s Rust, lifetime annotations make code more verbose, both for methods

fn get_mut<'a>(&'a mut self) -> &'a mut T

and for impl blocks:

impl<'a> Reader for BufReader<'a> { ... }

In the vast majority of cases, however, the lifetimes follow a very simple pattern.

By codifying this pattern into simple rules for filling in elided lifetimes, we can avoid writing any lifetimes in ~87% of the cases where they are currently required.

Doing so is a clear ergonomic win.

Detailed design

Today’s lifetime elision rules

Rust currently supports eliding lifetimes in functions, so that you can write

fn print(s: &str);
fn get_str() -> &str;

instead of

fn print<'a>(s: &'a str);
fn get_str<'a>() -> &'a str;

The elision rules work well for functions that consume references, but not for functions that produce them. The get_str signature above, for example, promises to produce a string slice that lives arbitrarily long, and is either incorrect or should be replaced by

fn get_str() -> &'static str;

Returning 'static is relatively rare, and it has been proposed to make leaving off the lifetime in output position an error for this reason.

Moreover, lifetimes cannot be elided in impl headers.

The proposed rules

Overview

This RFC proposes two changes to the lifetime elision rules:

1. Since eliding a lifetime in output position is usually wrong or undesirable under today’s elision rules, interpret it in a different and more useful way.

2. Interpret elided lifetimes for impl headers analogously to fn definitions.

Lifetime positions

A lifetime position is anywhere you can write a lifetime in a type:

&'a T
&'a mut T
T<'a>

As with today’s Rust, the proposed elision rules do not distinguish between different lifetime positions. For example, both &str and Ref<uint> have elided a single lifetime.

Lifetime positions can appear as either “input” or “output”:

• For fn definitions, input refers to the types of the formal arguments in the fn definition, while output refers to result types. So fn foo(s: &str) -> (&str, &str) has elided one lifetime in input position and two lifetimes in output position. Note that the input positions of a fn method definition do not include the lifetimes that occur in the method’s impl header (nor lifetimes that occur in the trait header, for a default method).

• For impl headers, input refers to the lifetimes appears in the type receiving the impl, while output refers to the trait, if any. So impl<'a> Foo<'a> has 'a in input position, while impl<'a, 'b, 'c> SomeTrait<'b, 'c> for Foo<'a, 'c> has 'a in input position, 'b in output position, and 'c in both input and output positions.

The rules

• Each elided lifetime in input position becomes a distinct lifetime parameter. This is the current behavior for fn definitions.

• If there is exactly one input lifetime position (elided or not), that lifetime is assigned to all elided output lifetimes.

• If there are multiple input lifetime positions, but one of them is &self or &mut self, the lifetime of self is assigned to all elided output lifetimes.

• Otherwise, it is an error to elide an output lifetime.

Notice that the actual signature of a fn or impl is based on the expansion rules above; the elided form is just a shorthand.

Examples

fn print(s: &str);                                      // elided
fn print<'a>(s: &'a str);                               // expanded

fn debug(lvl: uint, s: &str);                           // elided
fn debug<'a>(lvl: uint, s: &'a str);                    // expanded

fn substr(s: &str, until: uint) -> &str;                // elided
fn substr<'a>(s: &'a str, until: uint) -> &'a str;      // expanded

fn get_str() -> &str;                                   // ILLEGAL

fn frob(s: &str, t: &str) -> &str;                      // ILLEGAL

fn get_mut(&mut self) -> &mut T;                        // elided
fn get_mut<'a>(&'a mut self) -> &'a mut T;              // expanded

fn args<T:ToCStr>(&mut self, args: &[T]) -> &mut Command                  // elided
fn args<'a, 'b, T:ToCStr>(&'a mut self, args: &'b [T]) -> &'a mut Command // expanded

fn new(buf: &mut [u8]) -> BufWriter;                    // elided
fn new<'a>(buf: &'a mut [u8]) -> BufWriter<'a>          // expanded

impl Reader for BufReader { ... }                       // elided
impl<'a> Reader for BufReader<'a> { .. }                // expanded

impl Reader for (&str, &str) { ... }                    // elided
impl<'a, 'b> Reader for (&'a str, &'b str) { ... }      // expanded

impl StrSlice for &str { ... }                          // elided
impl<'a> StrSlice<'a> for &'a str { ... }               // expanded

trait Bar<'a> { fn bound(&'a self) -> &int { ... }    fn fresh(&self) -> &int { ... } }           // elided
trait Bar<'a> { fn bound(&'a self) -> &'a int { ... } fn fresh<'b>(&'b self) -> &'b int { ... } } // expanded

impl<'a> Bar<'a> for &'a str {
  fn bound(&'a self) -> &'a int { ... } fn fresh(&self) -> &int { ... }              // elided
}
impl<'a> Bar<'a> for &'a str {
  fn bound(&'a self) -> &'a int { ... } fn fresh<'b>(&'b self) -> &'b int { ... }    // expanded
}

// Note that when the impl reuses the same signature (with the same elisions)
// from the trait definition, the expanded forms will also match, and thus
// the `impl` will be compatible with the `trait`.

impl Bar for &str            { fn bound(&self) -> &int { ... } }           // elided
impl<'a> Bar<'a> for &'a str { fn bound<'b>(&'b self) -> &'b int { ... } } // expanded

// Note that the preceding example's expanded methods do not match the
// signatures from the above trait definition for `Bar`; in the general
// case, if the elided signatures between the `impl` and the `trait` do
// not match, an expanded `impl` may not be compatible with the given
// `trait` (and thus would not compile).

impl Bar for &str            { fn fresh(&self) -> &int { ... } }           // elided
impl<'a> Bar<'a> for &'a str { fn fresh<'b>(&'b self) -> &'b int { ... } } // expanded

impl Bar for &str {
  fn bound(&'a self) -> &'a int { ... } fn fresh(&self) -> &int { ... }    // ILLEGAL: unbound 'a
}

Error messages

Since the shorthand described above should eliminate most uses of explicit lifetimes, there is a potential “cliff”. When a programmer first encounters a situation that requires explicit annotations, it is important that the compiler gently guide them toward the concept of lifetimes.

An error can arise with the above shorthand only when the program elides an output lifetime and neither of the rules can determine how to annotate it.

For fn

The error message should guide the programmer toward the concept of lifetime by talking about borrowed values:

This function’s return type contains a borrowed value, but the signature does not say which parameter it is borrowed from. It could be one of a, b, or c. Mark the input parameter it borrows from using lifetimes, e.g. [generated example]. See [url] for an introduction to lifetimes.

This message is slightly inaccurate, since the presence of a lifetime parameter does not necessarily imply the presence of a borrowed value, but there are no known use-cases of phantom lifetime parameters.

For impl

The error case on impl is exceedingly rare: it requires (1) that the impl is for a trait with a lifetime argument, which is uncommon, and (2) that the Self type has multiple lifetime arguments.

Since there are no clear “borrowed values” for an impl, this error message speaks directly in terms of lifetimes. This choice seems warranted given that a programmer implementing a trait with lifetime parameters will almost certainly already understand lifetimes.

TraitName requires lifetime arguments, and the impl does not say which lifetime parameters of TypeName to use. Mark the parameters explicitly, e.g. [generated example]. See [url] for an introduction to lifetimes.

The impact

To assess the value of the proposed rules, we conducted a survey of the code defined in libstd (as opposed to the code it reexports). This corpus is large and central enough to be representative, but small enough to easily analyze.

We found that of the 169 lifetimes that currently require annotation for libstd, 147 would be elidable under the new rules, or 87%.

Note: this percentage does not include the large number of lifetimes that are already elided with today’s rules.

The detailed data is available at: https://gist.github.com/aturon/da49a6d00099fdb0e861

Drawbacks

Learning lifetimes

The main drawback of this change is pedagogical. If lifetime annotations are rarely used, newcomers may encounter error messages about lifetimes long before encountering lifetimes in signatures, which may be confusing. Counterpoints:

• This is already the case, to some extent, with the current elision rules.

• Most existing error messages are geared to talk about specific borrows not living long enough, pinpointing their locations in the source, rather than talking in terms of lifetime annotations. When the errors do mention annotations, it is usually to suggest specific ones.

• The proposed error messages above will help programmers transition out of the fully elided regime when they first encounter a signature requiring it.

• When combined with a good tutorial on the borrow/lifetime system (which should be introduced early in the documentation), the above should provide a reasonably gentle path toward using and understanding explicit lifetimes.

Programmers learn lifetimes once, but will use them many times. Better to favor long-term ergonomics, if a simple elision rule can cover 87% of current lifetime uses (let alone the currently elided cases).

Subtlety for non-& types

While the rules are quite simple and regular, they can be subtle when applied to types with lifetime positions. To determine whether the signature

fn foo(r: Bar) -> Bar

is actually using lifetimes via the elision rules, you have to know whether Bar has a lifetime parameter. But this subtlety already exists with the current elision rules. The benefit is that library types like Ref<'a, T> get the same status and ergonomics as built-ins like &'a T.

Alternatives

• Do not include output lifetime elision for impl. Since traits with lifetime parameters are quite rare, this would not be a great loss, and would simplify the rules somewhat.

• Only add elision rules for fn, in keeping with current practice.

• Only add elision for explicit & pointers, eliminating one of the drawbacks mentioned above. Doing so would impose an ergonomic penalty on abstractions, though: Ref would be more painful to use than &.

Unresolved questions

The fn and impl cases tackled above offer the biggest bang for the buck for lifetime elision. But we may eventually want to consider other opportunities.

Double lifetimes

Another pattern that sometimes arises is types like &'a Foo<'a>. We could consider an additional elision rule that expands &Foo to &'a Foo<'a>.

However, such a rule could be easily added later, and it is unclear how common the pattern is, so it seems best to leave that for a later RFC.

Lifetime elision in structs

We may want to allow lifetime elision in structs, but the cost/benefit analysis is much less clear. In particular, it could require chasing an arbitrary number of (potentially private) struct fields to discover the source of a lifetime parameter for a struct. There are also some good reasons to treat elided lifetimes in structs as 'static.

Again, since shorthand can be added backwards-compatibly, it seems best to wait.

• Start Date: 2014-07-02
• RFC PR: rust-lang/rfcs#151
• Rust Issue: rust-lang/rust#12831

Summary

Closures should capture their upvars by value unless the ref keyword is used.

Motivation

For unboxed closures, we will need to syntactically distinguish between captures by value and captures by reference.

Detailed design

This is a small part of #114, split off to separate it from the rest of the discussion going on in that RFC.

Closures should capture their upvars (closed-over variables) by value unless the ref keyword precedes the opening | of the argument list. Thus |x| x + 2 will capture x by value (and thus, if x is not Copy, it will move x into the closure), but ref |x| x + 2 will capture x by reference.

In an unboxed-closures world, the immutability/mutability of the borrow (as the case may be) is inferred from the type of the closure: Fn captures by immutable reference, while FnMut captures by mutable reference. In a boxed-closures world, the borrows are always mutable.

Drawbacks

It may be that ref is unwanted complexity; it only changes the semantics of 10%-20% of closures, after all. This does not add any core functionality to the language, as a reference can always be made explicitly and then captured. However, there are a lot of closures, and the workaround to capture a reference by value is painful.

Alternatives

As above, the impact of not doing this is that reference semantics would have to be achieved. However, the diff against current Rust was thousands of lines of pretty ugly code.

Another alternative would be to annotate each individual upvar with its capture semantics, like capture clauses in C++11. This proposal does not preclude adding that functionality should it be deemed useful in the future. Note that C++11 provides a syntax for capturing all upvars by reference, exactly as this proposal does.

Unresolved questions

None.

• Start Date: 2014-07-04
• RFC PR #: rust-lang/rfcs#155
• Rust Issue #: rust-lang/rust#17059

Summary

Require “anonymous traits”, i.e. impl MyStruct to occur only in the same module that MyStruct is defined.

Motivation

Before I can explain the motivation for this, I should provide some background as to how anonymous traits are implemented, and the sorts of bugs we see with the current behaviour. The conclusion will be that we effectively already only support impl MyStruct in the same module that MyStruct is defined, and making this a rule will simply give cleaner error messages.

• The compiler first sees impl MyStruct during the resolve phase, specifically in Resolver::build_reduced_graph(), called by Resolver::resolve() in src/librustc/middle/resolve.rs. This is before any type checking (or type resolution, for that matter) is done, so the compiler trusts for now that MyStruct is a valid type.
• If MyStruct is a path with more than one segment, such as mymod::MyStruct, it is silently ignored (how was this not flagged when the code was written??), which effectively causes static methods in such impls to be dropped on the floor. A silver lining here is that nothing is added to the current module namespace, so the shadowing bugs demonstrated in the next bullet point do not apply here. (To locate this bug in the code, find the match immediately following the FIXME (#3785) comment in resolve.rs.) This leads to the following

mod break1 {
    pub struct MyGuy;

    impl MyGuy {
        pub fn do1() { println!("do 1"); }
    }
}

impl break1::MyGuy {
    fn do2() { println!("do 2"); }
}

fn main() {
    break1::MyGuy::do1();
    break1::MyGuy::do2();
}

<anon>:15:5: 15:23 error: unresolved name `break1::MyGuy::do2`.
<anon>:15     break1::MyGuy::do2();

as noticed by @huonw in https://github.com/rust-lang/rust/issues/15060 .

• If one does not exist, the compiler creates a submodule MyStruct of the current module, with kind ImplModuleKind. Static methods are placed into this module. If such a module already exists, the methods are appended to it, to support multiple impl MyStruct blocks within the same module. If a module exists that is not ImplModuleKind, the compiler signals a duplicate module definition error.
• Notice at this point that if there is a use MyStruct, the compiler will act as though it is unaware of this. This is because imports are not resolved yet (they are in Resolver::resolve_imports() called immediately after Resolver::build_reduced_graph() is called). In the final resolution step, MyStruct will be searched in the namespace of the current module, checking imports only as a fallback (and only in some contexts), so the use MyStruct is effectively shadowed. If there is an impl MyStruct in the file being imported from, the user expects that the new impl MyStruct will append to that one, same as if they are in the original file. This leads to the original bug report https://github.com/rust-lang/rust/issues/15060 .
• In fact, even if no methods from the import are used, the name MyStruct will not be associated to a type, so that

trait T {}
impl<U: T> Vec<U> {
    fn from_slice<'a>(x: &'a [uint]) -> Vec<uint> {
        fail!()
    }
}
fn main() { let r = Vec::from_slice(&[1u]); }

error: found module name used as a type: impl Vec<U>::Vec<U> (id=5)
impl<U: T> Vec<U>

which @Ryman noticed in https://github.com/rust-lang/rust/issues/15060 . The reason for this is that in Resolver::resolve_crate(), the final step of Resolver::resolve(), the type of an anonymous impl is determined by NameBindings::def_for_namespace(TypeNS). This function searches the namespace TypeNS (which is not affected by imports) for a type; failing that it tries for a module; failing that it returns None. The result is that when typeck runs, it sees impl [module name] instead of impl [type name].

The main motivation of this RFC is to clear out these bugs, which do not make sense to a user of the language (and had me confused for quite a while).

A secondary motivation is to enforce consistency in code layout; anonymous traits are used the way that class methods are used in other languages, and the data and methods of a struct should be defined nearby.

Detailed design

I propose three changes to the language:

• impl on multiple-ident paths such as impl mymod::MyStruct is disallowed. Since this currently surprises the user by having absolutely no effect for static methods, support for this is already broken.
• impl MyStruct must occur in the same module that MyStruct is defined. This is to prevent the above problems with impl-across-modules. Migration path is for users to just move code between source files.

Drawbacks

Static methods on impls-away-from-definition never worked, while non-static methods can be implemented using non-anonymous traits. So there is no loss in expressivity. However, using a trait where before there was none may be clumsy, since it might not have a sensible name, and it must be explicitly imported by all users of the trait methods.

For example, in the stdlib src/libstd/io/fs.rs we see the code impl path::Path to attach (non-static) filesystem-related methods to the Path type. This would have to be done via a FsPath trait which is implemented on Path and exported alongside Path in the prelude.

It is worth noting that this is the only instance of this RFC conflicting with current usage in the stdlib or compiler.

Alternatives

• Leaving this alone and fixing the bugs directly. This is really hard. To do it properly, we would need to seriously refactor resolve.

Unresolved questions

None.

• Start Date: 2014-08-26
• RFC PR #: rust-lang/rfcs#160
• Rust Issue #: rust-lang/rust#16779

Summary

Introduce a new if let PAT = EXPR { BODY } construct. This allows for refutable pattern matching without the syntactic and semantic overhead of a full match, and without the corresponding extra rightward drift. Informally this is known as an “if-let statement”.

Motivation

Many times in the past, people have proposed various mechanisms for doing a refutable let-binding. None of them went anywhere, largely because the syntax wasn’t great, or because the suggestion introduced runtime failure if the pattern match failed.

This proposal ties the refutable pattern match to the pre-existing conditional construct (i.e. if statement), which provides a clear and intuitive explanation for why refutable patterns are allowed here (as opposed to a let statement which disallows them) and how to behave if the pattern doesn’t match.

The motivation for having any construct at all for this is to simplify the cases that today call for a match statement with a single non-trivial case. This is predominately used for unwrapping Option<T> values, but can be used elsewhere.

The idiomatic solution today for testing and unwrapping an Option<T> looks like

match optVal {
    Some(x) => {
        doSomethingWith(x);
    }
    None => {}
}

This is unnecessarily verbose, with the None => {} (or _ => {}) case being required, and introduces unnecessary rightward drift (this introduces two levels of indentation where a normal conditional would introduce one).

The alternative approach looks like this:

if optVal.is_some() {
    let x = optVal.unwrap();
    doSomethingWith(x);
}

This is generally considered to be a less idiomatic solution than the match. It has the benefit of fixing rightward drift, but it ends up testing the value twice (which should be optimized away, but semantically speaking still happens), with the second test being a method that potentially introduces failure. From context, the failure won’t happen, but it still imposes a semantic burden on the reader. Finally, it requires having a pre-existing let-binding for the optional value; if the value is a temporary, then a new let-binding in the parent scope is required in order to be able to test and unwrap in two separate expressions.

The if let construct solves all of these problems, and looks like this:

if let Some(x) = optVal {
    doSomethingWith(x);
}

Detailed design

The if let construct is based on the precedent set by Swift, which introduced its own if let statement. In Swift, if let var = expr { ... } is directly tied to the notion of optional values, and unwraps the optional value that expr evaluates to. In this proposal, the equivalent is if let Some(var) = expr { ... }.

Given the following rough grammar for an if condition:

if-expr     = 'if' if-cond block else-clause?
if-cond     = expression
else-clause = 'else' block | 'else' if-expr

The grammar is modified to add the following productions:

if-cond = 'let' pattern '=' expression

The expression is restricted to disallow a trailing braced block (e.g. for struct literals) the same way the expression in the normal if statement is, to avoid ambiguity with the then-block.

Contrary to a let statement, the pattern in the if let expression allows refutable patterns. The compiler should emit a warning for an if let expression with an irrefutable pattern, with the suggestion that this should be turned into a regular let statement.

Like the for loop before it, this construct can be transformed in a syntax-lowering pass into the equivalent match statement. The expression is given to match and the pattern becomes a match arm. If there is an else block, that becomes the body of the _ => {} arm, otherwise _ => {} is provided.

Optionally, one or more else if (not else if let) blocks can be placed in the same match using pattern guards on _. This could be done to simplify the code when pretty-printing the expansion result. Otherwise, this is an unnecessary transformation.

Due to some uncertainty regarding potentially-surprising fallout of AST rewrites, and some worries about exhaustiveness-checking (e.g. a tautological if let would be an error, which may be unexpected), this is put behind a feature gate named if_let.

Examples

Source:

if let Some(x) = foo() {
    doSomethingWith(x)
}

Result:

match foo() {
    Some(x) => {
        doSomethingWith(x)
    }
    _ => {}
}

Source:

if let Some(x) = foo() {
    doSomethingWith(x)
} else {
    defaultBehavior()
}

Result:

match foo() {
    Some(x) => {
        doSomethingWith(x)
    }
    _ => {
        defaultBehavior()
    }
}

Source:

if cond() {
    doSomething()
} else if let Some(x) = foo() {
    doSomethingWith(x)
} else {
    defaultBehavior()
}

Result:

if cond() {
    doSomething()
} else {
    match foo() {
        Some(x) => {
            doSomethingWith(x)
        }
        _ => {
            defaultBehavior()
        }
    }
}

With the optional addition specified above:

if let Some(x) = foo() {
    doSomethingWith(x)
} else if cond() {
    doSomething()
} else if other_cond() {
    doSomethingElse()
}

Result:

match foo() {
    Some(x) => {
        doSomethingWith(x)
    }
    _ if cond() => {
        doSomething()
    }
    _ if other_cond() => {
        doSomethingElse()
    }
    _ => {}
}

Drawbacks

It’s one more addition to the grammar.

Alternatives

This could plausibly be done with a macro, but the invoking syntax would be pretty terrible and would largely negate the whole point of having this sugar.

Alternatively, this could not be done at all. We’ve been getting alone just fine without it so far, but at the cost of making Option just a bit more annoying to work with.

Unresolved questions

It’s been suggested that alternates or pattern guards should be allowed. I think if you need those you could just go ahead and use a match, and that if let could be extended to support those in the future if a compelling use-case is found.

I don’t know how many match statements in our current code base could be replaced with this syntax. Probably quite a few, but it would be informative to have real data on this.

• Start Date: 2014-07-14
• RFC PR #: rust-lang/rfcs#164
• Rust Issue #: rust-lang/rust#16951

Summary

Rust’s support for pattern matching on slices has grown steadily and incrementally without a lot of oversight. We have concern that Rust is doing too much here, and that the complexity is not worth it. This RFC proposes to feature gate multiple-element slice matches in the head and middle positions ([xs.., 0, 0] and [0, xs.., 0]).

Motivation

Some general reasons and one specific: first, the implementation of Rust’s match machinery is notoriously complex, and not well-loved. Removing features is seen as a valid way to reduce complexity. Second, slice matching in particular, is difficult to implement, while also being of only moderate utility (there are many types of collections - slices just happen to be built into the language). Finally, the exhaustiveness check is not correct for slice patterns because of their complexity; it’s not known if it can be done correctly, nor whether it is worth the effort to do so.

Detailed design

The advanced_slice_patterns feature gate will be added. When the compiler encounters slice pattern matches in head or middle position it will emit a warning or error according to the current settings.

Drawbacks

It removes two features that some people like.

Alternatives

Fixing the exhaustiveness check would allow the feature to remain.

Unresolved questions

N/A

• Start Date: 2014-06-06
• RFC PR: rust-lang/rfcs#168
• Rust Issue: rust-lang/rust#15722
• Author: Tommit (edited by nrc)

Summary

Add syntax sugar for importing a module and items in that module in a single view item.

Motivation

Make use clauses more concise.

Detailed design

The mod keyword may be used in a braced list of modules in a use item to mean the prefix module for that list. For example, writing prefix::{mod, foo}; is equivalent to writing

use prefix;
use prefix::foo;

The mod keyword cannot be used outside of braces, nor can it be used inside braces which do not have a prefix path. Both of the following examples are illegal:

use module::mod;
use {mod, foo};

A programmer may write mod in a module list with only a single item. E.g., use prefix::{mod};, although this is considered poor style and may be forbidden by a lint. (The preferred version is use prefix;).

Drawbacks

Another use of the mod keyword.

We introduce a way (the only way) to have paths in use items which do not correspond with paths which can be used in the program. For example, with use foo::bar::{mod, baz}; the programmer can use foo::bar::baz in their program but not foo::bar::mod (instead foo::bar is imported).

Alternatives

Don’t do this.

Unresolved questions

N/A

• Start Date: 2014-07-16
• RFC PR #: #169
• Rust Issue #: https://github.com/rust-lang/rust/issues/16461

Summary

Change the rebinding syntax from use ID = PATH to use PATH as ID, so that paths all line up on the left side, and imported identifiers are all on the right side. Also modify extern crate syntax analogously, for consistency.

Motivation

Currently, the view items at the start of a module look something like this:

mod old_code {
  use a::b::c::d::www;
  use a::b::c::e::xxx;
  use yyy = a::b::yummy;
  use a::b::c::g::zzz;
}

This means that if you want to see what identifiers have been imported, your eyes need to scan back and forth on both the left-hand side (immediately beside the use) and the right-hand side (at the end of each line). In particular, note that yummy is not in scope within the body of old_code

This RFC proposes changing the grammar of Rust so that the example above would look like this:

mod new_code {
  use a::b::c::d::www;
  use a::b::c::e::xxx;
  use a::b::yummy as yyy;
  use a::b::c::g::zzz;
}

There are two benefits we can see by comparing mod old_code and mod new_code:

• As alluded to above, now all of the imported identifiers are on the right-hand side of the block of view items.

• Additionally, the left-hand side looks much more regular, since one sees the straight lines of a::b:: characters all the way down, which makes the actual differences between the different paths more visually apparent.

Detailed design

Currently, the grammar for use statements is something like:

  use_decl : "pub" ? "use" [ ident '=' path
                            | path_glob ] ;

Likewise, the grammar for extern crate declarations is something like:

  extern_crate_decl : "extern" "crate" ident [ '(' link_attrs ')' ] ? [ '=' string_lit ] ? ;

This RFC proposes changing the grammar for use statements to something like:

  use_decl : "pub" ? "use" [ path "as" ident
                            | path_glob ] ;

and the grammar for extern crate declarations to something like:

  extern_crate_decl : "extern" "crate" [ string_lit "as" ] ? ident [ '(' link_attrs ')' ] ? ;

Both use and pub use forms are changed to use path as ident instead of ident = path. The form use path as ident has the same constraints and meaning that use ident = path has today.

Nothing about path globs is changed; the view items that use ident = path are disjoint from the view items that use path globs, and that continues to be the case under path as ident.

The old syntaxes "use" ident '=' path and "extern" "crate" ident '=' string_lit are removed (or at least deprecated).

Drawbacks

• pub use export = import_path may be preferred over pub use import_path as export since people are used to seeing the name exported by a pub item on the left-hand side of an = sign. (See “Have distinct rebinding syntaxes for use and pub use” below.)

• The ‘as’ keyword is not currently used for any binding form in Rust. Adopting this RFC would change that precedent. (See “Change the signaling token” below.)

Alternatives

Keep things as they are

This just has the drawbacks outlined in the motivation: the left-hand side of the view items are less regular, and one needs to scan both the left- and right-hand sides to see all the imported identifiers.

Change the signaling token

Go ahead with switch, so imported identifier is on the left-hand side, but use a different token than as to signal a rebinding.

For example, we could use @, as an analogy with its use as a binding operator in match expressions:

mod new_code {
  use a::b::c::d::www;
  use a::b::c::e::xxx;
  use a::b::yummy @ yyy;
  use a::b::c::g::zzz;
}

(I do not object to path @ ident, though I find it somehow more “line-noisy” than as in this context.)

Or, we could use =:

mod new_code {
  use a::b::c::d::www;
  use a::b::c::e::xxx;
  use a::b::yummy = yyy;
  use a::b::c::g::zzz;
}

(I do object to path = ident, since typically when = is used to bind, the identifier being bound occurs on the left-hand side.)

Or, we could use :, by (weak) analogy with struct pattern syntax:

mod new_code {
  use a::b::c::d::www;
  use a::b::c::e::xxx;
  use a::b::yummy : yyy;
  use a::b::c::g::zzz;
}

(I cannot figure out if this is genius or madness. Probably madness, especially if one is allowed to omit the whitespace around the :)

Have distinct rebinding syntaxes for use and pub use

If people really like having ident = path for pub use, by the reasoning presented above that people are used to seeing the name exported by a pub item on the left-hand side of an = sign, then we could support that by continuing to support pub use ident = path.

If we were to go down that route, I would prefer to have distinct notions of the exported name and imported name, so that:

pub use a = foo::bar; would actually import bar (and a would just be visible as an export), and then one could rebind for export and import simultaneously, like so: pub use exported_bar = foo::bar as imported_bar;

But really, is pub use foo::bar as a all that bad?

Allow extern crate ident as ident

As written, this RFC allows for two variants of extern_crate_decl:

extern crate old_name;
extern crate "old_name" as new_name;

These are just analogous to the current options that use = instead of as.

However, the RFC comment dialogue suggested also allowing a renaming form that does not use a string literal:

extern crate old_name as new_name;

I have no opinion on whether this should be added or not. Arguably this choice is orthogonal to the goals of this RFC (since, if this is a good idea, it could just as well be implemented with the = syntax). Perhaps it should just be filed as a separate RFC on its own.

Unresolved questions

• In the revised extern crate form, is it best to put the link_attrs after the identifier, as written above? Or would it be better for them to come after the string_literal when using the extern crate string_literal as ident form?

• Start Date: 23-07-2014
• RFC PR: rust-lang/rfcs#179
• Rust Issue: rust-lang/rust#20496

Summary

Change pattern matching on an &mut T to &mut <pat>, away from its current &<pat> syntax.

Motivation

Pattern matching mirrors construction for almost all types, except &mut, which is constructed with &mut <expr> but destructured with &<pat>. This is almost certainly an unnecessary inconsistency.

This can and does lead to confusion, since people expect the pattern syntax to match construction, but a pattern like &mut (ref mut x, _) is actually currently a parse error:

fn main() {
    let &mut (ref mut x, _);
}

and-mut-pat.rs:2:10: 2:13 error: expected identifier, found path
and-mut-pat.rs:2     let &mut (ref mut x, _);
                          ^~~

Another (rarer) way it can be confusing is the pattern &mut x. It is expected that this binds x to the contents of &mut T pointer… which it does, but as a mutable binding (it is parsed as &(mut x)), meaning something like

for &mut x in some_iterator_over_and_mut {
    println!("{}", x)
}

gives an unused mutability warning. NB. it’s somewhat rare that one would want to pattern match to directly bind a name to the contents of a &mut (since the normal reason to have a &mut is to mutate the thing it points at, but this pattern is (byte) copying the data out, both before and after this change), but can occur if a type only offers a &mut iterator, i.e. types for which a & one is no more flexible than the &mut one.

Detailed design

Add <pat> := &mut <pat> to the pattern grammar, and require that it is used when matching on a &mut T.

Drawbacks

It makes matching through a &mut more verbose: for &mut (ref mut x, p_) in v.mut_iter() instead of for &(ref mut x, _) in v.mut_iter().

Macros wishing to pattern match on either & or &mut need to handle each case, rather than performing both with a single &. However, macros handling these types already need special mut vs. not handling if they ever name the types, or if they use ref vs. ref mut subpatterns.

It also makes obtaining the current behaviour (binding by-value the contents of a reference to a mutable local) slightly harder. For a &mut T the pattern becomes &mut mut x, and, at the moment, for a &T, it must be matched with &x and then rebound with let mut x = x; (since disambiguating like &(mut x) doesn’t yet work). However, based on some loose grepping of the Rust repo, both of these are very rare.

Alternatives

None.

Unresolved questions

None.

• Start Date: 2014-07-24
• RFC PR #: https://github.com/rust-lang/rfcs/pull/184
• Rust Issue #: https://github.com/rust-lang/rust/issues/16950

Summary

Add simple syntax for accessing values within tuples and tuple structs behind a feature gate.

Motivation

Right now accessing fields of tuples and tuple structs is incredibly painful—one must rely on pattern-matching alone to extract values. This became such a problem that twelve traits were created in the standard library (core::tuple::Tuple*) to make tuple value accesses easier, adding .valN(), .refN(), and .mutN() methods to help this. But this is not a very nice solution—it requires the traits to be implemented in the standard library, not the language, and for those traits to be imported on use. On the whole this is not a problem, because most of the time std::prelude::* is imported, but this is still a hack which is not a real solution to the problem at hand. It also only supports tuples of length up to twelve, which is normally not a problem but emphasises how bad the current situation is.

Detailed design

Add syntax of the form <expr>.<integer> for accessing values within tuples and tuple structs. This (and the functionality it provides) would only be allowed when the feature gate tuple_indexing is enabled. This syntax is recognised wherever an unsuffixed integer literal is found in place of the normal field or method name expected when accessing fields with .. Because the parser would be expecting an integer, not a float, an expression like expr.0.1 would be a syntax error (because 0.1 would be treated as a single token).

Tuple/tuple struct field access behaves the same way as accessing named fields on normal structs:

// With tuple struct
struct Foo(int, int);
let mut foo = Foo(3, -15);
foo.0 = 5;
assert_eq!(foo.0, 5);

// With normal struct
struct Foo2 { _0: int, _1: int }
let mut foo2 = Foo2 { _0: 3, _1: -15 };
foo2._0 = 5;
assert_eq!(foo2._0, 5);

Effectively, a tuple or tuple struct field is just a normal named field with an integer for a name.

Drawbacks

This adds more complexity that is not strictly necessary.

Alternatives

Stay with the status quo. Either recommend using a struct with named fields or suggest using pattern-matching to extract values. If extracting individual fields of tuples is really necessary, the TupleN traits could be used instead, and something like #[deriving(Tuple3)] could possibly be added for tuple structs.

Unresolved questions

None.

• Start Date: 2014-08-06
• RFC PR: rust-lang/rfcs#192
• Rust Issue: rust-lang/rust#16462

Summary

• Remove the special-case bound 'static and replace with a generalized lifetime bound that can be used on objects and type parameters.
• Remove the rules that aim to prevent references from being stored into objects and replace with a simple lifetime check.
• Tighten up type rules pertaining to reference lifetimes and well-formed types containing references.
• Introduce explicit lifetime bounds ('a:'b), with the meaning that the lifetime 'a outlives the lifetime 'b. These exist today but are always inferred; this RFC adds the ability to specify them explicitly, which is sometimes needed in more complex cases.

Motivation

Currently, the type system is not supposed to allow references to escape into object types. However, there are various bugs where it fails to prevent this from happening. Moreover, it is very useful (and frequently necessary) to store a reference into an object. Moreover, the current treatment of generic types is in some cases naive and not obviously sound.

Detailed design

Lifetime bounds on parameters

The heart of the new design is the concept of a lifetime bound. In fact, this (sort of) exists today in the form of the 'static bound:

 fn foo<A:'static>(x: A) { ... }

Here, the notation 'static means “all borrowed content within A outlives the lifetime 'static”. (Note that when we say that something outlives a lifetime, we mean that it lives at least that long. In other words, for any lifetime 'a, 'a outlives 'a. This is similar to how we say that every type T is a subtype of itself.)

In the newer design, it is possible to use an arbitrary lifetime as a bound, and not just 'static:

 fn foo<'a, A:'a>(x: A) { ... }

Explicit lifetime bounds are in fact only rarely necessary, for two reasons:

1. The compiler is often able to infer this relationship from the argument and return types. More on this below.
2. It is only important to bound the lifetime of a generic type like A when one of two things is happening (and both of these are cases where the inference generally is sufficient):
   • A borrowed pointer to an A instance (i.e., value of type &A) is being consumed or returned.
   • A value of type A is being closed over into an object reference (or closure, which per the unboxed closures RFC is really the same thing).

Note that, per RFC 11, these lifetime bounds may appear in types as well (this is important later on). For example, an iterator might be declared:

struct Items<'a, T:'a> {
    v: &'a Collection<T>
}

Here, the constraint T:'a indicates that the data being iterated over must live at least as long as the collection (logically enough).

Lifetime bounds on object types

Like parameters, all object types have a lifetime bound. Unlike parameter types, however, object types are required to have exactly one bound. This bound can be either specified explicitly or derived from the traits that appear in the object type. In general, the rule is as follows:

• If an explicit bound is specified, use that.
• Otherwise, let S be the set of lifetime bounds we can derive.
• Otherwise, if S contains ’static, use ’static.
• Otherwise, if S is a singleton set, use that.
• Otherwise, error.

Here are some examples:

trait IsStatic : 'static { }
trait Is<'a> : 'a { }

// Type               Bounds
// IsStatic           'static
// Is<'a>             'a
// IsStatic+Is<'a>    'static+'a
// IsStatic+'a        'static+'a
// IsStatic+Is<'a>+'b 'static,'a,'b

Object types must have exactly one bound – zero bounds is not acceptable. Therefore, if an object type with no derivable bounds appears, we will supply a default lifetime using the normal rules:

trait Writer { /* no derivable bounds */ }
struct Foo<'a> {
    Box<Writer>,      // Error: try Box<Writer+'static> or Box<Writer+'a>
    Box<Writer+Send>, // OK: Send implies 'static
    &'a Writer,       // Error: try &'a (Writer+'a)
}

fn foo(a: Box<Writer>, // OK: Sugar for Box<Writer+'a> where 'a fresh
       b: &Writer)     // OK: Sugar for &'b (Writer+'c) where 'b, 'c fresh
{ ... }

This kind of annotation can seem a bit tedious when using object types extensively, though type aliases can help quite a bit:

type WriterObj = Box<Writer+'static>;
type WriterRef<'a> = &'a (Writer+'a);

The unresolved questions section discussed possibles ways to lighten the burden.

See Appendix B for the motivation on why object types are permitted to have exactly one lifetime bound.

Specifying relations between lifetimes

Currently, when a type or fn has multiple lifetime parameters, there is no facility to explicitly specify a relationship between them. For example, in a function like this:

fn foo<'a, 'b>(...) { ... }

the lifetimes 'a and 'b are declared as independent. In some cases, though, it can be important that there be a relation between them. In most cases, these relationships can be inferred (and in fact are inferred today, see below), but it is useful to be able to state them explicitly (and necessary in some cases, see below).

A lifetime bound is written 'a:'b and it means that “'a outlives 'b”. For example, if foo were declared like so:

fn foo<'x, 'y:'x>(...) { ... }

that would indicate that the lifetime ’x was shorter than (or equal to) 'y.

The “type must outlive” and well-formedness relation

Many of the rules to come make use of a “type must outlive” relation, written T outlives 'a. This relation means primarily that all borrowed data in T is known to have a lifetime of at least ’a (hence the name). However, the relation also guarantees various basic lifetime constraints are met. For example, for every reference type &'b U that is found within T, it would be required that U outlives 'b (and that 'b outlives 'a).

In fact, T outlives 'a is defined on another function WF(T:'a), which yields up a list of lifetime relations that must hold for T to be well-formed and to outlive 'a. It is not necessary to understand the details of this relation in order to follow the rest of the RFC, I will defer its precise specification to an appendix below.

For this section, it suffices to give some examples:

// int always outlives any region
WF(int : 'a) = []

// a reference with lifetime 'a outlives 'b if 'a outlives 'b
WF(&'a int : 'b) = ['a : 'b]

// the outer reference must outlive 'c, and the inner reference
// must outlive the outer reference
WF(&'a &'b int : 'c) = ['a : 'c, 'b : 'a]

// Object type with bound 'static
WF(SomeTrait+'static : 'a) = ['static : 'a]

// Object type with bound 'a 
WF(SomeTrait+'a : 'b) = ['a : 'b]

Rules for when object closure is legal

Whenever data of type T is closed over to form an object, the type checker will require that T outlives 'a where 'a is the primary lifetime bound of the object type.

Rules for types to be well-formed

Currently we do not apply any tests to the types that appear in type declarations. Per RFC 11, however, this should change, as we intend to enforce trait bounds on types, wherever those types appear. Similarly, we should be requiring that types are well-formed with respect to the WF function. This means that a type like the following would be illegal without a lifetime bound on the type parameter T:

struct Ref<'a, T> { c: &'a T }

This is illegal because the field c has type &'a T, which is only well-formed if T:'a. Per usual practice, this RFC does not propose any form of inference on struct declarations and instead requires all conditions to be spelled out (this is in contrast to fns and methods, see below).

Rules for expression type validity

We should add the condition that for every expression with lifetime 'e and type T, then T outlives 'e. We already enforce this in many special cases but not uniformly.

Inference

The compiler will infer lifetime bounds on both type parameters and region parameters as follows. Within a function or method, we apply the wellformedness function WF to each function or parameter type. This yields up a set of relations that must hold. The idea here is that the caller could not have type checked unless the types of the arguments were well-formed, so that implies that the callee can assume that those well-formedness constraints hold.

As an example, in the following function:

fn foo<'a, A>(x: &'a A) { ... }

the callee here can assume that the type parameter A outlives the lifetime 'a, even though that was not explicitly declared.

Note that the inference also pulls in constraints that were declared on the types of arguments. So, for example, if there is a type Items declared as follows:

struct Items<'a, T:'a> { ... }

And a function that takes an argument of type Items:

fn foo<'a, T>(x: Items<'a, T>) { ... }

The inference rules will conclude that T:'a because the Items type was declared with that bound.

In practice, these inference rules largely remove the need to manually declare lifetime relations on types. When porting the existing library and rustc over to these rules, I had to add explicit lifetime bounds to exactly one function (but several types, almost exclusively iterators).

Note that this sort of inference is already done. This RFC simply proposes a more extensive version that also includes bounds of the form X:'a, where X is a type parameter.

What does all this mean in practice?

This RFC has a lot of details. The main implications for end users are:

1. Object types must specify a lifetime bound when they appear in a type. This most commonly means changing Box<Trait> to Box<Trait+'static> and &'a Trait to &'a Trait+'a.

2. For types that contain references to generic types, lifetime bounds are needed in the type definition. This comes up most often in iterators:

   struct Items<'a, T:'a> {
       x: &'a [T]
   }
   

   Here, the presence of &'a [T] within the type definition requires that the type checker can show that T outlives 'a which in turn requires the bound T:'a on the type definition. These bounds are rarely outside of type definitions, because they are almost always implied by the types of the arguments.

3. It is sometimes, but rarely, necessary to use lifetime bounds, specifically around double indirections (references to references, often the second reference is contained within a struct). For example:

   struct GlobalContext<'global> {
       arena: &'global Arena
   }
   
   struct LocalContext<'local, 'global:'local> {
       x: &'local mut Context<'global>
   }
   

   Here, we must know that the lifetime 'global outlives 'local in order for this type to be well-formed.

Phasing

Some parts of this RFC require new syntax and thus must be phased in. The current plan is to divide the implementation three parts:

1. Implement support for everything in this RFC except for region bounds and requiring that every expression type be well-formed. Enforcing the latter constraint leads to type errors that require lifetime bounds to resolve.
2. Implement support for 'a:'b notation to be parsed under a feature gate issue_5723_bootstrap.
3. Implement the final bits of the RFC:
   • Bounds on lifetime parameters
   • Wellformedness checks on every expression
   • Wellformedness checks in type definitions

Parts 1 and 2 can be landed simultaneously, but part 3 requires a snapshot. Parts 1 and 2 have largely been written. Depending on precisely how the timing works out, it might make sense to just merge parts 1 and 3.

Drawbacks / Alternatives

If we do not implement some solution, we could continue with the current approach (but patched to be sound) of banning references from being closed over in object types. I consider this a non-starter.

Unresolved questions

Inferring wellformedness bounds

Under this RFC, it is required to write bounds on struct types which are in principle inferable from their contents. For example, iterators tend to follow a pattern like:

struct Items<'a, T:'a> {
    x: &'a [T]
}

Note that T is bounded by 'a. It would be possible to infer these bounds, but I’ve stuck to our current principle that type definitions are always fully spelled out. The danger of inference is that it becomes unclear why a particular constraint exists if one must traverse the type hierarchy deeply to find its origin. This could potentially be addressed with better error messages, though our track record for lifetime error messages is not very good so far.

Also, there is a potential interaction between this sort of inference and the description of default trait bounds below.

Default trait bounds

When referencing a trait object, it is almost always the case that one follows certain fixed patterns:

• Box<Trait+'static>
• Rc<Trait+'static> (once DST works)
• &'a (Trait+'a)
• and so on.

You might think that we should simply provide some kind of defaults that are sensitive to where the Trait appears. The same is probably true of struct type parameters (in other words, &'a SomeStruct<'a> is a very common pattern).

However, there are complications:

• What about a type like struct Ref<'a, T:'a> { x: &'a T }? Ref<'a, Trait> should really work the same way as &'a Trait. One way that I can see to do this is to drive the defaulting based on the default trait bounds of the T type parameter – but if we do that, it is both a non-local default (you have to consult the definition of Ref) and interacts with the potential inference described in the previous section.

• There are reasons to want a type like Box<Trait+'a>. For example, the macro parser includes a function like:

  fn make_macro_ext<'cx>(cx: &'cx Context, ...) -> Box<MacroExt+'cx>
  

  In other words, this function returns an object that closes over the macro context. In such a case, if Box<MacroExt> implies a static bound, then taking ownership of this macro object would require a signature like:

  fn take_macro_ext<'cx>(b: Box<MacroExt+'cx>) {  }
  

  Note that the 'cx variable is only used in one place. It’s purpose is just to disable the 'static default that would otherwise be inserted.

Appendix: Definition of the outlives relation and well-formedness

To make this more specific, we can “formally” model the Rust type system as:

T = scalar (int, uint, fn(...))   // Boring stuff
  | *const T                      // Unsafe pointer
  | *mut T                        // Unsafe pointer
  | Id<P>                         // Nominal type (struct, enum)
  | &'x T                         // Reference
  | &'x mut T                     // Mutable reference
  | {TraitReference<P>}+'x        // Object type
  | X                             // Type variable
P = {'x} + {T}

We can define a function WF(T : 'a) which, given a type T and lifetime 'a yields a list of 'b:'c or X:'d pairs. For each pair 'b:'c, the lifetime 'b must outlive the lifetime 'c for the type T to be well-formed in a location with lifetime 'a. For each pair X:'d, the type parameter X must outlive the lifetime 'd.

• WF(int : 'a) yields an empty list
• WF(X:'a) where X is a type parameter yields (X:'a).
• WF(Foo<P>:'a) where Foo<P> is an enum or struct type yields:
  • For each lifetime parameter 'b that is contravariant or invariant, 'b : 'a.
  • For each type parameter T that is covariant or invariant, the results of WF(T : 'a).
  • The lifetime bounds declared on Foo’s lifetime or type parameters.
  • The reasoning here is that if we can reach borrowed data with lifetime 'a through Foo<'a>, then 'a must be contra- or invariant. Covariant lifetimes only occur in “setter” situations. Analogous reasoning applies to the type case.
• WF(T:'a) where T is an object type:
  • For the primary bound 'b, 'b : 'a.
  • For each derived bound 'c of T, 'b : 'c
    • Motivation: The primary bound of an object type implies that all other bounds are met. This simplifies some of the other formulations and does not represent a loss of expressiveness.

We can then say that T outlives 'a if all lifetime relations returned by WF(T:'a) hold.

Appendix B: Why object types must have exactly one bound

The motivation is that handling multiple bounds is overwhelmingly complicated to reason about and implement. In various places, constraints arise of the form all i. exists j. R[i] <= R[j], where R is a list of lifetimes. This is challenging for lifetime inference, since there are many options for it to choose from, and thus inference is no longer a fixed-point iteration. Moreover, it doesn’t seem to add any particular expressiveness.

The places where this becomes important are:

• Checking lifetime bounds when data is closed over into an object type
• Subtyping between object types, which would most naturally be contravariant in the lifetime bound

Similarly, requiring that the “master” bound on object lifetimes outlives all other bounds also aids inference. Now, given a type like the following:

trait Foo<'a> : 'a { }
trait Bar<'b> : 'b { }

...

let x: Box<Foo<'a>+Bar<'b>>

the inference engine can create a fresh lifetime variable '0 for the master bound and then say that '0:'a and '0:'b. Without the requirement that '0 be a master bound, it would be somewhat unclear how '0 relates to 'a and 'b (in fact, there would be no necessary relation). But if there is no necessary relation, then when closing over data, one would have to ensure that the closed over data outlives all derivable lifetime bounds, which again creates a constraint of the form all i. exists j..

• Start Date: 2014-08-09
• RFC PR #: rust-lang/rfcs#194
• Rust Issue: rust-lang/rust#17490

Summary

The #[cfg(...)] attribute provides a mechanism for conditional compilation of items in a Rust crate. This RFC proposes to change the syntax of #[cfg] to make more sense as well as enable expansion of the conditional compilation system to attributes while maintaining a single syntax.

Motivation

In the current implementation, #[cfg(...)] takes a comma separated list of key, key = "value", not(key), or not(key = "value"). An individual #[cfg(...)] attribute “matches” if all of the contained cfg patterns match the compilation environment, and an item preserved if it either has no #[cfg(...)] attributes or any of the #[cfg(...)] attributes present match.

This is problematic for several reasons:

• It is excessively verbose in certain situations. For example, implementing the equivalent of (a AND (b OR c OR d)) requires three separate attributes and a to be duplicated in each.
• It differs from all other attributes in that all #[cfg(...)] attributes on an item must be processed together instead of in isolation. This change will move #[cfg(...)] closer to implementation as a normal syntax extension.

Detailed design

The <p> inside of #[cfg(<p>)] will be called a cfg pattern and have a simple recursive syntax:

• key is a cfg pattern and will match if key is present in the compilation environment.
• key = "value" is a cfg pattern and will match if a mapping from key to value is present in the compilation environment. At present, key-value pairs only exist for compiler defined keys such as target_os and endian.
• not(<p>) is a cfg pattern if <p> is and matches if <p> does not match.
• all(<p>, ...) is a cfg pattern if all of the comma-separated <p>s are cfg patterns and all of them match.
• any(<p>, ...) is a cfg pattern if all of the comma-separated <p>s are cfg patterns and any of them match.

If an item is tagged with #[cfg(<p>)], that item will be stripped from the AST if the cfg pattern <p> does not match.

One implementation hazard is that the semantics of

#[cfg(a)]
#[cfg(b)]
fn foo() {}

will change from “include foo if either of a and b are present in the compilation environment” to “include foo if both of a and b are present in the compilation environment”. To ease the transition, the old semantics of multiple #[cfg(...)] attributes will be maintained as a special case, with a warning. After some reasonable period of time, the special case will be removed.

In addition, #[cfg(a, b, c)] will be accepted with a warning and be equivalent to #[cfg(all(a, b, c))]. Again, after some reasonable period of time, this behavior will be removed as well.

The cfg!() syntax extension will be modified to accept cfg patterns as well. A #[cfg_attr(<p>, <attr>)] syntax extension will be added (PR 16230) which will expand to #[<attr>] if the cfg pattern <p> matches. The test harness’s #[ignore] attribute will have its built-in cfg filtering functionality stripped in favor of #[cfg_attr(<p>, ignore)].

Drawbacks

While the implementation of this change in the compiler will be straightforward, the effects on downstream code will be significant, especially in the standard library.

Alternatives

all and any could be renamed to and and or, though I feel that the proposed names read better with the function-like syntax and are consistent with Iterator::all and Iterator::any.

Issue #2119 proposed the addition of || and && operators and parentheses to the attribute syntax to result in something like #[cfg(a || (b && c)]. I don’t favor this proposal since it would result in a major change to the attribute syntax for relatively little readability gain.

Unresolved questions

How long should multiple #[cfg(...)] attributes on a single item be forbidden? It should probably be at least until after 0.12 releases.

Should we permanently keep the behavior of treating #[cfg(a, b)] as #[cfg(all(a, b))]? It is the common case, and adding this interpretation can reduce the noise level a bit. On the other hand, it may be a bit confusing to read as it’s not immediately clear if it will be processed as and(..) or all(..).

• Start Date: 2014-08-04
• RFC PR #: rust-lang/rfcs#195
• Rust Issue #: rust-lang/rust#17307

Summary

This RFC extends traits with associated items, which make generic programming more convenient, scalable, and powerful. In particular, traits will consist of a set of methods, together with:

• Associated functions (already present as “static” functions)
• Associated consts
• Associated types
• Associated lifetimes

These additions make it much easier to group together a set of related types, functions, and constants into a single package.

This RFC also provides a mechanism for multidispatch traits, where the impl is selected based on multiple types. The connection to associated items will become clear in the detailed text below.

Note: This RFC was originally accepted before RFC 246 introduced the distinction between const and static items. The text has been updated to clarify that associated consts will be added rather than statics, and to provide a summary of restrictions on the initial implementation of associated consts. Other than that modification, the proposal has not been changed to reflect newer Rust features or syntax.

Motivation

A typical example where associated items are helpful is data structures like graphs, which involve at least three types: nodes, edges, and the graph itself.

In today’s Rust, to capture graphs as a generic trait, you have to take the additional types associated with a graph as parameters:

trait Graph<N, E> {
    fn has_edge(&self, &N, &N) -> bool;
    ...
}

The fact that the node and edge types are parameters is confusing, since any concrete graph type is associated with a unique node and edge type. It is also inconvenient, because code working with generic graphs is likewise forced to parameterize, even when not all of the types are relevant:

fn distance<N, E, G: Graph<N, E>>(graph: &G, start: &N, end: &N) -> uint { ... }

With associated types, the graph trait can instead make clear that the node and edge types are determined by any impl:

trait Graph {
    type N;
    type E;
    fn has_edge(&self, &N, &N) -> bool;
}

and clients can abstract over them all at once, referring to them through the graph type:

fn distance<G: Graph>(graph: &G, start: &G::N, end: &G::N) -> uint { ... }

The following subsections expand on the above benefits of associated items, as well as some others.

Associated types: engineering benefits for generics

As the graph example above illustrates, associated types do not increase the expressiveness of traits per se, because you can always use extra type parameters to a trait instead. However, associated types provide several engineering benefits:

• Readability and scalability

  Associated types make it possible to abstract over a whole family of types at once, without having to separately name each of them. This improves the readability of generic code (like the distance function above). It also makes generics more “scalable”: traits can incorporate additional associated types without imposing an extra burden on clients that don’t care about those types.

  In today’s Rust, by contrast, adding additional generic parameters to a trait often feels like a very “heavyweight” move.

• Ease of refactoring/evolution

  Because users of a trait do not have to separately parameterize over its associated types, new associated types can be added without breaking all existing client code.

  In today’s Rust, by contrast, associated types can only be added by adding more type parameters to a trait, which breaks all code mentioning the trait.

Clearer trait matching

Type parameters to traits can either be “inputs” or “outputs”:

• Inputs. An “input” type parameter is used to determine which impl to use.

• Outputs. An “output” type parameter is uniquely determined by the impl, but plays no role in selecting the impl.

Input and output types play an important role for type inference and trait coherence rules, which is described in more detail later on.

In the vast majority of current libraries, the only input type is the Self type implementing the trait, and all other trait type parameters are outputs. For example, the trait Iterator<A> takes a type parameter A for the elements being iterated over, but this type is always determined by the concrete Self type (e.g. Items<u8>) implementing the trait: A is typically an output.

Additional input type parameters are useful for cases like binary operators, where you may want the impl to depend on the types of both arguments. For example, you might want a trait

trait Add<Rhs, Sum> {
    fn add(&self, rhs: &Rhs) -> Sum;
}

to view the Self and Rhs types as inputs, and the Sum type as an output (since it is uniquely determined by the argument types). This would allow impls to vary depending on the Rhs type, even though the Self type is the same:

impl Add<int, int> for int { ... }
impl Add<Complex, Complex> for int { ... }

Today’s Rust does not make a clear distinction between input and output type parameters to traits. If you attempted to provide the two impls above, you would receive an error like:

error: conflicting implementations for trait `Add`

This RFC clarifies trait matching by:

• Treating all trait type parameters as input types, and
• Providing associated types, which are output types.

In this design, the Add trait would be written and implemented as follows:

// Self and Rhs are *inputs*
trait Add<Rhs> {
    type Sum; // Sum is an *output*
    fn add(&self, &Rhs) -> Sum;
}

impl Add<int> for int {
    type Sum = int;
    fn add(&self, rhs: &int) -> int { ... }
}

impl Add<Complex> for int {
    type Sum = Complex;
    fn add(&self, rhs: &Complex) -> Complex { ... }
}

With this approach, a trait declaration like trait Add<Rhs> { ... } is really defining a family of traits, one for each choice of Rhs. One can then provide a distinct impl for every member of this family.

Expressiveness

Associated types, lifetimes, and functions can already be expressed in today’s Rust, though it is unwieldy to do so (as argued above).

But associated consts cannot be expressed using today’s traits.

For example, today’s Rust includes a variety of numeric traits, including Float, which must currently expose constants as static functions:

trait Float {
    fn nan() -> Self;
    fn infinity() -> Self;
    fn neg_infinity() -> Self;
    fn neg_zero() -> Self;
    fn pi() -> Self;
    fn two_pi() -> Self;
    ...
}

Because these functions cannot be used in constant expressions, the modules for float types also export a separate set of constants as consts, not using traits.

Associated constants would allow the consts to live directly on the traits:

trait Float {
    const NAN: Self;
    const INFINITY: Self;
    const NEG_INFINITY: Self;
    const NEG_ZERO: Self;
    const PI: Self;
    const TWO_PI: Self;
    ...
}

Why now?

The above motivations aside, it may not be obvious why adding associated types now (i.e., pre-1.0) is important. There are essentially two reasons.

First, the design presented here is not backwards compatible, because it re-interprets trait type parameters as inputs for the purposes of trait matching. The input/output distinction has several ramifications on coherence rules, type inference, and resolution, which are all described later on in the RFC.

Of course, it might be possible to give a somewhat less ideal design where associated types can be added later on without changing the interpretation of existing trait type parameters. For example, type parameters could be explicitly marked as inputs, and otherwise assumed to be outputs. That would be unfortunate, since associated types would also be outputs – leaving the language with two ways of specifying output types for traits.

But the second reason is for the library stabilization process:

• Since most existing uses of trait type parameters are intended as outputs, they should really be associated types instead. Making promises about these APIs as they currently stand risks locking the libraries into a design that will seem obsolete as soon as associated items are added. Again, this risk could probably be mitigated with a different, backwards-compatible associated item design, but at the cost of cruft in the language itself.

• The binary operator traits (e.g. Add) should be multidispatch. It does not seem possible to stabilize them now in a way that will support moving to multidispatch later.

• There are some thorny problems in the current libraries, such as the _equiv methods accumulating in HashMap, that can be solved using associated items. (See “Defaults” below for more on this specific example.) Additional examples include traits for error propagation and for conversion (to be covered in future RFCs). Adding these traits would improve the quality and consistency of our 1.0 library APIs.

Detailed design

Trait headers

Trait headers are written according to the following grammar:

TRAIT_HEADER =
  'trait' IDENT [ '<' INPUT_PARAMS '>' ] [ ':' BOUNDS ] [ WHERE_CLAUSE ]

INPUT_PARAMS = INPUT_PARAM { ',' INPUT_PARAM }* [ ',' ]
INPUT_PARAM  = IDENT [ ':' BOUNDS ]

BOUNDS = BOUND { '+' BOUND }* [ '+' ]
BOUND  = IDENT [ '<' ARGS '>' ]

ARGS   = INPUT_ARGS
       | OUTPUT_CONSTRAINTS
       | INPUT_ARGS ',' OUTPUT_CONSTRAINTS

INPUT_ARGS = TYPE { ',' TYPE }*

OUTPUT_CONSTRAINTS = OUTPUT_CONSTRAINT { ',' OUTPUT_CONSTRAINT }*
OUTPUT_CONSTRAINT  = IDENT '=' TYPE

NOTE: The grammar for WHERE_CLAUSE and BOUND is explained in detail in the subsection “Constraining associated types” below.

All type parameters to a trait are considered inputs, and can be used to select an impl; conceptually, each distinct instantiation of the types yields a distinct trait. More details are given in the section “The input/output type distinction” below.

Trait bodies: defining associated items

Trait bodies are expanded to include three new kinds of items: consts, types, and lifetimes:

TRAIT = TRAIT_HEADER '{' TRAIT_ITEM* '}'
TRAIT_ITEM =
  ... <existing productions>
  | 'const' IDENT ':' TYPE [ '=' CONST_EXP ] ';'
  | 'type' IDENT [ ':' BOUNDS ] [ WHERE_CLAUSE ] [ '=' TYPE ] ';'
  | 'lifetime' LIFETIME_IDENT ';'

Traits already support associated functions, which had previously been called “static” functions.

The BOUNDS and WHERE_CLAUSE on associated types are obligations for the implementor of the trait, and assumptions for users of the trait:

trait Graph {
    type N: Show + Hash;
    type E: Show + Hash;
    ...
}

impl Graph for MyGraph {
    // Both MyNode and MyEdge must implement Show and Hash
    type N = MyNode;
    type E = MyEdge;
    ...
}

fn print_nodes<G: Graph>(g: &G) {
    // here, can assume G::N implements Show
    ...
}

Namespacing/shadowing for associated types

Associated types may have the same name as existing types in scope, except for type parameters to the trait:

struct Foo { ... }

trait Bar<Input> {
    type Foo; // this is allowed
    fn into_foo(self) -> Foo; // this refers to the trait's Foo

    type Input; // this is NOT allowed
}

By not allowing name clashes between input and output types, keep open the possibility of later allowing syntax like:

Bar<Input=u8, Foo=uint>

where both input and output parameters are constrained by name. And anyway, there is no use for clashing input/output names.

In the case of a name clash like Foo above, if the trait needs to refer to the outer Foo for some reason, it can always do so by using a type alias external to the trait.

Defaults

Notice that associated consts and types both permit defaults, just as trait methods and functions can provide defaults.

Defaults are useful both as a code reuse mechanism, and as a way to expand the items included in a trait without breaking all existing implementors of the trait.

Defaults for associated types, however, present an interesting question: can default methods assume the default type? In other words, is the following allowed?

trait ContainerKey : Clone + Hash + Eq {
    type Query: Hash = Self;
    fn compare(&self, other: &Query) -> bool { self == other }
    fn query_to_key(q: &Query) -> Self { q.clone() };
}

impl ContainerKey for String {
    type Query = str;
    fn compare(&self, other: &str) -> bool {
        self.as_slice() == other
    }
    fn query_to_key(q: &str) -> String {
        q.into_string()
    }
}

impl<K,V> HashMap<K,V> where K: ContainerKey {
    fn find(&self, q: &K::Query) -> &V { ... }
}

In this example, the ContainerKey trait is used to associate a “Query” type (for lookups) with an owned key type. This resolves the thorny “equiv” problem in HashMap, where the hash map keys are Strings but you want to index the hash map with &str values rather than &String values, i.e. you want the following to work:

// H: HashMap<String, SomeType>
H.find("some literal")

rather than having to write

H.find(&"some literal".to_string())`

The current solution involves duplicating the API surface with _equiv methods that use the somewhat subtle Equiv trait, but the associated type approach makes it easy to provide a simple, single API that covers the same use cases.

The defaults for ContainerKey just assume that the owned key and lookup key types are the same, but the default methods have to assume the default associated types in order to work.

For this to work, it must not be possible for an implementor of ContainerKey to override the default Query type while leaving the default methods in place, since those methods may no longer typecheck.

We deal with this in a very simple way:

• If a trait implementor overrides any default associated types, they must also override all default functions and methods.

• Otherwise, a trait implementor can selectively override individual default methods/functions, as they can today.

Trait implementations

Trait impl syntax is much the same as before, except that const, type, and lifetime items are allowed:

IMPL_ITEM =
  ... <existing productions>
  | 'const' IDENT ':' TYPE '=' CONST_EXP ';'
  | 'type' IDENT' '=' 'TYPE' ';'
  | 'lifetime' LIFETIME_IDENT '=' LIFETIME_REFERENCE ';'

Any type implementation must satisfy all bounds and where clauses in the corresponding trait item.

Referencing associated items

Associated items are referenced through paths. The expression path grammar was updated as part of UFCS, but to accommodate associated types and lifetimes we need to update the type path grammar as well.

The full grammar is as follows:

EXP_PATH
  = EXP_ID_SEGMENT { '::' EXP_ID_SEGMENT }*
  | TYPE_SEGMENT { '::' EXP_ID_SEGMENT }+
  | IMPL_SEGMENT { '::' EXP_ID_SEGMENT }+
EXP_ID_SEGMENT   = ID [ '::' '<' TYPE { ',' TYPE }* '>' ]

TY_PATH
  = TY_ID_SEGMENT { '::' TY_ID_SEGMENT }*
  | TYPE_SEGMENT { '::' TY_ID_SEGMENT }*
  | IMPL_SEGMENT { '::' TY_ID_SEGMENT }+

TYPE_SEGMENT = '<' TYPE '>'
IMPL_SEGMENT = '<' TYPE 'as' TRAIT_REFERENCE '>'
TRAIT_REFERENCE = ID [ '<' TYPE { ',' TYPE * '>' ]

Here are some example paths, along with what they might be referencing

// Expression paths ///////////////////////////////////////////////////////////////

a::b::c         // reference to a function `c` in module `a::b`
a::<T1, T2>     // the function `a` instantiated with type arguments `T1`, `T2`
Vec::<T>::new   // reference to the function `new` associated with `Vec<T>`
<Vec<T> as SomeTrait>::some_fn
                // reference to the function `some_fn` associated with `SomeTrait`,
                //   as implemented by `Vec<T>`
T::size_of      // the function `size_of` associated with the type or trait `T`
<T>::size_of    // the function `size_of` associated with `T` _viewed as a type_
<T as SizeOf>::size_of
                // the function `size_of` associated with `T`'s impl of `SizeOf`

// Type paths /////////////////////////////////////////////////////////////////////

a::b::C         // reference to a type `C` in module `a::b`
A<T1, T2>       // type A instantiated with type arguments `T1`, `T2`
Vec<T>::Iter    // reference to the type `Iter` associated with `Vec<T>
<Vec<T> as SomeTrait>::SomeType
                // reference to the type `SomeType` associated with `SomeTrait`,
                //   as implemented by `Vec<T>`

Ways to reference items

Next, we’ll go into more detail on the meaning of each kind of path.

For the sake of discussion, we’ll suppose we’ve defined a trait like the following:

trait Container {
    type E;
    fn empty() -> Self;
    fn insert(&mut self, E);
    fn contains(&self, &E) -> bool where E: PartialEq;
    ...
}

impl<T> Container for Vec<T> {
    type E = T;
    fn empty() -> Vec<T> { Vec::new() }
    ...
}

Via an ID_SEGMENT prefix

When the prefix resolves to a type

The most common way to get at an associated item is through a type parameter with a trait bound:

fn pick<C: Container>(c: &C) -> Option<&C::E> { ... }

fn mk_with_two<C>() -> C where C: Container, C::E = uint {
    let mut cont = C::empty();  // reference to associated function
    cont.insert(0);
    cont.insert(1);
    cont
}

For these references to be valid, the type parameter must be known to implement the relevant trait:

// Knowledge via bounds
fn pick<C: Container>(c: &C) -> Option<&C::E> { ... }

// ... or equivalently,  where clause
fn pick<C>(c: &C) -> Option<&C::E> where C: Container { ... }

// Knowledge via ambient constraints
struct TwoContainers<C1: Container, C2: Container>(C1, C2);
impl<C1: Container, C2: Container> TwoContainers<C1, C2> {
    fn pick_one(&self) -> Option<&C1::E> { ... }
    fn pick_other(&self) -> Option<&C2::E> { ... }
}

Note that Vec<T>::E and Vec::<T>::empty are also valid type and function references, respectively.

For cases like C::E or Vec<T>::E, the path begins with an ID_SEGMENT prefix that itself resolves to a type: both C and Vec<T> are types. In general, a path PREFIX::REST_OF_PATH where PREFIX resolves to a type is equivalent to using a TYPE_SEGMENT prefix <PREFIX>::REST_OF_PATH. So, for example, following are all equivalent:

fn pick<C: Container>(c: &C) -> Option<&C::E> { ... }
fn pick<C: Container>(c: &C) -> Option<&<C>::E> { ... }
fn pick<C: Container>(c: &C) -> Option<&<<C>::E>> { ... }

The behavior of TYPE_SEGMENT prefixes is described in the next subsection.

When the prefix resolves to a trait

However, it is possible for an ID_SEGMENT prefix to resolve to a trait, rather than a type. In this case, the behavior of an ID_SEGMENT varies from that of a TYPE_SEGMENT in the following way:

// a reference Container::insert is roughly equivalent to:
fn trait_insert<C: Container>(c: &C, e: C::E);

// a reference <Container>::insert is roughly equivalent to:
fn object_insert<E>(c: &Container<E=E>, e: E);

That is, if PREFIX is an ID_SEGMENT that resolves to a trait Trait:

• A path PREFIX::REST resolves to the item/path REST defined within Trait, while treating the type implementing the trait as a type parameter.

• A path <PREFIX>::REST treats PREFIX as a (DST-style) type, and is hence usable only with trait objects. See the UFCS RFC for more detail.

Note that a path like Container::E, while grammatically valid, will fail to resolve since there is no way to tell which impl to use. A path like Container::empty, however, resolves to a function roughly equivalent to:

fn trait_empty<C: Container>() -> C;

Via a TYPE_SEGMENT prefix

The following text is slightly changed from the UFCS RFC.

When a path begins with a TYPE_SEGMENT, it is a type-relative path. If this is the complete path (e.g., <int>), then the path resolves to the specified type. If the path continues (e.g., <int>::size_of) then the next segment is resolved using the following procedure. The procedure is intended to mimic method lookup, and hence any changes to method lookup may also change the details of this lookup.

Given a path <T>::m::...:

1. Search for members of inherent impls defined on T (if any) with the name m. If any are found, the path resolves to that item.

2. Otherwise, let IN_SCOPE_TRAITS be the set of traits that are in scope and which contain a member named m:

   • Let IMPLEMENTED_TRAITS be those traits from IN_SCOPE_TRAITS for which an implementation exists that (may) apply to T.
     • There can be ambiguity in the case that T contains type inference variables.
   • If IMPLEMENTED_TRAITS is not a singleton set, report an ambiguity error. Otherwise, let TRAIT be the member of IMPLEMENTED_TRAITS.
   • If TRAIT is ambiguously implemented for T, report an ambiguity error and request further type information.
   • Otherwise, rewrite the path to <T as Trait>::m::... and continue.

Via a IMPL_SEGMENT prefix

The following text is somewhat different from the UFCS RFC.

When a path begins with an IMPL_SEGMENT, it is a reference to an item defined from a trait. Note that such paths must always have a follow-on member m (that is, <T as Trait> is not a complete path, but <T as Trait>::m is).

To resolve the path, first search for an applicable implementation of Trait for T. If no implementation can be found – or the result is ambiguous – then report an error. Note that when T is a type parameter, a bound T: Trait guarantees that there is such an implementation, but does not count for ambiguity purposes.

Otherwise, resolve the path to the member of the trait with the substitution Self => T and continue.

This apparently straightforward algorithm has some subtle consequences, as illustrated by the following example:

trait Foo {
    type T;
    fn as_T(&self) -> &T;
}

// A blanket impl for any Show type T
impl<T: Show> Foo for T {
    type T = T;
    fn as_T(&self) -> &T { self }
}

fn bounded<U: Foo>(u: U) where U::T: Show {
    // Here, we just constrain the associated type directly
    println!("{}", u.as_T())
}

fn blanket<U: Show>(u: U) {
    // the blanket impl applies to U, so we know that `U: Foo` and
    // <U as Foo>::T = U (and, of course, U: Show)
    println!("{}", u.as_T())
}

fn not_allowed<U: Foo>(u: U) {
    // this will not compile, since <U as Trait>::T is not known to
    // implement Show
    println!("{}", u.as_T())
}

This example includes three generic functions that make use of an associated type; the first two will typecheck, while the third will not.

• The first case, bounded, places a Show constraint directly on the otherwise-abstract associated type U::T. Hence, it is allowed to assume that U::T: Show, even though it does not know the concrete implementation of Foo for U.

• The second case, blanket, places a Show constraint on the type U, which means that the blanket impl of Foo applies even though we do not know the concrete type that U will be. That fact means, moreover, that we can compute exactly what the associated type U::T will be, and know that it will satisfy Show. Coherence guarantees that that the blanket impl is the only one that could apply to U. (See the section “Impl specialization” under “Unresolved questions” for a deeper discussion of this point.)

• The third case assumes only that U: Foo, and therefore nothing is known about the associated type U::T. In particular, the function cannot assume that U::T: Show.

The resolution rules also interact with instantiation of type parameters in an intuitive way. For example:

trait Graph {
    type N;
    type E;
    ...
}

impl Graph for MyGraph {
    type N = MyNode;
    type E = MyEdge;
    ...
}

fn pick_node<G: Graph>(t: &G) -> &G::N {
    // the type G::N is abstract here
    ...
}

let G = MyGraph::new();
...
pick_node(G) // has type: <MyGraph as Graph>::N = MyNode

Assuming there are no blanket implementations of Graph, the pick_node function knows nothing about the associated type G::N. However, a client of pick_node that instantiates it with a particular concrete graph type will also know the concrete type of the value returned from the function – here, MyNode.

Scoping of trait and impl items

Associated types are frequently referred to in the signatures of a trait’s methods and associated functions, and it is natural and convenient to refer to them directly.

In other words, writing this:

trait Graph {
    type N;
    type E;
    fn has_edge(&self, &N, &N) -> bool;
    ...
}

is more appealing than writing this:

trait Graph {
    type N;
    type E;
    fn has_edge(&self, &Self::N, &Self::N) -> bool;
    ...
}

This RFC proposes to treat both trait and impl bodies (both inherent and for traits) the same way we treat mod bodies: all items being defined are in scope. In particular, methods are in scope as UFCS-style functions:

trait Foo {
    type AssocType;
    lifetime 'assoc_lifetime;
    const ASSOC_CONST: uint;
    fn assoc_fn() -> Self;

    // Note: 'assoc_lifetime and AssocType in scope:
    fn method(&self, Self) -> &'assoc_lifetime AssocType;

    fn default_method(&self) -> uint {
        // method in scope UFCS-style, assoc_fn in scope
        let _ = method(self, assoc_fn());
        ASSOC_CONST // in scope
    }
}

// Same scoping rules for impls, including inherent impls:
struct Bar;
impl Bar {
    fn foo(&self) { ... }
    fn bar(&self) {
        foo(self); // foo in scope UFCS-style
        ...
    }
}

Items from super traits are not in scope, however. See the discussion on super traits below for more detail.

These scope rules provide good ergonomics for associated types in particular, and a consistent scope model for language constructs that can contain items (like traits, impls, and modules). In the long run, we should also explore imports for trait items, i.e. use Trait::some_method, but that is out of scope for this RFC.

Note that, according to this proposal, associated types/lifetimes are not in scope for the optional where clause on the trait header. For example:

trait Foo<Input>
    // type parameters in scope, but associated types are not:
    where Bar<Input, Self::Output>: Encodable {

    type Output;
    ...
}

This setup seems more intuitive than allowing the trait header to refer directly to items defined within the trait body.

It’s also worth noting that trait-level where clauses are never needed for constraining associated types anyway, because associated types also have where clauses. Thus, the above example could (and should) instead be written as follows:

trait Foo<Input> {
    type Output where Bar<Input, Output>: Encodable;
    ...
}

Constraining associated types

Associated types are not treated as parameters to a trait, but in some cases a function will want to constrain associated types in some way. For example, as explained in the Motivation section, the Iterator trait should treat the element type as an output:

trait Iterator {
    type A;
    fn next(&mut self) -> Option<A>;
    ...
}

For code that works with iterators generically, there is no need to constrain this type:

fn collect_into_vec<I: Iterator>(iter: I) -> Vec<I::A> { ... }

But other code may have requirements for the element type:

• That it implements some traits (bounds).
• That it unifies with a particular type.

These requirements can be imposed via where clauses:

fn print_iter<I>(iter: I) where I: Iterator, I::A: Show { ... }
fn sum_uints<I>(iter: I) where I: Iterator, I::A = uint { ... }

In addition, there is a shorthand for equality constraints:

fn sum_uints<I: Iterator<A = uint>>(iter: I) { ... }

In general, a trait like:

trait Foo<Input1, Input2> {
    type Output1;
    type Output2;
    lifetime 'a;
    const C: bool;
    ...
}

can be written in a bound like:

T: Foo<I1, I2>
T: Foo<I1, I2, Output1 = O1>
T: Foo<I1, I2, Output2 = O2>
T: Foo<I1, I2, Output1 = O1, Output2 = O2>
T: Foo<I1, I2, Output1 = O1, 'a = 'b, Output2 = O2>
T: Foo<I1, I2, Output1 = O1, 'a = 'b, C = true, Output2 = O2>

The output constraints must come after all input arguments, but can appear in any order.

Note that output constraints are allowed when referencing a trait in a type or a bound, but not in an IMPL_SEGMENT path:

• As a type: fn foo(obj: Box<Iterator<A = uint>> is allowed.
• In a bound: fn foo<I: Iterator<A = uint>>(iter: I) is allowed.
• In an IMPL_SEGMENT: <I as Iterator<A = uint>>::next is not allowed.

The reason not to allow output constraints in IMPL_SEGMENT is that such paths are references to a trait implementation that has already been determined – it does not make sense to apply additional constraints to the implementation when referencing it.

Output constraints are a handy shorthand when using trait bounds, but they are a necessity for trait objects, which we discuss next.

Trait objects

When using trait objects, the Self type is “erased”, so different types implementing the trait can be used under the same trait object type:

impl Show for Foo { ... }
impl Show for Bar { ... }

fn make_vec() -> Vec<Box<Show>> {
    let f = Foo { ... };
    let b = Bar { ... };
    let mut v = Vec::new();
    v.push(box f as Box<Show>);
    v.push(box b as Box<Show>);
    v
}

One consequence of erasing Self is that methods using the Self type as arguments or return values cannot be used on trait objects, since their types would differ for different choices of Self.

In the model presented in this RFC, traits have additional input parameters beyond Self, as well as associated types that may vary depending on all of the input parameters. This raises the question: which of these types, if any, are erased in trait objects?

The approach we take here is the simplest and most conservative: when using a trait as a type (i.e., as a trait object), all input and output types must be provided as part of the type. In other words, only the Self type is erased, and all other types are specified statically in the trait object type.

Consider again the following example:

trait Foo<Input1, Input2> {
    type Output1;
    type Output2;
    lifetime 'a;
    const C: bool;
    ...
}

Unlike the case for static trait bounds, which do not have to specify any of the associated types, lifetimes, or consts, (but do have to specify the input types), trait object types must specify all of the types:

fn consume_foo<T: Foo<I1, I2>>(t: T) // this is valid
fn consume_obj(t: Box<Foo<I1, I2>>)  // this is NOT valid

// but this IS valid:
fn consume_obj(t: Box<Foo<I1, I2, Output1 = O2, Output2 = O2, 'a = 'static, C = true>>)

With this design, it is clear that none of the non-Self types are erased as part of trait objects. But it leaves wiggle room to relax this restriction later on: trait object types that are not allowed under this design can be given meaning in some later design.

Inherent associated items

All associated items are also allowed in inherent impls, so a definition like the following is allowed:

struct MyGraph { ... }
struct MyNode { ... }
struct MyEdge { ... }

impl MyGraph {
    type N = MyNode;
    type E = MyEdge;

    // Note: associated types in scope, just as with trait bodies
    fn has_edge(&self, &N, &N) -> bool {
        ...
    }

    ...
}

Inherent associated items are referenced similarly to trait associated items:

fn distance(g: &MyGraph, from: &MyGraph::N, to: &MyGraph::N) -> uint { ... }

Note, however, that output constraints do not make sense for inherent outputs:

// This is *not* a legal type:
MyGraph<N = SomeNodeType>

The input/output type distinction

When designing a trait that references some unknown type, you now have the option of taking that type as an input parameter, or specifying it as an output associated type. What are the ramifications of this decision?

Coherence implications

Input types are used when determining which impl matches, even for the same Self type:

trait Iterable1<A> {
    type I: Iterator<A>;
    fn iter(self) -> I;
}

// These impls have distinct input types, so are allowed
impl Iterable1<u8> for Foo { ... }
impl Iterable1<char> for Foo { ... }

trait Iterable2 {
    type A;
    type I: Iterator<A>;
    fn iter(self) -> I;
}

// These impls apply to a common input (Foo), so are NOT allowed
impl Iterable2 for Foo { ... }
impl Iterable2 for Foo { ... }

More formally, the coherence property is revised as follows:

• Given a trait and values for all its type parameters (inputs, including Self), there is at most one applicable impl.

In the trait reform RFC, coherence is guaranteed by maintaining two other key properties, which are revised as follows:

Orphan check: Every implementation must meet one of the following conditions:

1. The trait being implemented (if any) must be defined in the current crate.

2. At least one of the input type parameters (including but not necessarily Self) must meet the following grammar, where C is a struct or enum defined within the current crate:

   T = C
     | [T]
     | [T, ..n]
     | &T
     | &mut T
     | ~T
     | (..., T, ...)
     | X<..., T, ...> where X is not bivariant with respect to T
   

Overlapping instances: No two implementations can be instantiable with the same set of types for the input type parameters.

See the trait reform RFC for more discussion of these properties.

Type inference implications

Finally, output type parameters can be inferred/resolved as soon as there is a matching impl based on the input type parameters. Because of the coherence property above, there can be at most one.

On the other hand, even if there is only one applicable impl, type inference is not allowed to infer the input type parameters from it. This restriction makes it possible to ensure crate concatenation: adding another crate may add impls for a given trait, and if type inference depended on the absence of such impls, importing a crate could break existing code.

In practice, these inference benefits can be quite valuable. For example, in the Add trait given at the beginning of this RFC, the Sum output type is immediately known once the input types are known, which can avoid the need for type annotations.

Limitations

The main limitation of associated items as presented here is about associated types in particular. You might be tempted to write a trait like the following:

trait Iterable {
    type A;
    type I: Iterator<&'a A>; // what is the lifetime here?
    fn iter<'a>(&'a self) -> I;  // and how to connect it to self?
}

The problem is that, when implementing this trait, the return type I of iter must generally depend on the lifetime of self. For example, the corresponding method in Vec looks like the following:

impl<T> Vec<T> {
    fn iter(&'a self) -> Items<'a, T> { ... }
}

This means that, given a Vec<T>, there isn’t a single type Items<T> for iteration – rather, there is a family of types, one for each input lifetime. In other words, the associated type I in the Iterable needs to be “higher-kinded”: not just a single type, but rather a family:

trait Iterable {
    type A;
    type I<'a>: Iterator<&'a A>;
    fn iter<'a>(&self) -> I<'a>;
}

In this case, I is parameterized by a lifetime, but in other cases (like map) an associated type needs to be parameterized by a type.

In general, such higher-kinded types (HKTs) are a much-requested feature for Rust, and they would extend the reach of associated types. But the design and implementation of higher-kinded types is, by itself, a significant investment. The point of view of this RFC is that associated items bring the most important changes needed to stabilize our existing traits (and add a few key others), while HKTs will allow us to define important traits in the future but are not necessary for 1.0.

Encoding higher-kinded types

That said, it’s worth pointing out that variants of higher-kinded types can be encoded in the system being proposed here.

For example, the Iterable example above can be written in the following somewhat contorted style:

trait IterableOwned {
    type A;
    type I: Iterator<A>;
    fn iter_owned(self) -> I;
}

trait Iterable {
    fn iter<'a>(&'a self) -> <&'a Self>::I where &'a Self: IterableOwned {
        IterableOwned::iter_owned(self)
    }
}

The idea here is to define a trait that takes, as input type/lifetimes parameters, the parameters to any HKTs. In this case, the trait is implemented on the type &'a Self, which includes the lifetime parameter.

We can in fact generalize this technique to encode arbitrary HKTs:

// The kind * -> *
trait TypeToType<Input> {
    type Output;
}
type Apply<Name, Elt> where Name: TypeToType<Elt> = Name::Output;

struct Vec_;
struct DList_;

impl<T> TypeToType<T> for Vec_ {
    type Output = Vec<T>;
}

impl<T> TypeToType<T> for DList_ {
    type Output = DList<T>;
}

trait Mappable
{
    type E;
    type HKT where Apply<HKT, E> = Self;

    fn map<F>(self, f: E -> F) -> Apply<HKT, F>;
}

While the above demonstrates the versatility of associated types and where clauses, it is probably too much of a hack to be viable for use in libstd.

Associated consts in generic code

If the value of an associated const depends on a type parameter (including Self), it cannot be used in a constant expression. This restriction will almost certainly be lifted in the future, but this raises questions outside the scope of this RFC.

Staging

Associated lifetimes are probably not necessary for the 1.0 timeframe. While we currently have a few traits that are parameterized by lifetimes, most of these can go away once DST lands.

On the other hand, associated lifetimes are probably trivial to implement once associated types have been implemented.

Other interactions

Interaction with implied bounds

As part of the implied bounds idea, it may be desirable for this:

fn pick_node<G>(g: &G) -> &<G as Graph>::N

to be sugar for this:

fn pick_node<G: Graph>(g: &G) -> &<G as Graph>::N

But this feature can easily be added later, as part of a general implied bounds RFC.

Future-proofing: specialization of impls

In the future, we may wish to relax the “overlapping instances” rule so that one can provide “blanket” trait implementations and then “specialize” them for particular types. For example:

trait Sliceable {
    type Slice;
    // note: not using &self here to avoid need for HKT
    fn as_slice(self) -> Slice;
}

impl<'a, T> Sliceable for &'a T {
    type Slice = &'a T;
    fn as_slice(self) -> &'a T { self }
}

impl<'a, T> Sliceable for &'a Vec<T> {
    type Slice = &'a [T];
    fn as_slice(self) -> &'a [T] { self.as_slice() }
}

But then there’s a difficult question:

fn dice<A>(a: &A) -> &A::Slice where &A: Sliceable {
    a // is this allowed?
}

Here, the blanket and specialized implementations provide incompatible associated types. When working with the trait generically, what can we assume about the associated type? If we assume it is the blanket one, the type may change during monomorphization (when specialization takes effect)!

The RFC does allow generic code to “see” associated types provided by blanket implementations, so this is a potential problem.

Our suggested strategy is the following. If at some later point we wish to add specialization, traits would have to opt in explicitly. For such traits, we would not allow generic code to “see” associated types for blanket implementations; instead, output types would only be visible when all input types were concretely known. This approach is backwards-compatible with the RFC, and is probably a good idea in any case.

Alternatives

Multidispatch through tuple types

This RFC clarifies trait matching by making trait type parameters inputs to matching, and associated types outputs.

A more radical alternative would be to remove type parameters from traits, and instead support multiple input types through a separate multidispatch mechanism.

In this design, the Add trait would be written and implemented as follows:

// Lhs and Rhs are *inputs*
trait Add for (Lhs, Rhs) {
    type Sum; // Sum is an *output*
    fn add(&Lhs, &Rhs) -> Sum;
}

impl Add for (int, int) {
    type Sum = int;
    fn add(left: &int, right: &int) -> int { ... }
}

impl Add for (int, Complex) {
    type Sum = Complex;
    fn add(left: &int, right: &Complex) -> Complex { ... }
}

The for syntax in the trait definition is used for multidispatch traits, here saying that impls must be for pairs of types which are bound to Lhs and Rhs respectively. The add function can then be invoked in UFCS style by writing

Add::add(some_int, some_complex)

Advantages of the tuple approach:

• It does not force a distinction between Self and other input types, which in some cases (including binary operators like Add) can be artificial.

• Makes it possible to specify input types without specifying the trait: <(A, B)>::Sum rather than <A as Add<B>>::Sum.

Disadvantages of the tuple approach:

• It’s more painful when you do want a method rather than a function.

• Requires where clauses when used in bounds: where (A, B): Trait rather than A: Trait<B>.

• It gives two ways to write single dispatch: either without for, or using for with a single-element tuple.

• There’s a somewhat jarring distinction between single/multiple dispatch traits, making the latter feel “bolted on”.

• The tuple syntax is unusual in acting as a binder of its types, as opposed to the Trait<A, B> syntax.

• Relatedly, the generics syntax for traits is immediately understandable (a family of traits) based on other uses of generics in the language, while the tuple notation stands alone.

• Less clear story for trait objects (although the fact that Self is the only erased input type in this RFC may seem somewhat arbitrary).

On balance, the generics-based approach seems like a better fit for the language design, especially in its interaction with methods and the object system.

A backwards-compatible version

Yet another alternative would be to allow trait type parameters to be either inputs or outputs, marking the inputs with a keyword in:

trait Add<in Rhs, Sum> {
    fn add(&Lhs, &Rhs) -> Sum;
}

This would provide a way of adding multidispatch now, and then adding associated items later on without breakage. If, in addition, output types had to come after all input types, it might even be possible to migrate output type parameters like Sum above into associated types later.

This is perhaps a reasonable fallback, but it seems better to introduce a clean design with both multidispatch and associated items together.

Unresolved questions

Super traits

This RFC largely ignores super traits.

Currently, the implementation of super traits treats them identically to a where clause that bounds Self, and this RFC does not propose to change that. However, a follow-up RFC should clarify that this is the intended semantics for super traits.

Note that this treatment of super traits is, in particular, consistent with the proposed scoping rules, which do not bring items from super traits into scope in the body of a subtrait; they must be accessed via Self::item_name.

Equality constraints in where clauses

This RFC allows equality constraints on types for associated types, but does not propose a similar feature for where clauses. That will be the subject of a follow-up RFC.

Multiple trait object bounds for the same trait

The design here makes it possible to write bounds or trait objects that mention the same trait, multiple times, with different inputs:

fn mulit_add<T: Add<int> + Add<Complex>>(t: T) -> T { ... }
fn mulit_add_obj(t: Box<Add<int> + Add<Complex>>) -> Box<Add<int> + Add<Complex>> { ... }

This seems like a potentially useful feature, and should be unproblematic for bounds, but may have implications for vtables that make it problematic for trait objects. Whether or not such trait combinations are allowed will likely depend on implementation concerns, which are not yet clear.

Generic associated consts in match patterns

It seems desirable to allow constants that depend on type parameters in match patterns, but it’s not clear how to do so while still checking exhaustiveness and reachability of the match arms. Most likely this requires new forms of where clause, to constrain associated constant values.

For now, we simply defer the question.

Generic associated consts in array sizes

It would be useful to be able to use trait-associated constants in generic code.

// Shouldn't this be OK?
const ALIAS_N: usize = <T>::N;
let x: [u8; <T>::N] = [0u8; ALIAS_N];
// Or...
let x: [u8; T::N + 1] = [0u8; T::N + 1];

However, this causes some problems. What should we do with the following case in type checking, where we need to prove that a generic is valid for any T?

let x: [u8; T::N + T::N] = [0u8; 2 * T::N];

We would like to handle at least some obvious cases (e.g. proving that T::N == T::N), but without trying to prove arbitrary statements about arithmetic. The question of how to do this is deferred.

• Start Date: 2014-09-11
• RFC PR #: rust-lang/rfcs#198
• Rust Issue #: rust-lang/rust#17177

Summary

This RFC adds overloaded slice notation:

• foo[] for foo.as_slice()
• foo[n..m] for foo.slice(n, m)
• foo[n..] for foo.slice_from(n)
• foo[..m] for foo.slice_to(m)
• mut variants of all the above

via two new traits, Slice and SliceMut.

It also changes the notation for range match patterns to ..., to signify that they are inclusive whereas .. in slices are exclusive.

Motivation

There are two primary motivations for introducing this feature.

Ergonomics

Slicing operations, especially as_slice, are a very common and basic thing to do with vectors, and potentially many other kinds of containers. We already have notation for indexing via the Index trait, and this RFC is essentially a continuation of that effort.

The as_slice operator is particularly important. Since we’ve moved away from auto-slicing in coercions, explicit as_slice calls have become extremely common, and are one of the leading ergonomic/first impression problems with the language. There are a few other approaches to address this particular problem, but these alternatives have downsides that are discussed below (see “Alternatives”).

Error handling conventions

We are gradually moving toward a Python-like world where notation like foo[n] calls fail! when n is out of bounds, while corresponding methods like get return Option values rather than failing. By providing similar notation for slicing, we open the door to following the same convention throughout vector-like APIs.

Detailed design

The design is a straightforward continuation of the Index trait design. We introduce two new traits, for immutable and mutable slicing:

trait Slice<Idx, S> {
    fn as_slice<'a>(&'a self) -> &'a S;
    fn slice_from(&'a self, from: Idx) -> &'a S;
    fn slice_to(&'a self, to: Idx) -> &'a S;
    fn slice(&'a self, from: Idx, to: Idx) -> &'a S;
}

trait SliceMut<Idx, S> {
    fn as_mut_slice<'a>(&'a mut self) -> &'a mut S;
    fn slice_from_mut(&'a mut self, from: Idx) -> &'a mut S;
    fn slice_to_mut(&'a mut self, to: Idx) -> &'a mut S;
    fn slice_mut(&'a mut self, from: Idx, to: Idx) -> &'a mut S;
}

(Note, the mutable names here are part of likely changes to naming conventions that will be described in a separate RFC).

These traits will be used when interpreting the following notation:

Immutable slicing

• foo[] for foo.as_slice()
• foo[n..m] for foo.slice(n, m)
• foo[n..] for foo.slice_from(n)
• foo[..m] for foo.slice_to(m)

Mutable slicing

• foo[mut] for foo.as_mut_slice()
• foo[mut n..m] for foo.slice_mut(n, m)
• foo[mut n..] for foo.slice_from_mut(n)
• foo[mut ..m] for foo.slice_to_mut(m)

Like Index, uses of this notation will auto-deref just as if they were method invocations. So if T implements Slice<uint, [U]>, and s: Smaht<T>, then s[] compiles and has type &[U].

Note that slicing is “exclusive” (so [n..m] is the interval n <= x < m), while .. in match patterns is “inclusive”. To avoid confusion, we propose to change the match notation to ... to reflect the distinction. The reason to change the notation, rather than the interpretation, is that the exclusive (respectively inclusive) interpretation is the right default for slicing (respectively matching).

Rationale for the notation

The choice of square brackets for slicing is straightforward: it matches our indexing notation, and slicing and indexing are closely related.

Some other languages (like Python and Go – and Fortran) use : rather than .. in slice notation. The choice of .. here is influenced by its use elsewhere in Rust, for example for fixed-length array types [T, ..n]. The .. for slicing has precedent in Perl and D.

See Wikipedia for more on the history of slice notation in programming languages.

The mut qualifier

It may be surprising that mut is used as a qualifier in the proposed slice notation, but not for the indexing notation. The reason is that indexing includes an implicit dereference. If v: Vec<Foo> then v[n] has type Foo, not &Foo or &mut Foo. So if you want to get a mutable reference via indexing, you write &mut v[n]. More generally, this allows us to do resolution/typechecking prior to resolving the mutability.

This treatment of Index matches the C tradition, and allows us to write things like v[0] = foo instead of *v[0] = foo.

On the other hand, this approach is problematic for slicing, since in general it would yield an unsized type (under DST) – and of course, slicing is meant to give you a fat pointer indicating the size of the slice, which we don’t want to immediately deref. But the consequence is that we need to know the mutability of the slice up front, when we take it, since it determines the type of the expression.

Drawbacks

The main drawback is the increase in complexity of the language syntax. This seems minor, especially since the notation here is essentially “finishing” what was started with the Index trait.

Limitations in the design

Like the Index trait, this forces the result to be a reference via &, which may rule out some generalizations of slicing.

One way of solving this problem is for the slice methods to take self (by value) rather than &self, and in turn to implement the trait on &T rather than T. Whether this approach is viable in the long run will depend on the final rules for method resolution and auto-ref.

In general, the trait system works best when traits can be applied to types T rather than borrowed types &T. Ultimately, if Rust gains higher-kinded types (HKT), we could change the slice type S in the trait to be higher-kinded, so that it is a family of types indexed by lifetime. Then we could replace the &'a S in the return value with S<'a>. It should be possible to transition from the current Index and Slice trait designs to an HKT version in the future without breaking backwards compatibility by using blanket implementations of the new traits (say, IndexHKT) for types that implement the old ones.

Alternatives

For improving the ergonomics of as_slice, there are two main alternatives.

Coercions: auto-slicing

One possibility would be re-introducing some kind of coercion that automatically slices. We used to have a coercion from (in today’s terms) Vec<T> to &[T]. Since we no longer coerce owned to borrowed values, we’d probably want a coercion &Vec<T> to &[T] now:

fn use_slice(t: &[u8]) { ... }

let v = vec!(0u8, 1, 2);
use_slice(&v)           // automatically coerce here
use_slice(v.as_slice()) // equivalent

Unfortunately, adding such a coercion requires choosing between the following:

• Tie the coercion to Vec and String. This would reintroduce special treatment of these otherwise purely library types, and would mean that other library types that support slicing would not benefit (defeating some of the purpose of DST).

• Make the coercion extensible, via a trait. This is opening pandora’s box, however: the mechanism could likely be (ab)used to run arbitrary code during coercion, so that any invocation foo(a, b, c) might involve running code to pre-process each of the arguments. While we may eventually want such user-extensible coercions, it is a big step to take with a lot of potential downside when reasoning about code, so we should pursue more conservative solutions first.

Deref

Another possibility would be to make String implement Deref<str> and Vec<T> implement Deref<[T]>, once DST lands. Doing so would allow explicit coercions like:

fn use_slice(t: &[u8]) { ... }

let v = vec!(0u8, 1, 2);
use_slice(&*v)          // take advantage of deref
use_slice(v.as_slice()) // equivalent

There are at least two downsides to doing so, however:

• It is not clear how the method resolution rules will ultimately interact with Deref. In particular, a leading proposal is that for a smart pointer s: Smaht<T> when you invoke s.m(...) only inherent methods m are considered for Smaht<T>; trait methods are only considered for the maximally-derefed value *s.

  With such a resolution strategy, implementing Deref for Vec would make it impossible to use trait methods on the Vec type except through UFCS, severely limiting the ability of programmers to usefully implement new traits for Vec.

• The idea of Vec as a smart pointer around a slice, and the use of &*v as above, is somewhat counterintuitive, especially for such a basic type.

Ultimately, notation for slicing seems desirable on its own merits anyway, and if it can eliminate the need to implement Deref for Vec and String, all the better.

• Start Date: 2014-08-28
• RFC PR #: rust-lang/rfcs#199
• Rust Issue #: rust-lang/rust#16810

Summary

This is a conventions RFC for settling naming conventions when there are by value, by reference, and by mutable reference variants of an operation.

Motivation

Currently the libraries are not terribly consistent about how to signal mut variants of functions; sometimes it is by a mut_ prefix, sometimes a _mut suffix, and occasionally with _mut_ appearing in the middle. These inconsistencies make APIs difficult to remember.

While there are arguments in favor of each of the positions, we stand to gain a lot by standardizing, and to some degree we just need to make a choice.

Detailed design

Functions often come in multiple variants: immutably borrowed, mutably borrowed, and owned.

The canonical example is iterator methods:

• iter works with immutably borrowed data
• mut_iter works with mutably borrowed data
• move_iter works with owned data

For iterators, the “default” (unmarked) variant is immutably borrowed. In other cases, the default is owned.

The proposed rules depend on which variant is the default, but use suffixes to mark variants in all cases.

The rules

Immutably borrowed by default

If foo uses/produces an immutable borrow by default, use:

• The _mut suffix (e.g. foo_mut) for the mutably borrowed variant.
• The _move suffix (e.g. foo_move) for the owned variant.

However, in the case of iterators, the moving variant can also be understood as an into conversion, into_iter, and for x in v.into_iter() reads arguably better than for x in v.iter_move(), so the convention is into_iter.

NOTE: This convention covers only the method names for iterators, not the names of the iterator types. That will be the subject of a follow up RFC.

Owned by default

If foo uses/produces owned data by default, use:

• The _ref suffix (e.g. foo_ref) for the immutably borrowed variant.
• The _mut suffix (e.g. foo_mut) for the mutably borrowed variant.

Exceptions

For mutably borrowed variants, if the mut qualifier is part of a type name (e.g. as_mut_slice), it should appear as it would appear in the type.

References to type names

Some places in the current libraries, we say things like as_ref and as_mut, and others we say get_ref and get_mut_ref.

Proposal: generally standardize on mut as a shortening of mut_ref.

The rationale

Why suffixes?

Using a suffix makes it easier to visually group variants together, especially when sorted alphabetically. It puts the emphasis on the functionality, rather than the qualifier.

Why move?

Historically, Rust has used move as a way to signal ownership transfer and to connect to C++ terminology. The main disadvantage is that it does not emphasize ownership, which is our current narrative. On the other hand, in Rust all data is owned, so using _owned as a qualifier is a bit strange.

The Copy trait poses a problem for any terminology about ownership transfer. The proposed mental model is that with Copy data you are “moving a copy”.

See Alternatives for more discussion.

Why mut rather then mut_ref?

It’s shorter, and pairs like as_ref and as_mut have a pleasant harmony that doesn’t place emphasis on one kind of reference over the other.

Alternatives

Prefix or mixed qualifiers

Using prefixes for variants is another possibility, but there seems to be little upside.

It’s possible to rationalize our current mix of prefixes and suffixes via grammatical distinctions, but this seems overly subtle and complex, and requires a strong command of English grammar to work well.

No suffix exception

The rules here make an exception when mut is part of a type name, as in as_mut_slice, but we could instead always place the qualifier as a suffix: as_slice_mut. This would make APIs more consistent in some ways, less in others: conversion functions would no longer consistently use a transcription of their type name.

This is perhaps not so bad, though, because as it is we often abbreviate type names. In any case, we need a convention (separate RFC) for how to refer to type names in methods.

owned instead of move

The overall narrative about Rust has been evolving to focus on ownership as the essential concept, with borrowing giving various lesser forms of ownership, so _owned would be a reasonable alternative to _move.

On the other hand, the ref variants do not say “borrowed”, so in some sense this choice is inconsistent. In addition, the terminology is less familiar to those coming from C++.

val instead of owned

Another option would be val or value instead of owned. This suggestion plays into the “by reference” and “by value” distinction, and so is even more congruent with ref than move is. On the other hand, it’s less clear/evocative than either move or owned.

• Start Date: 2014-07-17
• RFC PR #: rust-lang/rfcs#201
• Rust Issue #: rust-lang/rust#17747

Summary

This RFC improves interoperation between APIs with different error types. It proposes to:

• Increase the flexibility of the try! macro for clients of multiple libraries with disparate error types.

• Standardize on basic functionality that any error type should have by introducing an Error trait.

• Support easy error chaining when crossing abstraction boundaries.

The proposed changes are all library changes; no language changes are needed – except that this proposal depends on multidispatch happening.

Motivation

Typically, a module (or crate) will define a custom error type encompassing the possible error outcomes for the operations it provides, along with a custom Result instance baking in this type. For example, we have io::IoError and io::IoResult<T> = Result<T, io::IoError>, and similarly for other libraries. Together with the try! macro, the story for interacting with errors for a single library is reasonably good.

However, we lack infrastructure when consuming or building on errors from multiple APIs, or abstracting over errors.

Consuming multiple error types

Our current infrastructure for error handling does not cope well with mixed notions of error.

Abstractly, as described by this issue, we cannot do the following:

fn func() -> Result<T, Error> {
    try!(may_return_error_type_A());
    try!(may_return_error_type_B());
}

Concretely, imagine a CLI application that interacts both with files and HTTP servers, using std::io and an imaginary http crate:

fn download() -> Result<(), CLIError> {
    let contents = try!(http::get(some_url));
    let file = try!(File::create(some_path));
    try!(file.write_str(contents));
    Ok(())
}

The download function can encounter both io and http errors, and wants to report them both under the common notion of CLIError. But the try! macro only works for a single error type at a time.

There are roughly two scenarios where multiple library error types need to be coalesced into a common type, each with different needs: application error reporting, and library error reporting

Application error reporting: presenting errors to a user

An application is generally the “last stop” for error handling: it’s the point at which remaining errors are presented to the user in some form, when they cannot be handled programmatically.

As such, the data needed for application-level errors is usually related to human interaction. For a CLI application, a short text description and longer verbose description are usually all that’s needed. For GUI applications, richer data is sometimes required, but usually not a full enum describing the full range of errors.

Concretely, then, for something like the download function above, for a CLI application, one might want CLIError to roughly be:

struct CLIError<'a> {
    description: &'a str,
    detail: Option<String>,
    ... // possibly more fields here; see detailed design
}

Ideally, one could use the try! macro as in the download example to coalesce a variety of error types into this single, simple struct.

Library error reporting: abstraction boundaries

When one library builds on others, it needs to translate from their error types to its own. For example, a web server framework may build on a library for accessing a SQL database, and needs some way to “lift” SQL errors to its own notion of error.

In general, a library may not want to reveal the upstream libraries it relies on – these are implementation details which may change over time. Thus, it is critical that the error type of upstream libraries not leak, and “lifting” an error from one library to another is a way of imposing an abstraction boundaries.

In some cases, the right way to lift a given error will depend on the operation and context. In other cases, though, there will be a general way to embed one kind of error in another (usually via a “cause chain”). Both scenarios should be supported by Rust’s error handling infrastructure.

Abstracting over errors

Finally, libraries sometimes need to work with errors in a generic way. For example, the serialize::Encoder type takes is generic over an arbitrary error type E. At the moment, such types are completely arbitrary: there is no Error trait giving common functionality expected of all errors. Consequently, error-generic code cannot meaningfully interact with errors.

(See this issue for a concrete case where a bound would be useful; note, however, that the design below does not cover this use-case, as explained in Alternatives.)

Languages that provide exceptions often have standard exception classes or interfaces that guarantee some basic functionality, including short and detailed descriptions and “causes”. We should begin developing similar functionality in libstd to ensure that we have an agreed-upon baseline error API.

Detailed design

We can address all of the problems laid out in the Motivation section by adding some simple library code to libstd, so this RFC will actually give a full implementation.

Note, however, that this implementation relies on the multidispatch proposal currently under consideration.

The proposal consists of two pieces: a standardized Error trait and extensions to the try! macro.

The Error trait

The standard Error trait follows very the widespread pattern found in Exception base classes in many languages:

pub trait Error: Send + Any {
    fn description(&self) -> &str;

    fn detail(&self) -> Option<&str> { None }
    fn cause(&self) -> Option<&Error> { None }
}

Every concrete error type should provide at least a description. By making this a slice-returning method, it is possible to define lightweight enum error types and then implement this method as returning static string slices depending on the variant.

The cause method allows for cause-chaining when an error crosses abstraction boundaries. The cause is recorded as a trait object implementing Error, which makes it possible to read off a kind of abstract backtrace (often more immediately helpful than a full backtrace).

The Any bound is needed to allow downcasting of errors. This RFC stipulates that it must be possible to downcast errors in the style of the Any trait, but leaves unspecified the exact implementation strategy. (If trait object upcasting was available, one could simply upcast to Any; otherwise, we will likely need to duplicate the downcast APIs as blanket impls on Error objects.)

It’s worth comparing the Error trait to the most widespread error type in libstd, IoError:

pub struct IoError {
    pub kind: IoErrorKind,
    pub desc: &'static str,
    pub detail: Option<String>,
}

Code that returns or asks for an IoError explicitly will be able to access the kind field and thus react differently to different kinds of errors. But code that works with a generic Error (e.g., application code) sees only the human-consumable parts of the error. In particular, application code will often employ Box<Error> as the error type when reporting errors to the user. The try! macro support, explained below, makes doing so ergonomic.

An extended try! macro

The other piece to the proposal is a way for try! to automatically convert between different types of errors.

The idea is to introduce a trait FromError<E> that says how to convert from some lower-level error type E to Self. The try! macro then passes the error it is given through this conversion before returning:

// E here is an "input" for dispatch, so conversions from multiple error
// types can be provided
pub trait FromError<E> {
    fn from_err(err: E) -> Self;
}

impl<E> FromError<E> for E {
    fn from_err(err: E) -> E {
        err
    }
}

impl<E: Error> FromError<E> for Box<Error> {
    fn from_err(err: E) -> Box<Error> {
        box err as Box<Error>
    }
}

macro_rules! try (
    ($expr:expr) => ({
        use error;
        match $expr {
            Ok(val) => val,
            Err(err) => return Err(error::FromError::from_err(err))
        }
    })
)

This code depends on multidispatch, because the conversion depends on both the source and target error types. (In today’s Rust, the two implementations of FromError given above would be considered overlapping.)

Given the blanket impl of FromError<E> for E, all existing uses of try! would continue to work as-is.

With this infrastructure in place, application code can generally use Box<Error> as its error type, and try! will take care of the rest:

fn download() -> Result<(), Box<Error>> {
    let contents = try!(http::get(some_url));
    let file = try!(File::create(some_path));
    try!(file.write_str(contents));
    Ok(())
}

Library code that defines its own error type can define custom FromError implementations for lifting lower-level errors (where the lifting should also perform cause chaining) – at least when the lifting is uniform across the library. The effect is that the mapping from one error type into another only has to be written one, rather than at every use of try!:

impl FromError<ErrorA> MyError { ... }
impl FromError<ErrorB> MyError { ... }

fn my_lib_func() -> Result<T, MyError> {
    try!(may_return_error_type_A());
    try!(may_return_error_type_B());
}

Drawbacks

The main drawback is that the try! macro is a bit more complicated.

Unresolved questions

Conventions

This RFC does not define any particular conventions around cause chaining or concrete error types. It will likely take some time and experience using the proposed infrastructure before we can settle these conventions.

Extensions

The functionality in the Error trait is quite minimal, and should probably grow over time. Some additional functionality might include:

Features on the Error trait

• Generic creation of Errors. It might be useful for the Error trait to expose an associated constructor. See this issue for an example where this functionality would be useful.

• Mutation of Errors. The Error trait could be expanded to provide setters as well as getters.

The main reason not to include the above two features is so that Error can be used with extremely minimal data structures, e.g. simple enums. For such data structures, it’s possible to produce fixed descriptions, but not mutate descriptions or other error properties. Allowing generic creation of any Error-bounded type would also require these enums to include something like a GenericError variant, which is unfortunate. So for now, the design sticks to the least common denominator.

Concrete error types

On the other hand, for code that doesn’t care about the footprint of its error types, it may be useful to provide something like the following generic error type:

pub struct WrappedError<E> {
    pub kind: E,
    pub description: String,
    pub detail: Option<String>,
    pub cause: Option<Box<Error>>
}

impl<E: Show> WrappedError<E> {
    pub fn new(err: E) {
        WrappedErr {
            kind: err,
            description: err.to_string(),
            detail: None,
            cause: None
        }
    }
}

impl<E> Error for WrappedError<E> {
    fn description(&self) -> &str {
        self.description.as_slice()
    }
    fn detail(&self) -> Option<&str> {
        self.detail.as_ref().map(|s| s.as_slice())
    }
    fn cause(&self) -> Option<&Error> {
        self.cause.as_ref().map(|c| &**c)
    }
}

This type can easily be added later, so again this RFC sticks to the minimal functionality for now.

• Start Date: 2014-08-15
• RFC PR: rust-lang/rfcs#202
• Rust Issue: rust-lang/rust#16967

Summary

Change syntax of subslices matching from ..xs to xs.. to be more consistent with the rest of the language and allow future backwards compatible improvements.

Small example:

match slice {
    [xs.., _] => xs,
    [] => fail!()
}

This is basically heavily stripped version of RFC 101.

Motivation

In Rust, symbol after .. token usually describes number of things, as in [T, ..N] type or in [e, ..N] expression. But in following pattern: [_, ..xs], xs doesn’t describe any number, but the whole subslice.

I propose to move dots to the right for several reasons (including one mentioned above):

1. Looks more natural (but that might be subjective).
2. Consistent with the rest of the language.
3. C++ uses args... in variadic templates.
4. It allows extending slice pattern matching as described in RFC 101.

Detailed design

Slice matching grammar would change to (assuming trailing commas; grammar syntax as in Rust manual):

slice_pattern : "[" [[pattern | subslice_pattern] ","]* "]" ;
subslice_pattern : ["mut"? ident]? ".." ["@" slice_pattern]? ;

To compare, currently it looks like:

slice_pattern : "[" [[pattern | subslice_pattern] ","]* "]" ;
subslice_pattern : ".." ["mut"? ident ["@" slice_pattern]?]? ;

Drawbacks

Backward incompatible.

Alternatives

Don’t do it at all.

Unresolved questions

Whether subslice matching combined with @ should be written as xs.. @[1, 2] or maybe in another way: xs @[1, 2]...

• Start Date: 2014-09-03
• RFC PR: rust-lang/rfcs#212
• Rust Issue: rust-lang/rust#16968

Summary

Restore the integer inference fallback that was removed. Integer literals whose type is unconstrained will default to i32, unlike the previous fallback to int. Floating point literals will default to f64.

Motivation

History lesson

Rust has had a long history with integer and floating-point literals. Initial versions of Rust required all literals to be explicitly annotated with a suffix (if no suffix is provided, then int or float was used; note that the float type has since been removed). This meant that, for example, if one wanted to count up all the numbers in a list, one would write 0u and 1u so as to employ unsigned integers:

let mut count = 0u; // let `count` be an unsigned integer
while cond() {
    ...
    count += 1u;    // `1u` must be used as well
}

This was particularly troublesome with arrays of integer literals, which could be quite hard to read:

let byte_array = [0u8, 33u8, 50u8, ...];

It also meant that code which was very consciously using 32-bit or 64-bit numbers was hard to read.

Therefore, we introduced integer inference: unlabeled integer literals are not given any particular integral type rather a fresh “integral type variable” (floating point literals work in an analogous way). The idea is that the vast majority of literals will eventually interact with an actual typed variable at some point, and hence we can infer what type they ought to have. For those cases where the type cannot be automatically selected, we decided to fallback to our older behavior, and have integer/float literals be typed as int/float (this is also what Haskell does). Some time later, we did various measurements and found that in real world code this fallback was rarely used. Therefore, we decided that to remove the fallback.

Experience with lack of fallback

Unfortunately, when doing the measurements that led us to decide to remove the int fallback, we neglected to consider coding “in the small” (specifically, we did not include tests in the measurements). It turns out that when writing small programs, which includes not only “hello world” sort of things but also tests, the lack of integer inference fallback is quite annoying. This is particularly troublesome since small program are often people’s first exposure to Rust. The problems most commonly occur when integers are “consumed” by printing them out to the screen or by asserting equality, both of which are very common in small programs and testing.

There are at least three common scenarios where fallback would be beneficial:

Accumulator loops. Here a counter is initialized to 0 and then incremented by 1. Eventually it is printed or compared against a known value.

let mut c = 0;
loop {
    ...;
    c += 1;
}
println!("{}", c); // Does not constrain type of `c`
assert_eq(c, 22);

Calls to range with constant arguments. Here a call to range like range(0, 10) is used to execute something 10 times. It is important that the actual counter is either unused or only used in a print out or comparison against another literal:

for _ in range(0, 10) {
}

Large constants. In small tests it is convenient to make dummy test data. This frequently takes the form of a vector or map of ints.

let mut m = HashMap::new();
m.insert(1, 2);
m.insert(3, 4);
assert_eq(m.find(&3).map(|&i| i).unwrap(), 4);

Lack of bugs

To our knowledge, there has not been a single bug exposed by removing the fallback to the int type. Moreover, such bugs seem to be extremely unlikely.

The primary reason for this is that, in production code, the i32 fallback is very rarely used. In a sense, the same measurements that were used to justify removing the int fallback also justify keeping it. As the measurements showed, the vast, vast majority of integer literals wind up with a constrained type, unless they are only used to print out and do assertions with. Specifically, any integer that is passed as a parameter, returned from a function, or stored in a struct or array, must wind up with a specific type.

Rationale for the choice of defaulting to i32

In contrast to the first revision of the RFC, the fallback type suggested is i32. This is justified by a case analysis which showed that there does not exist a compelling reason for having a signed pointer-sized integer type as the default.

There are reasons for using i32 instead: It’s familiar to programmers from the C programming language (where the default int type is 32-bit in the major calling conventions), it’s faster than 64-bit integers in arithmetic today, and is superior in memory usage while still providing a reasonable range of possible values.

To expand on the performance argument: i32 obviously uses half of the memory of i64 meaning half the memory bandwidth used, half as much cache consumption and twice as much vectorization – additionally arithmetic (like multiplication and division) is faster on some of the modern CPUs.

Case analysis

This is an analysis of cases where int inference might be thought of as useful:

Indexing into an array with unconstrained integer literal:

let array = [0u8, 1, 2, 3];
let index = 3;
array[index]

In this case, index is already automatically inferred to be a uint.

Using a default integer for tests, tutorials, etc.: Examples of this include “The Guide”, the Rust API docs and the Rust standard library unit tests. This is better served by a smaller, faster and platform independent type as default.

Using an integer for an upper bound or for simply printing it: This is also served very well by i32.

Counting of loop iterations: This is a part where int is as badly suited as i32, so at least the move to i32 doesn’t create new hazards (note that the number of elements of a vector might not necessarily fit into an int).

In addition to all the points above, having a platform-independent type obviously results in less differences between the platforms in which the programmer “doesn’t care” about the integer type they are using.

Future-proofing for overloaded literals

It is possible that, in the future, we will wish to allow vector and strings literals to be overloaded so that they can be resolved to user-defined types. In that case, for backwards compatibility, it will be necessary for those literals to have some sort of fallback type. (This is a relatively weak consideration.)

Detailed design

Integral literals are currently type-checked by creating a special class of type variable. These variables are subject to unification as normal, but can only unify with integral types. This RFC proposes that, at the end of type inference, when all constraints are known, we will identify all integral type variables that have not yet been bound to anything and bind them to i32. Similarly, floating point literals will fallback to f64.

For those who wish to be very careful about which integral types they employ, a new lint (unconstrained_literal) will be added which defaults to allow. This lint is triggered whenever the type of an integer or floating point literal is unconstrained.

Downsides

Although there seems to be little motivation for int to be the default, there might be use cases where int is a more correct fallback than i32.

Additionally, it might seem weird to some that i32 is a default, when int looks like the default from other languages. The name of int however is not in the scope of this RFC.

Alternatives

• No fallback. Status quo.

• Fallback to something else. We could potentially fallback to int like the original RFC suggested or some other integral type rather than i32.

• Fallback in a more narrow range of cases. We could attempt to identify integers that are “only printed” or “only compared”. There is no concrete proposal in this direction and it seems to lead to an overly complicated design.

• Default type parameters influencing inference. There is a separate, follow-up proposal being prepared that uses default type parameters to influence inference. This would allow some examples, like range(0, 10) to work even without integral fallback, because the range function itself could specify a fallback type. However, this does not help with many other examples.

History

2014-11-07: Changed the suggested fallback from int to i32, add rationale.

• Start Date: 2015-02-04
• RFC PR: rust-lang/rfcs#213
• Rust Issue: rust-lang/rust#27336

Summary

Rust currently includes feature-gated support for type parameters that specify a default value. This feature is not well-specified. The aim of this RFC is to fully specify the behavior of defaulted type parameters:

1. Type parameters in any position can specify a default.
2. Within fn bodies, defaulted type parameters are used to drive inference.
3. Outside of fn bodies, defaulted type parameters supply fixed defaults.
4. _ can be used to omit the values of type parameters and apply a suitable default:
   • In a fn body, any type parameter can be omitted in this way, and a suitable type variable will be used.
   • Outside of a fn body, only defaulted type parameters can be omitted, and the specified default is then used.

Points 2 and 4 extend the current behavior of type parameter defaults, aiming to address some shortcomings of the current implementation.

This RFC would remove the feature gate on defaulted type parameters.

Motivation

Why defaulted type parameters

Defaulted type parameters are very useful in two main scenarios:

1. Extended a type without breaking existing clients.
2. Allowing customization in ways that many or most users do not care about.

Often, these two scenarios occur at the same time. A classic historical example is the HashMap type from Rust’s standard library. This type now supports the ability to specify custom hashers. For most clients, this is not particularly important and this initial versions of the HashMap type were not customizable in this regard. But there are some cases where having the ability to use a custom hasher can make a huge difference. Having the ability to specify defaults for type parameters allowed the HashMap type to add a new type parameter H representing the hasher type without breaking any existing clients and also without forcing all clients to specify what hasher to use.

However, customization occurs in places other than types. Consider the function range(). In early versions of Rust, there was a distinct range function for each integral type (e.g. uint::range, int::range, etc). These functions were eventually consolidated into a single range() function that is defined generically over all “enumerable” types:

trait Enumerable : Add<Self,Self> + PartialOrd + Clone + One;
pub fn range<A:Enumerable>(start: A, stop: A) -> Range<A> {
    Range{state: start, stop: stop, one: One::one()}
}

This version is often more convenient to use, particularly in a generic context.

However, the generic version does have the downside that when the bounds of the range are integral, inference sometimes lacks enough information to select a proper type:

// ERROR -- Type argument unconstrained, what integral type did you want?
for x in range(0, 10) { ... }

Thus users are forced to write:

for x in range(0u, 10u) { ... }

This RFC describes how to integrate default type parameters with inference such that the type parameter on range can specify a default (uint, for example):

pub fn range<A:Enumerable=uint>(start: A, stop: A) -> Range<A> {
    Range{state: start, stop: stop, one: One::one()}
}

Using this definition, a call like range(0, 10) is perfectly legal. If it turns out that the type argument is not other constraint, uint will be used instead.

Extending types without breaking clients.

Without defaults, once a library is released to “the wild”, it is not possible to add type parameters to a type without breaking all existing clients. However, it frequently happens that one wants to take an existing type and make it more flexible that it used to be. This often entails adding a new type parameter so that some type which was hard-coded before can now be customized. Defaults provide a means to do this while having older clients transparently fallback to the older behavior.

Historical example: Extending HashMap to support various hash algorithms.

Detailed Design

Remove feature gate

This RFC would remove the feature gate on defaulted type parameters.

Type parameters with defaults

Defaults can be placed on any type parameter, whether it is declared on a type definition (struct, enum), type alias (type), trait definition (trait), trait implementation (impl), or a function or method (fn).

Once a given type parameter declares a default value, all subsequent type parameters in the list must declare default values as well:

// OK. All defaulted type parameters come at the end.
fn foo<A,B=uint,C=uint>() { .. }

// ERROR. B has a default, but C does not.
fn foo<A,B=uint,C>() { .. }

The default value of a type parameter X may refer to other type parameters declared on the same item. However, it may only refer to type parameters declared before X in the list of type parameters:

// OK. Default value of `B` refers to `A`, which is not defaulted.
fn foo<A,B=A>() { .. }

// OK. Default value of `C` refers to `B`, which comes before
// `C` in the list of parameters.
fn foo<A,B=uint,C=B>() { .. }

// ERROR. Default value of `B` refers to `C`, which comes AFTER
// `B` in the list of parameters.
fn foo<A,B=C,C=uint>() { .. }

Instantiating defaults

This section specifies how to interpret a reference to a generic type. Rather than writing out a rather tedious (and hard to understand) description of the algorithm, the rules are instead specified by a series of examples. The high-level idea of the rules is as follows:

• Users must always provide some value for non-defaulted type parameters. Defaulted type parameters may be omitted.
• The _ notation can always be used to explicitly omit the value of a type parameter:
  • Inside a fn body, any type parameter may be omitted. Inference is used.
  • Outside a fn body, only defaulted type parameters may be omitted. The default value is used.
  • Motivation: This is consistent with Rust tradition, which generally requires explicit types or a mechanical defaulting process outside of fn bodies.

References to generic types

We begin with examples of references to the generic type Foo:

struct Foo<A,B,C=DefaultHasher,D=C> { ... }

Foo defines four type parameters, the final two of which are defaulted. First, let us consider what happens outside of a fn body. It is mandatory to supply explicit values for all non-defaulted type parameters:

// ERROR: 2 parameters required, 0 provided.
fn f(_: &Foo) { ... }

Defaulted type parameters are filled in based on the defaults given:

// Legal: Equivalent to `Foo<int,uint,DefaultHasher,DefaultHasher>`
fn f(_: &Foo<int,uint>) { ... }

Naturally it is legal to specify explicit values for the defaulted type parameters if desired:

// Legal: Equivalent to `Foo<int,uint,uint,char,u8>`
fn f(_: &Foo<int,uint,char,u8>) { ... }

It is also legal to provide just one of the defaulted type parameters and not the other:

// Legal: Equivalent to `Foo<int,uint,char,char>`
fn f(_: &Foo<int,uint,char>) { ... }

If the user wishes to supply the value of the type parameter D explicitly, but not C, then _ can be used to request the default:

// Legal: Equivalent to `Foo<int,uint,DefaultHasher,uint>`
fn f(_: &Foo<int,uint,_,uint>) { ... }

Note that, outside of a fn body, _ can only be used with defaulted type parameters:

// ERROR: outside of a fn body, `_` cannot be
// used for a non-defaulted type parameter
fn f(_: &Foo<int,_>) { ... }

Inside a fn body, the rules are much the same, except that _ is legal everywhere. Every reference to _ creates a fresh type variable $n. If the type parameter whose value is omitted has an associate default, that default is used as the fallback for $n (see the section “Type variables with fallbacks” for more information). Here are some examples:

fn f() {
    // Error: `Foo` requires at least 2 type parameters, 0 supplied.
    let x: Foo = ...;

    // All of these 4 examples are OK and equivalent. Each
    // results in a type `Foo<$0,$1,$2,$3>` and `$0`-`$4` are type
    // variables. `$2` has a fallback of `DefaultHasher` and `$3`
    // has a fallback of `$2`.
    let x: Foo<_,_> = ...;
    let x: Foo<_,_,_> = ...;
    let x: Foo<_,_,_,_> = ...;

    // Results in a type `Foo<int,uint,$0,char>` where `$0`
    // has a fallback of `DefaultHasher`.
    let x: Foo<int,uint,_,char> = ...;
}

References to generic traits

The rules for traits are the same as the rules for types. Consider a trait Foo:

trait Foo<A,B,C=uint,D=C> { ... }

References to this trait can omit values for C and D in precisely the same way as was shown for types:

// All equivalent to Foo<i8,u8,uint,uint>:
fn foo<T:Foo<i8,u8>>() { ... }
fn foo<T:Foo<i8,u8,_>>() { ... }
fn foo<T:Foo<i8,u8,_,_>>() { ... }

// Equivalent to Foo<i8,u8,char,char>:
fn foo<T:Foo<i8,u8,char,_>>() { ... }

References to generic functions

The rules for referencing generic functions are the same as for types, except that it is legal to omit values for all type parameters if desired. In that case, the behavior is the same as it would be if _ were used as the value for every type parameter. Note that functions can only be referenced from within a fn body.

References to generic impls

Users never explicitly “reference” an impl. Rather, the trait matching system implicitly instantiates impls as part of trait matching. This implies that all type parameters are always instantiated with type variables. These type variables are assigned fallbacks according to the defaults given.

Type variables with fallbacks

We extend the inference system so that when a type variable is created, it can optionally have a fallback value, which is another type.

In the type checker, whenever we create a fresh type variable to represent a type parameter with an associated default, we will use that default as the fallback value for this type variable.

Example:

fn foo<A,B=A>(a: A, b: B) { ... }

fn bar() {
    // Here, the values of the type parameters are given explicitly.
    let f: fn(uint, uint) = foo::<uint, uint>;

    // Here the value of the first type parameter is given explicitly,
    // but not the second. Because the second specifies a default, this
    // is permitted. The type checker will create a fresh variable `$0`
    // and attempt to infer the value of this defaulted type parameter.
    let g: fn(uint, $0) = foo::<uint>;

    // Here, the values of the type parameters are not given explicitly,
    // and hence the type checker will create fresh variables
    // `$1` and `$2` for both of them.
    let h: fn($1, $2) = foo;
}

In this snippet, there are three references to the generic function foo, each of which specifies progressively fewer types. As a result, the type checker winds up creating three type variables, which are referred to in the example as $0, $1, and $2 (not that this $ notation is just for explanatory purposes and is not actual Rust syntax).

The fallback values of $0, $1, and $2 are as follows:

• $0 was created to represent the type parameter B defined on foo. This means that $0 will have a fallback value of uint, since the type variable A was specified to be uint in the expression that created $0.
• $1 was created to represent the type parameter A, which has no default. Therefore $1 has no fallback.
• $2 was created to represent the type parameter B. It will have the fallback value of $1, which was the value of A within the expression where $2 was created.

Trait resolution, fallbacking, and inference

Prior to this RFC, type-checking a function body proceeds roughly as follows:

1. The function body is analyzed. This results in an accumulated set of type variables, constraints, and trait obligations.
2. Those trait obligations are then resolved until a fixed point is reached.
3. If any trait obligations remain unresolved, an error is reported.
4. If any type variables were never bound to a concrete value, an error is reported.

To accommodate fallback, the new procedure is somewhat different:

1. The function body is analyzed. This results in an accumulated set of type variables, constraints, and trait obligations.
2. Execute in a loop:
3. Run trait resolution until a fixed point is reached.
4. Create a (initially empty) set UB of unbound type and integral/float variables. This set represents the set of variables for which fallbacks should be applied.
5. Add all unbound integral and float variables to the set UB
6. For each type variable X:
   • If X has no fallback defined, skip.
   • If X is not bound, add X to UB
   • If X is bound to an unbound integral variable I, add X to UB and remove I from UB (if present).
   • If X is bound to an unbound float variable F, add X to UB and remove F from UB (if present).
7. If UB is the empty set, break out of the loop.
8. For each member of UB:
   • If the member is an integral type variable I, set I to int.
   • If the member is a float variable F, set I to f64.
   • Otherwise, the member must be a variable X with a defined fallback. Set X to its fallback.
     • Note that this “set” operations can fail, which indicates conflicting defaults. A suitable error message should be given.
9. If any type parameters still have no value assigned to them, report an error.
10. If any trait obligations could not be resolved, report an error.

There are some subtle points to this algorithm:

When defaults are to be applied, we first gather up the set of variables that have applicable defaults (step 2.2) and then later unconditionally apply those defaults (step 2.4). In particular, we do not loop over each type variable, check whether it is unbound, and apply the default only if it is unbound. The reason for this is that it can happen that there are contradictory defaults and we want to ensure that this results in an error:

fn foo<F:Default=uint>() -> F { }
fn bar<B=int>(b: B) { }
fn baz() {
    // Here, F is instantiated with $0=uint
    let x: $0 = foo();

    // Here, B is instantiated with $1=uint, and constraint $0 <: $1 is added.
    bar(x);
}

In this example, two type variables are created. $0 is the value of F in the call to foo() and $1 is the value of B in the call to bar(). The fact that x, which has type $0, is passed as an argument to bar() will add the constraint that $0 <: $1, but at no point are any concrete types given. Therefore, once type checking is complete, we will apply defaults. Using the algorithm given above, we will determine that both $0 and $1 are unbound and have suitable defaults. We will then unify $0 with uint. This will succeed and, because $0 <: $1, cause $1 to be unified with uint. Next, we will try to unify $1 with its default, int. This will lead to an error. If we combined the checking of whether $1 was unbound with the unification with the default, we would have first unified $0 and then decided that $1 did not require unification.

In the general case, a loop is required to continue resolving traits and applying defaults in sequence. Resolving traits can lead to unifications, so it is clear that we must resolve all traits that we can before we apply any defaults. However, it is also true that adding defaults can create new trait obligations that must be resolved.

Here is an example where processing trait obligations creates defaults, and processing defaults created trait obligations:

trait Foo { }
trait Bar { }

impl<T:Bar=uint> Foo for Vec<T> { } // Impl 1
impl Bar for uint { } // Impl 2

fn takes_foo<F:Foo>(f: F) { }

fn main() {
    let x = Vec::new(); // x: Vec<$0>
    takes_foo(x); // adds oblig Vec<$0> : Foo
}

When we finish type checking main, we are left with a variable $0 and a trait obligation Vec<$0> : Foo. Processing the trait obligation selects the impl 1 as the way to fulfill this trait obligation. This results in:

1. a new type variable $1, which represents the parameter T on the impl. $1 has a default, uint.
2. the constraint that $0=$1.
3. a new trait obligation $1 : Bar.

We cannot process the new trait obligation yet because the type variable $1 is still unbound. (We know that it is equated with $0, but we do not have any concrete types yet, just variables.) After trait resolution reaches a fixed point, defaults are applied. $1 is equated with uint which in turn propagates to $0. At this point, there is still an outstanding trait obligation uint : Bar. This trait obligation can be resolved to impl 2.

The previous example consisted of “1.5” iterations of the loop. That is, although trait resolution runs twice, defaults are only needed one time:

1. Trait resolution executed to resolve Vec<$0> : Foo.
2. Defaults were applied to unify $1 = $0 = uint.
3. Trait resolution executed to resolve uint : Bar
4. No more defaults to apply, done.

The next example does 2 full iterations of the loop.

trait Foo { }
trait Bar<U> { }
trait Baz { }

impl<U,T:Bar<U>=Vec<U>> Foo for Vec<T> { } // Impl 1
impl<V=uint> Bar for Vec<V> { } // Impl 2

fn takes_foo<F:Foo>(f: F) { }

fn main() {
    let x = Vec::new(); // x: Vec<$0>
    takes_foo(x); // adds oblig Vec<$0> : Foo
}

Here the process is as follows:

1. Trait resolution executed to resolve Vec<$0> : Foo. The result is two fresh variables, $1 (for U) and $2=Vec<$1> (for $T), the constraint that $0=$2, and the obligation $2 : Bar<$1>.
2. Defaults are applied to unify $2 = $0 = Vec<$1>.
3. Trait resolution executed to resolve $2 : Bar<$1>. The result is a fresh variable $3=uint (for $V) and the constraint that $1=$3.
4. Defaults are applied to unify $3 = $1 = uint.

It should be clear that one can create examples in this vein so as to require any number of loops.

Interaction with integer/float literal fallback. This RFC gives defaulted type parameters precedence over integer/float literal fallback. This seems preferable because such types can be more specific. Below are some examples. See also the alternatives section.

// Here the type of the integer literal 22 is inferred
// to `int` using literal fallback.
fn foo<T>(t: T) { ... }
foo(22)

// Here the type of the integer literal 22 is inferred
// to `uint` because the default on `T` overrides the
// standard integer literal fallback.
fn foo<T=uint>(t: T) { ... }
foo(22)

// Here the type of the integer literal 22 is inferred
// to `char`, leading to an error. This can be resolved
// by using an explicit suffix like `22i`.
fn foo<T=char>(t: T) { ... }
foo(22)

Termination. Any time that there is a loop, one must inquire after termination. In principle, the loop above could execute indefinitely. This is because trait resolution is not guaranteed to terminate – basically there might be a cycle between impls such that we continue creating new type variables and new obligations forever. The trait matching system already defends against this with a recursion counter. That same recursion counter is sufficient to guarantee termination even when the default mechanism is added to the mix. This is because the default mechanism can never itself create new trait obligations: it can only cause previous ambiguous trait obligations to now be matchable (because unbound variables become bound). But the actual need to iteration through the loop is still caused by trait matching generating recursive obligations, which have an associated depth limit.

Compatibility analysis

One of the major design goals of defaulted type parameters is to permit new parameters to be added to existing types or methods in a backwards compatible way. This remains possible under the current design.

Note though that adding a default to an existing type parameter can lead to type errors in clients. This can occur if clients were already relying on an inference fallback from some other source and there is now an ambiguity. Naturally clients can always fix this error by specifying the value of the type parameter in question manually.

Downsides and alternatives

Avoid inference

Rather than adding the notion of fallbacks to type variables, defaults could be mechanically added, even within fn bodies, as they are today. But this is disappointing because it means that examples like range(0,10), where defaults could inform inference, still require explicit annotation. Without the notion of fallbacks, it is also difficult to say what defaulted type parameters in methods or impls should mean.

More advanced interaction between integer literal inference

There were some other proposals to have a more advanced interaction between custom fallbacks and literal inference. For example, it is possible to imagine that we allow literal inference to take precedence over type default fallbacks, unless the fallback is itself integral. The problem is that this is both complicated and possibly not forwards compatible if we opt to allow a more general notion of literal inference in the future (in other words, if integer literals may be mapped to more than just the built-in integral types). Furthermore, these rules would create strictly fewer errors, and hence can be added in the future if desired.

Notation

Allowing _ notation outside of fn body means that it’s meaning changes somewhat depending on context. However, this is consistent with the meaning of omitted lifetimes, which also change in the same way (mechanical default outside of fn body, inference within).

An alternative design is to use the K=V notation proposed in the associated items RFC for specify the values of default type parameters. However, this is somewhat odd, because default type parameters appear in a positional list, and thus it is surprising that values for the non-defaulted parameters are given positionally, but values for the defaulted type parameters are given with labels.

Another alternative would to simply prohibit users from specifying the value of a defaulted type parameter unless values are given for all previous defaulted typed parameters. But this is clearly annoying in those cases where defaulted type parameters represent distinct axes of customization.

Hat Tip

eddyb introduced defaulted type parameters and also opened the first pull request that used them to inform inference.

• Start Date: 2014-08-27
• RFC PR: rust-lang/rfcs#214
• Rust Issue: rust-lang/rust#17687

Summary

Introduce a new while let PAT = EXPR { BODY } construct. This allows for using a refutable pattern match (with optional variable binding) as the condition of a loop.

Motivation

Just as if let was inspired by Swift, it turns out Swift supports while let as well. This was not discovered until much too late to include it in the if let RFC. It turns out that this sort of looping is actually useful on occasion. For example, the desugaring for loop is actually a variant on this; if while let existed it could have been implemented to map for PAT in EXPR { BODY } to

// the match here is so `for` can accept an rvalue for the iterator,
// and was used in the "real" desugaring version.
match &mut EXPR {
    i => {
        while let Some(PAT) = i.next() {
            BODY
        }
    }
}

(note that the non-desugared form of for is no longer equivalent).

More generally, this construct can be used any time looping + pattern-matching is desired.

This also makes the language a bit more consistent; right now, any condition that can be used with if can be used with while. The new if let adds a form of if that doesn’t map to while. Supporting while let restores the equivalence of these two control-flow constructs.

Detailed design

while let operates similarly to if let, in that it desugars to existing syntax. Specifically, the syntax

['ident:] while let PAT = EXPR {
    BODY
}

desugars to

['ident:] loop {
    match EXPR {
        PAT => BODY,
        _ => break
    }
}

Just as with if let, an irrefutable pattern given to while let is considered an error. This is largely an artifact of the fact that the desugared match ends up with an unreachable pattern, and is not actually a goal of this syntax. The error may be suppressed in the future, which would be a backwards-compatible change.

Just as with if let, while let will be introduced under a feature gate (named while_let).

Drawbacks

Yet another addition to the grammar. Unlike if let, it’s not obvious how useful this syntax will be.

Alternatives

As with if let, this could plausibly be done with a macro, but it would be ugly and produce bad error messages.

while let could be extended to support alternative patterns, just as match arms do. This is not part of the main proposal for the same reason it was left out of if let, which is that a) it looks weird, and b) it’s a bit of an odd coupling with the let keyword as alternatives like this aren’t going to be introducing variable bindings. However, it would make while let more general and able to replace more instances of loop { match { ... } } than is possible with the main design.

Unresolved questions

None.

• Start Date: 2014-08-28
• RFC PR: (https://github.com/rust-lang/rfcs/pull/216)
• Rust Issue: (https://github.com/rust-lang/rust/issues/17320)

Summary

Add additional iterator-like Entry objects to collections. Entries provide a composable mechanism for in-place observation and mutation of a single element in the collection, without having to “re-find” the element multiple times. This deprecates several “internal mutation” methods like hashmap’s find_or_insert_with.

Motivation

As we approach 1.0, we’d like to normalize the standard APIs to be consistent, composable, and simple. However, this currently stands in opposition to manipulating the collections in an efficient manner. For instance, if one wishes to build an accumulating map on top of one of the concrete maps, they need to distinguish between the case when the element they’re inserting is already in the map, and when it’s not. One way to do this is the following: