Struct Input

pub struct Input { /* private fields */ }

Input for the parser – a sequence of tokens.

As of now, parser doesn’t have access to the text of the tokens, and makes decisions based solely on their classification. Unlike LexerToken, the Tokens doesn’t include whitespace and comments. Main input to the parser.

Struct of arrays internally, but this shouldn’t really matter.

Implementations

impl Input

pub impl used by callers to create Tokens.

pub fn push(&mut self, kind: SyntaxKind)

pub fn push_ident(&mut self, contextual_kind: SyntaxKind)

pub fn was_joint(&mut self)

Sets jointness for the last token we’ve pushed.

This is a separate API rather than an argument to the push to make it convenient both for textual and mbe tokens. With text, you know whether the previous token was joint, with mbe, you know whether the current one is joint. This API allows for styles of usage:

// In text:
tokens.was_joint(prev_joint);
tokens.push(curr);

// In MBE:
token.push(curr);
tokens.push(curr_joint)

Trait Implementations

impl Default for Input

fn default() -> Input

Returns the “default value” for a type.