Syntax Reference

remlineinfo!(x)

Removes line info expressions. (i.e. Expr(:line, 1))

to_codeobject(x::EXPR)

Convert an EXPR into the object that Meta.parse would have produced from the original string, which could e.g. be an Expr, Symbol, or literal.

get_sig(x)

Returns the full signature of function, macro and datatype definitions. Should only be called when has_sig(x) == true.

is_getfield(x::EXPR)

Is this an expression of the form a.b.

is_wrapped_assignment(x::EXPR)

Is x an assignment expression, ignoring any surrounding parentheses.

lex_ws_comment(l::Lexer, c)

Having hit an initial whitespace/comment/semicolon continues collecting similar Chars until they end. Returns a WS token with an indication of newlines/ semicolons. Indicating a semicolons takes precedence over line breaks as the former is equivalent to the former in most cases.

parse(str, cont = false)

Parses the passed string. If cont is true then will continue parsing until the end of the string returning the resulting expressions in a TOPLEVEL block.

parse_compound(ps::ParseState, ret::EXPR)

Attempts to parse a compound expression given the preceding expression ret.

parse_doc(ps::ParseState)

Used for top-level parsing - attaches documentation (such as this) to expressions.

parse_expression(ps)

Parses an expression until closer(ps) == true. Expects to enter the ParseState the token before the the beginning of the expression and ends on the last token.

Acceptable starting tokens are:

• A keyword
• An opening parentheses or brace.
• An operator.
• An instance (e.g. identifier, number, etc.)
• An @.

parse_paren(ps, ret)

Parses an expression starting with a (.

EXPR represents Julia expressions overlaying a span of bytes in the source text. The full span starts at the first syntactically significant token and includes any trailing whitespace/comments.

Iterating or directly indexing EXPR results in a sequence of child EXPR in source order, including most syntax trivia but not including whitespace, comments and semicolons.

The fields of EXPR are:

• head represents the type of the expression

  • For internal tree nodes it usually matches the associated Expr's head field. But not always because there's some additional heads, for example :brackets for grouping parentheses, :globalrefdoc, :quotenode, etc
  • For leaf nodes (ie, individual tokens), it's capitalized. Eg, :INTEGER for integer tokens, :END for end, :LPAREN for [, etc.
  • For syntactic operators such as = and <: (which have the operator itself as the expression head in normal Expr), the head is an EXPR.

• args are the significant subexpressions, in the order used by Base.Expr. For leaf nodes, this is nothing.

• trivia are any nontrivial tokens which are trivial after parsing.

  • This includes things like the parentheses in (1 + 2), and the keywords in begin x end
  • Whitespace and comments are not included in trivia

• fullspan is the total number of bytes of text covered by this expression, including any trailing whitespace or comment trivia.

• span is the number of bytes of text covered by the syntactically relevant part of this expression (ie, not including trailing whitespace or comment trivia).

• val is the source text covered by span

• parent is the parent node in the expression tree, or Nothing for the root.

• meta contains metadata. This includes some ad-hoc information supplied by the parser. (But can also be used downstream in linting or static analysis.)

Whitespace, comments and semicolons are not represented explicitly. Rather, they're tacked onto the end of leaf tokens in args or trivia, in the last fullspan-span bytes of the token.

_do_kw_convert(ps::ParseState, a::EXPR)

Should a be converted to a keyword-argument expression?

_kw_convert(a::EXPR, blockwrap = false)

Converted an assignment expression to a keyword-argument expression.

can_become_chain(x::EXPR, op::EXPR)

Is x a binary call for + or * that can be extended to include more arguments?

can_become_comparison(x::EXPR)

Is x a binary comparison call (e.g. a < b) that can be extended to include more arguments?

check_span(x, neq = [])

Recursively checks whether the span of an expression equals the sum of the span of its components. Returns a vector of failing expressions.

closer(ps::ParseState)

A magical function determining whether the parsing of an expression should continue or stop.

compare(x,y)

Recursively checks whether two Base.Expr are the same. Returns unequal sub- expressions.

convertsigtotuple(sig::EXPR)

When parsing a function or macro signature, should it be converted to a tuple?

disallowednumberjuxt(ret::EXPR)

Does this number literal end in a decimal and so cannot precede a paren for implicit multiplication?

docable(head)

When parsing a block of expressions, can documentation be attached? Prefixed docs at the top-level are handled within parse(ps::ParseState, cont = false).

find_arg_at(x, i)

Returns the index of the node of x within which the byte offset i falls.

firstdiff(s0::AbstractString, s1::AbstractString)

Returns the last byte index, i, for which s0 and s1 are the same such that: s0[1:i] == s1[1:i]

has_error(ps::ParseState)
has_error(x::EXPR)

Determine whether a parsing error occured while processing text with the given ParseState, or exists as a (sub) expression of x.

revfirstdiff(s0::AbstractString, s1::AbstractString)

Reversed version of firstdiff but returns two indices, one for each string.

str_value(x)

Attempt to get a string representation of a nodeless expression.

@closer ps rule body

Continues parsing closing on rule.

@default ps body

Parses the next expression using default closure rules.

@nocloser ps rule body

Continues parsing not closing on rule.

@precedence ps prec body

Continues parsing binary operators until it hits a more loosely binding operator (with precdence lower than prec).

is_range(x::EXPR)

Is x a valid iterator for use in for loops or generators?

Continue parsing statements until an element of closers is hit (usually end). Statements are grouped in a Block EXPR.

parse_call(ps, ret)

Parses a function call. Expects to start before the opening parentheses and is passed the expression declaring the function name, ret.

Parses a comma separated list, optionally allowing for conversion of assignment (=) expressions to Kw.

Helper function for parsing import/using statements.

parse_filter(ps::ParseState, arg)

Parse a conditional filter following a generator.

parse_generator(ps)

Having hit for not at the beginning of an expression return a generator. Comprehensions are parsed as SQUAREs containing a generator.

Parses an iterator, allowing for the preceding keyword outer. Returns an error expression if an invalid expression is parsed (anything other than =, in, ∈).

parse_iterators(ps::ParseState, allowfilter = false)

Parses a group of iterators e.g. used in a for loop or generator. Can allow for a succeeding Filter expression.

parse_macrocall(ps)

Parses a macro call. Expects to start on the @.

parse_parameters(ps::ParseState, args::Vector{EXPR}, args1::Vector{EXPR} = EXPR[]; usekw = true)

Parses parameter arguments for a function call (e.g. following a semicolon).

parse_blockexpr(ps::ParseState, head)

General function for parsing block expressions comprised of a series of statements terminated by an end.

parse_blockexpr_sig(ps::ParseState, head)

Utility function to parse the signature of a block statement (i.e. any statement preceding the main body of the block). Returns nothing in some cases (e.g. begin end)

parse_if(ps, nested=false)

Parse an if block.

parse_kw(ps::ParseState)

Dispatch function for when the parser has reached a keyword.

parse_array(ps)

Having hit '[' return either:

• A vect
• A vcat
• A ncat
• A comprehension
• An array (vcat of hcats)

parse_curly(ps, ret)

Parses the juxtaposition of ret with an opening brace. Parses a comma seperated list.

parse_ref(ps, ret)

Handles cases where an expression - ret - is followed by [. Parses the following bracketed expression and modifies it's .head appropriately.

parse_tuple(ps, ret)

ret is followed by a comma so tries to parse the rest of the tuple.

parse_unary(ps)

Having hit a unary operator at the start of an expression return a call.

parse_prefixed_string_cmd(ps::ParseState, ret::EXPR)

Parse prefixed strings and commands such as pre"text".

parsestringor_cmd(ps)

When trying to make an INSTANCE from a string token we must check for interpolating operators.