dlangui.dml.tokenizer

Members

Classes

ParserException

class ParserException

Undocumented in source.

Tokenizer

class Tokenizer

simple tokenizer for DlangUI ML

Enums

TokenType

enum TokenType

Undocumented in source.

Functions

tokenize

Token[] tokenize(string code, string[] _singleLineCommentPrefixes, bool skipSpace, bool skipEols, bool skipComments)

tokenize source into array of tokens (excluding EOF)

trimSpaceTokens

Token[] trimSpaceTokens(Token[] tokens, bool trimBeginning, bool trimEnd)

exclude whitespace tokens at beginning and end of token sequence

Structs

Token

struct Token

Undocumented in source.

• annotations
• dmlhighlight
• parser
• tokenizer