Parsek/org.kobjects.parsek.tokenizer/Scanner

Scanner

open class Scanner<T>(val input: Iterator<Token<T>>, val eofType: T, val eofText: String = "")

Class that wraps a "plain" lexer (implemented as a token iterator) for more comfortable token analysis in a parser.

Constructors

Scanner

constructor(input: Iterator<Token<T>>, eofType: T, eofText: String = "")

Properties

current

val current: Token<T>

val eof: Boolean

val eofText: String

val eofType: T

val input: Iterator<Token<T>>

Functions

consume

fun consume(): Token<T>

Consumes the current token and returns its text content.

fun consume(type: T, errorMessage: (Token<T>) -> String = { "Token type $type expected." }): Token<T>

Consumes a token with the given type and returns it. If the current token has a different type, an exception is thrown.

fun consume(text: String, ignoreCase: Boolean = false, errorMessage: (Token<T>) -> String = {"Expected: '$text'"}): Token<T>

Consume and return a token with the given text value. If the current token type does not match, an exception is thrown.

ensureParsingException

fun ensureParsingException(e: Exception): ParsingException

Wraps the given exception in a parsing exception if it's not a parsing exception already. Parsing exceptions are returned unchanged. Useful to associate other exceptions encountered during parsing with a token (including the corresponding input position).

exception

fun exception(message: String): ParsingException

Creates an illegal state exception with position context information.

lookAhead

fun lookAhead(index: Int): Token<T>

require

fun require(condition: Boolean, message: () -> String)

requireEof

fun requireEof(message: () -> String = { "EOF expected." })

toString

open override fun toString(): String

tryConsume

fun tryConsume(value: String, ignoreCase: Boolean = false): Boolean

If the current token text value matches the given string, it is consumed and true is returned. Otherwise, false is returned.