Package-level declarations

Contains the lexer, scanner, token representation, and common regular expressions.

Types

Link copied to clipboard
abstract class Filter<T>(val filtered: Iterator<Token<T>>) : Iterator<Token<T>>

A Filter for the lexer, useful for filtering (e.g.) insignificant newlines. The accept function is not just a parameter but needs to be overridden in a subclass as it's expected to be stateful.

Link copied to clipboard
open class Lexer<T>(input: String, vararg types: Pair<Regex, T?>, offset: Token<T>? = null) : Iterator<Token<T>>

Splits input into tokens, exposed as a lazy Iterator.

Link copied to clipboard
class ParsingException(val token: Token<*>, message: String?, cause: Throwable? = null) : Exception

Exception type for parsing errors containing a reference to the current token.

Link copied to clipboard

A set of regular expressions that might be useful for parsing.

Link copied to clipboard
open class Scanner<T>(val input: Iterator<Token<T>>, val eofType: T)

Wraps a token Iterator (or tokenizes an input String directly via regular expression rules) with current-token inspection, unlimited lookahead, and consumption helpers for recursive-descent and Pratt parsers.

Link copied to clipboard
data class Token<T>(val localPos: Int, val pos: Int, val line: Int, val col: Int, val type: T, val text: String)

Tokens in parsek are represented by this class.