Package-level declarations

State machine for block-level lexing.

Lexes a flat list of lexemes into block-level tokens.

Detects the HTML block type (1–7) for a line, or 0 if not an HTML block start. Requires 0–3 leading spaces.

Checks if a line contains the end condition for an HTML block type.

Converts a list of lexemes back to their source text representation.

Splits a flat list of lexemes into logical lines. Each line is a list of lexemes up to (and including) a Lexeme.Newline. The last line may not end with a newline.

Strips exactly count spaces worth of indent from the front of a line. Tabs count as enough spaces to reach the next tab stop (multiple of 4).

Tries to lex an ATX heading: 0–3 spaces, 1–6 #s, then space/tab/line-end. Returns a pair of (AtxHeadingMarker, AtxHeadingContent?) or null.

Tries to lex a blank line: a line with only whitespace (and newline).

Tries to lex a closing code fence that matches the given opening.

Tries to lex a code fence opening: 0–3 spaces, 3+ backticks or tildes. Returns (CodeFenceOpen, CodeFenceInfo?) or null.

Tries to lex an indented code line: 4+ spaces (or 1+ tab) of leading indent. Returns the line with the indent prefix stripped.

Tries to lex a setext heading underline: 0–3 spaces, then 1+ = or -, optional trailing spaces.

Tries to lex a thematic break: 0–3 spaces, then 3+ of same marker (-,*,_) interspersed with optional spaces/tabs, then line ending/EOF.