lexer_state Struct Reference

Lexer state object. More...

#include <lexer.h>

Data Fields
const char *	buf
int	cursor
int	overread
void **	ref_to_data
char	lexeme [4096]
int	lexeme_cursor
int	end

Detailed Description

Lexer state object.

Field Documentation

buf

lexer_state::buf

Character buffer that is being processed

cursor

lexer_state::cursor

Current position in the buffer

end

lexer_state::end

Boolean that indicates the end of buffer's string.

lexeme

lexer_state::lexeme

For each successive call to lexer__next_terminal the terminal's lexeme string is stored in this array.

overread

lexer_state::overread

Following to the longest match rule, this variable keeps count of how many characters have we skipped since the last acceptable lexeme was found. If another pattern begins and it is not zero, that number is subtracted from the current cursor position, the last acceptable lexeme is determined and the search for the next terminal will begin from there.

ref_to_data

lexer_state::ref_to_data

Pointer to the data pointer of a state. When an acceptable lexeme is found we store the pointer to the pointer to the data of the accepting state that determined the terminal.