This class implements a simple scanner. More...

Inheritance diagram for Ocean::IO::Scanner:

Data Structures
class	Token
	This class implements a token for the scanner. More...

Public Member Functions
	Scanner (const std::shared_ptr< std::istream > &stream, float progress=nullptr, bool cancel=nullptr)
	Creates a new scanner using a stream as input. More...

	Scanner (const std::string &filename, const std::string &buffer, float progress=nullptr, bool cancel=nullptr)
	Creates a new scanner using a file or a memory buffer as input. More...

	Scanner (const std::string &filename, std::string &&buffer, float progress=nullptr, bool cancel=nullptr)
	Creates a new scanner using a file or a memory buffer as input. More...

virtual	~Scanner ()
	Destructs a scanner. More...

const Token &	token ()
	Returns the recent token. More...

const Token &	lineToken ()
	Returns a line token starting at the current position. More...

Token	tokenPop ()
	Return the recent token and pops it afterwards. More...

const Token &	look ()
	Returns a lookout to the next token. More...

void	pop ()
	Pops the recent token. More...

size_t	line () const
	Returns the recent line. More...

size_t	column () const
	Returns the recent column. More...

size_t	position () const
	Returns the position of the scanner. More...

size_t	size () const
	Returns the size of the scanner. More...

const std::string &	filename () const
	Returns the name of the input file, if the input is a file. More...

bool	isValid () const
	Returns whether the scanner is valid and ready to use. More...

Static Public Member Functions
static bool	findNextToken (const char *pointer, const size_t size, const size_t start, size_t &tokenStart, size_t &tokenLength)
	Finds the next token in a given string starting from a specified position. More...

static bool	findNextToken (const char *pointer, const size_t start, size_t &tokenStart, size_t &tokenLength)
	Finds the next token in a given string starting from a specified position. More...

static bool	isWhitespace (const char &character)
	Returns whether a given character is a white space character. More...

Static Public Attributes
static constexpr uint32_t	invalidId = uint32_t(-1)
	Definition of an invalid keyword or symbol id. More...

Protected Types
enum	FirstChar : uint16_t { CHAR_INVALID = 0 , CHAR_CHARACTER = 1 , CHAR_IDENTIFIER = 2 , CHAR_NUMBER = 4 , CHAR_INTEGER = 8 , CHAR_KEYWORD = 16 , CHAR_STRING = 32 , CHAR_SYMBOL = 64 , CHAR_REMARK = 128 , CHAR_SPACE = 256 }
	Definition of first character types. More...

typedef std::unordered_map< std::string, uint32_t >	IdMap
	Definition of an unordered map mapping strings to ids. More...

typedef std::unordered_set< std::string >	LineRemarks
	Definition of an unordered set holding line remark symbols. More...

typedef std::unordered_map< std::string, std::string >	ScopeRemarks
	Definition of an unordered map mapping begin remark symbols to end remark symbols. More...

typedef std::array< uint16_t, 256 >	CharTable
	Definition of a character table. More...

Protected Member Functions
	Scanner (float progress, bool cancel)
	Creates a new scanner. More...

uint8_t	get (const size_t offset=0)
	Returns one character. More...

std::string	data (const size_t size) const
	Returns data of a specified size starting at the recent position. More...

std::string	data (const size_t offset, const size_t size) const
	Returns data of a specified size starting at the offset position. More...

void	consume (const size_t chars=1)
	Consumes one or more character. More...

bool	refillIntermediateBuffer ()
	Refills the intermediate buffer. More...

uint32_t	keywordId (const std::string &data) const
	Returns the keyword id of a given string. More...

uint32_t	symbolId (const std::string &data) const
	Returns the symbol id of a given string. More...

void	setKeywordProperty (const bool caseSensitive)
	Sets whether the keywords are case sensitive or not. More...

void	registerKeyword (const std::string &keyword, const uint32_t id)
	Registers a new keyword. More...

void	registerSymbol (const std::string &symbol, const uint32_t id)
	Registers a new symbol. More...

void	registerLineRemark (const std::string &lineRemark)
	Registers a line remark symbol. More...

void	registerScopeRemark (const std::string &begin, const std::string &end)
	Registers a scope remark symbol. More...

bool	registerWhiteSpaceCharacter (const uint8_t character)
	Registers a white space character. More...

virtual Token	readToken (const bool consumeBytes=true)
	Reads and returns the next token. More...

uint8_t	readWhiteSpace (bool crossLines=true)
	Reads white space. More...

std::string	discardNonWhiteSpace ()
	Discards non white space and jumps to the first white space position. More...

bool	readRemark ()
	Reads remark comments. More...

bool	readLineRemark ()
	Reads a line remark comment. More...

bool	readScopeRemark ()
	Reads a scope remark comment. More...

bool	readCharacter (Token &token, const bool consumeBytes)
	Tries to read a character as next token. More...

bool	readIdentifier (Token &token, const bool consumeBytes)
	Tries to read a identifier as next token. More...

bool	readInteger (Token &token, const bool consumeBytes)
	Tries to read an integer as next token. More...

bool	readKeyword (Token &token, const bool consumeBytes)
	Tries to read a keyword as next token. More...

bool	readLine (Token &token, const bool consumeBytes)
	Tries to read a remaining line as next token. More...

bool	readNumber (Token &token, const bool consumeBytes)
	Tries to read a number as next token. More...

bool	readString (Token &token, const bool consumeBytes)
	Tries to read a string as next token. More...

bool	readSymbol (Token &token, const bool consumeBytes)
	Tries to read a symbol as next token. More...

Protected Attributes
Token	recentToken_
	Recent token. More...

Token	nextToken_
	Next token. More...

std::shared_ptr< std::istream >	stream_
	The input stream from which the scanner receives the data. More...

std::string	filename_
	The name of the input file, if the input is a file. More...

float *	progress_ = nullptr
	The scanner's progress in percent, with range [0, 1]. More...

bool *	cancel_ = nullptr
	Cancel flag. More...

Memory	intermediateBuffer_
	Local intermediate buffer. More...

uint8_t *	intermediateBufferPointer_ = nullptr
	The current pointer inside the intermediate buffer. More...

size_t	intermediateBufferSize_ = 0
	Number of remaining characters in the intermediate buffer. More...

Memory	extraBuffer_
	Local extra buffer, used if the intermediate buffer is too small. More...

uint8_t *	extraBufferPointer_ = nullptr
	Pointer inside the extra buffer. More...

size_t	extraBufferSize_ = 0
	Number of remaining characters inside the extra buffer. More...

size_t	line_ = 1
	Holds the current line. More...

size_t	column_ = 1
	Holds the current column. More...

size_t	position_ = 0
	Holds the current position of the scanner. More...

IdMap	keywordMap_
	Map mapping keyword strings to identifier ids. More...

bool	keywordsAreCaseSensitive_ = true
	Determines whether all keywords are case sensitive. More...

IdMap	symbolMap_
	Map mapping symbol strings to symbol ids. More...

LineRemarks	lineRemarks_
	Registered line remarks. More...

size_t	maximalLengthLineRemarks_ = 0
	Length of the maximal line remark. More...

ScopeRemarks	scopeRemarks_
	Scope remarks. More...

size_t	maximalLengthScopeRemarks_ = 0
	Length of the maximal scope remarks. More...

CharTable	firstCharTable_
	Table holding the definition of allowed first characters. More...

CharTable	followingCharTable_
	Table holding the definition of allowed following characters. More...

CharTable	invalidCharTable_
	Table holding the definition of not allowed following characters. More...

Static Protected Attributes
static constexpr size_t	minBufferSize_ = 2048
	Definition of the minimum intermediate buffer size. More...

static constexpr size_t	maxBufferSize_ = 8192
	Definition of the maximum intermediate buffer size. More...

Private Member Functions
uint8_t	getExtra (const size_t offset=0)
	Returns one character from the extra buffer. More...

bool	refillExtraBuffer (const size_t minIndex)
	Refills the extra buffer. More...

Static Private Member Functions
static std::shared_ptr< std::istream >	createInputStream (const std::string &filename, std::string &&buffer)
	Creates a file input stream or a string input stream depending on the given input. More...

static std::shared_ptr< std::istream >	createInputStream (const std::string &filename, const std::string &buffer)
	Creates a file input stream or a string input stream depending on the given input. More...

Detailed Description

This class implements a simple scanner.

Member Typedef Documentation

◆ CharTable

typedef std::array<uint16_t, 256> Ocean::IO::Scanner::CharTable

protected

Definition of a character table.

◆ IdMap

typedef std::unordered_map<std::string, uint32_t> Ocean::IO::Scanner::IdMap

protected

Definition of an unordered map mapping strings to ids.

◆ LineRemarks

typedef std::unordered_set<std::string> Ocean::IO::Scanner::LineRemarks

protected

Definition of an unordered set holding line remark symbols.

◆ ScopeRemarks

typedef std::unordered_map<std::string, std::string> Ocean::IO::Scanner::ScopeRemarks

protected

Definition of an unordered map mapping begin remark symbols to end remark symbols.

Member Enumeration Documentation

◆ FirstChar

enum Ocean::IO::Scanner::FirstChar : uint16_t

protected

Definition of first character types.

Enumerator
CHAR_INVALID	Invalid.
CHAR_CHARACTER	Character.
CHAR_IDENTIFIER	Identifier.
CHAR_NUMBER	Number.
CHAR_INTEGER	Integer.
CHAR_KEYWORD	Keyword.
CHAR_STRING	String.
CHAR_SYMBOL	Symbol.
CHAR_REMARK	Remark.
CHAR_SPACE	White space.

Constructor & Destructor Documentation

◆ Scanner() [1/4]

Ocean::IO::Scanner::Scanner	(	const std::shared_ptr< std::istream > &	stream,
		float *	progress = `nullptr`,
		bool *	cancel = `nullptr`
	)

explicit

Creates a new scanner using a stream as input.

Parameters

stream	The stream to be use as input
progress	Optional resulting scanner progress in percent, with range [0, 1]
cancel	Optional scanner cancel flag

◆ Scanner() [2/4]

Ocean::IO::Scanner::Scanner	(	const std::string &	filename,
		const std::string &	buffer,
		float *	progress = `nullptr`,
		bool *	cancel = `nullptr`
	)

inline

Creates a new scanner using a file or a memory buffer as input.

Parameters

filename	The name of the file to be used as input, `buffer` must be empty
buffer	The buffer to be used as input, `filename` must be empty
progress	Optional resulting scanner progress in percent, with range [0, 1]
cancel	Optional scanner cancel flag

◆ Scanner() [3/4]

Ocean::IO::Scanner::Scanner	(	const std::string &	filename,
		std::string &&	buffer,
		float *	progress = `nullptr`,
		bool *	cancel = `nullptr`
	)

inline

Creates a new scanner using a file or a memory buffer as input.

Parameters

filename	The name of the file to be used as input, `buffer` must be empty
buffer	The buffer to be used as input, `filename` must be empty
progress	Optional resulting scanner progress in percent, with range [0, 1]
cancel	Optional scanner cancel flag

◆ ~Scanner()

virtual Ocean::IO::Scanner::~Scanner ( )

virtual

Destructs a scanner.

◆ Scanner() [4/4]

Ocean::IO::Scanner::Scanner	(	float *	progress,
		bool *	cancel
	)

protected

Creates a new scanner.

The scanner may forward an entire progress state, if the pointer value is defined.
Beware: Make sure that the value exists during the whole scanning timer!
Further, the scanner may be canceled by an explicit flag.
In the case the scanner is canceled an end of file token is returned.
Beware: As for the progress value, also the cancel object must exist during the whole scanning progress, if provided

Parameters

progress	Optional progress parameter to forward the scanning progress with range [0, 1], use nullptr if the progress state is not necessary
cancel	Optional cancel state to cancel the scanner progress by setting the flag to 'true', use nullptr if the cancel state is not necessary

Member Function Documentation

◆ column()

size_t Ocean::IO::Scanner::column ( ) const

inline

Returns the recent column.

Returns: Recent column

◆ consume()

void Ocean::IO::Scanner::consume ( const size_t chars = 1 )

protected

Consumes one or more character.

Parameters

chars Number of characters to consume

◆ createInputStream() [1/2]

std::shared_ptr< std::istream > Ocean::IO::Scanner::createInputStream	(	const std::string &	filename,
		const std::string &	buffer
	)

inlinestaticprivate

Creates a file input stream or a string input stream depending on the given input.

Parameters

filename	The name of the file to be used as input, `buffer` must be empty
buffer	The buffer to be used as input, `filename` must be empty

◆ createInputStream() [2/2]

std::shared_ptr< std::istream > Ocean::IO::Scanner::createInputStream	(	const std::string &	filename,
		std::string &&	buffer
	)

inlinestaticprivate

Creates a file input stream or a string input stream depending on the given input.

Parameters

filename	The name of the file to be used as input, `buffer` must be empty
buffer	The buffer to be used as input, `filename` must be empty

◆ data() [1/2]

std::string Ocean::IO::Scanner::data	(	const size_t	offset,
		const size_t	size
	)		const

protected

Returns data of a specified size starting at the offset position.

Beware: Make sure that enough pending buffer is available

Parameters

offset	Start position relative to the current position
size	Size of the data to receive

Returns: Requested data

◆ data() [2/2]

std::string Ocean::IO::Scanner::data ( const size_t size ) const

protected

Returns data of a specified size starting at the recent position.

Beware: Make sure that enough pending buffer is available

Parameters

size	Size of the data to receive

Returns: Requested data

◆ discardNonWhiteSpace()

std::string Ocean::IO::Scanner::discardNonWhiteSpace ( )

protected

Discards non white space and jumps to the first white space position.

Returns: Discarded elements

◆ filename()

const std::string & Ocean::IO::Scanner::filename ( ) const

inline

Returns the name of the input file, if the input is a file.

Returns: The scanner's input filename, empty if the input was a buffer

◆ findNextToken() [1/2]

static bool Ocean::IO::Scanner::findNextToken	(	const char *	pointer,
		const size_t	size,
		const size_t	start,
		size_t &	tokenStart,
		size_t &	tokenLength
	)

static

Finds the next token in a given string starting from a specified position.

A token is enclosed by white characters or by the borders of the given string, the length of the given string is explicitly defined by the parameter 'size'.

Parameters

pointer	The pointer to the string in which the next token is to be found, must be valid
size	The length of the given string in characters, with range [1, infinity)
start	The first character within the given string that defines the first possible character of the token, with range [0, size - 1]
tokenStart	The resulting start location within the given string of the found token, with range [start, strlen(pointer) - 1]
tokenLength	The resulting length of the found token, with range [1, strlen(pointer) - start]

Returns: True, if a second token may follow; False, if the token is the last token

◆ findNextToken() [2/2]

static bool Ocean::IO::Scanner::findNextToken	(	const char *	pointer,
		const size_t	start,
		size_t &	tokenStart,
		size_t &	tokenLength
	)

static

Finds the next token in a given string starting from a specified position.

A token is enclosed by white characters or by the borders of the given string, the end is identified by a null character.

Parameters

pointer	The pointer to the string in which the next token is to be found, can be nullptr
start	The first character within the given string that defines the first possible character of the token, with range [0, strlen(pointer)]
tokenStart	The resulting start location within the given string of the found token, with range [start, strlen(pointer) - 1]
tokenLength	The resulting length of the found token, with range [1, strlen(pointer) - start]

Returns: True, if a second token may follow; False, if the token is the last token

◆ get()

uint8_t Ocean::IO::Scanner::get ( const size_t offset = 0 )

protected

Returns one character.

Parameters

offset Offset to the recent position

Returns: Character

◆ getExtra()

uint8_t Ocean::IO::Scanner::getExtra ( const size_t offset = 0 )

private

Returns one character from the extra buffer.

Parameters

offset Offset inside the recent extra buffer

Returns: Character

◆ isValid()

bool Ocean::IO::Scanner::isValid ( ) const

inline

Returns whether the scanner is valid and ready to use.

Returns: True, if so

◆ isWhitespace()

bool Ocean::IO::Scanner::isWhitespace ( const char & character )

inlinestatic

Returns whether a given character is a white space character.

A white space character can be one of the following:

' ', '\t', '\n', or '\r'

Parameters

character The character to be checked

Returns: True, if so

◆ keywordId()

uint32_t Ocean::IO::Scanner::keywordId ( const std::string & data ) const

protected

Returns the keyword id of a given string.

Parameters

data	Data to convert to a keyword

Returns: Id of the identifier, otherwise invalidId

◆ line()

size_t Ocean::IO::Scanner::line ( ) const

inline

Returns the recent line.

Returns: Recent line

◆ lineToken()

const Token& Ocean::IO::Scanner::lineToken ( )

Returns a line token starting at the current position.

A line token does not handle remarks.

Returns: Line token

◆ look()

const Token& Ocean::IO::Scanner::look ( )

Returns a lookout to the next token.

Returns: Next token

◆ pop()

void Ocean::IO::Scanner::pop ( )

Pops the recent token.

◆ position()

size_t Ocean::IO::Scanner::position ( ) const

Returns the position of the scanner.

Returns: Position of the scanner in characters

◆ readCharacter()

bool Ocean::IO::Scanner::readCharacter	(	Token &	token,
		const bool	consumeBytes
	)

protected

Tries to read a character as next token.

Parameters

token	Returning token
consumeBytes	Determines whether the scanner consumes the read characters

Returns: True, if succeeded

◆ readIdentifier()

bool Ocean::IO::Scanner::readIdentifier	(	Token &	token,
		const bool	consumeBytes
	)

protected

Tries to read a identifier as next token.

Parameters

token	Returning token
consumeBytes	Determines whether the scanner consumes the read characters

Returns: True, if succeeded

◆ readInteger()

bool Ocean::IO::Scanner::readInteger	(	Token &	token,
		const bool	consumeBytes
	)

protected

Tries to read an integer as next token.

Parameters

token	Returning token
consumeBytes	Determines whether the scanner consumes the read characters

Returns: True, if succeeded

◆ readKeyword()

bool Ocean::IO::Scanner::readKeyword	(	Token &	token,
		const bool	consumeBytes
	)

protected

Tries to read a keyword as next token.

Parameters

token	Returning token
consumeBytes	Determines whether the scanner consumes the read characters

Returns: True, if succeeded

◆ readLine()

bool Ocean::IO::Scanner::readLine	(	Token &	token,
		const bool	consumeBytes
	)

protected

Tries to read a remaining line as next token.

Parameters

token	Returning token
consumeBytes	Determines whether the scanner consumes the read characters

Returns: True, if succeeded

◆ readLineRemark()

bool Ocean::IO::Scanner::readLineRemark ( )

protected

Reads a line remark comment.

Returns: True, if a comment was read

◆ readNumber()

bool Ocean::IO::Scanner::readNumber	(	Token &	token,
		const bool	consumeBytes
	)

protected

Tries to read a number as next token.

Parameters

token	Returning token
consumeBytes	Determines whether the scanner consumes the read characters

Returns: True, if succeeded

◆ readRemark()

bool Ocean::IO::Scanner::readRemark ( )

protected

Reads remark comments.

Returns: True, if a comment was read

◆ readScopeRemark()

bool Ocean::IO::Scanner::readScopeRemark ( )

protected

Reads a scope remark comment.

Returns: True, if a comment was read

◆ readString()

bool Ocean::IO::Scanner::readString	(	Token &	token,
		const bool	consumeBytes
	)

protected

Tries to read a string as next token.

Parameters

token	Returning token
consumeBytes	Determines whether the scanner consumes the read characters

Returns: True, if succeeded

◆ readSymbol()

bool Ocean::IO::Scanner::readSymbol	(	Token &	token,
		const bool	consumeBytes
	)

protected

Tries to read a symbol as next token.

Parameters

token	Returning token
consumeBytes	Determines whether the scanner consumes the read characters

Returns: True, if succeeded

◆ readToken()

virtual Token Ocean::IO::Scanner::readToken ( const bool consumeBytes = true )

protectedvirtual

Reads and returns the next token.

Parameters

consumeBytes Determines whether the scanner consume the read characters.

Returns: New token

◆ readWhiteSpace()

uint8_t Ocean::IO::Scanner::readWhiteSpace ( bool crossLines = true )

protected

Reads white space.

Parameters

crossLines Determines whether the white space can be separated over several lines

Returns: Next not-white-space character

◆ refillExtraBuffer()

bool Ocean::IO::Scanner::refillExtraBuffer ( const size_t minIndex )

private

Refills the extra buffer.

Parameters

minIndex Minimal index of the character needed inside the extra buffer

Returns: True, if enough characters could be read

◆ refillIntermediateBuffer()

bool Ocean::IO::Scanner::refillIntermediateBuffer ( )

protected

Refills the intermediate buffer.

Returns: True, if the buffer could be refilled with new characters

◆ registerKeyword()

void Ocean::IO::Scanner::registerKeyword	(	const std::string &	keyword,
		const uint32_t	id
	)

protected

Registers a new keyword.

Parameters

keyword	New keyword
id	Id of the keyword

◆ registerLineRemark()

void Ocean::IO::Scanner::registerLineRemark ( const std::string & lineRemark )

protected

Registers a line remark symbol.

Parameters

lineRemark Line remark symbol

◆ registerScopeRemark()

void Ocean::IO::Scanner::registerScopeRemark	(	const std::string &	begin,
		const std::string &	end
	)

protected

Registers a scope remark symbol.

Parameters

begin	Begin remark symbol
end	End remark symbol

◆ registerSymbol()

void Ocean::IO::Scanner::registerSymbol	(	const std::string &	symbol,
		const uint32_t	id
	)

protected

Registers a new symbol.

Parameters

symbol	New symbol
id	Id of the symbol

◆ registerWhiteSpaceCharacter()

bool Ocean::IO::Scanner::registerWhiteSpaceCharacter ( const uint8_t character )

protected

Registers a white space character.

Parameters

character White space character to register

Returns: True, if succeeded

◆ setKeywordProperty()

void Ocean::IO::Scanner::setKeywordProperty ( const bool caseSensitive )

protected

Sets whether the keywords are case sensitive or not.

As default all keywords are case sensitive.
Beware: This property has to be set before the first keyword is registered!

Parameters

caseSensitive True, if all keywords will be case sensitive

◆ size()

size_t Ocean::IO::Scanner::size ( ) const

Returns the size of the scanner.

Returns: Size of the scanner in characters

◆ symbolId()

uint32_t Ocean::IO::Scanner::symbolId ( const std::string & data ) const

protected

Returns the symbol id of a given string.

Parameters

data	Data to convert to a symbol

Returns: Id of the symbol, otherwise invalidId

◆ token()

const Token& Ocean::IO::Scanner::token ( )

Returns the recent token.

Returns: Recent token

◆ tokenPop()

Token Ocean::IO::Scanner::tokenPop ( )

Return the recent token and pops it afterwards.

Returns: Recent token.

Field Documentation

◆ cancel_

bool* Ocean::IO::Scanner::cancel_ = nullptr

protected

Cancel flag.

◆ column_

size_t Ocean::IO::Scanner::column_ = 1

protected

Holds the current column.

◆ extraBuffer_

Memory Ocean::IO::Scanner::extraBuffer_

protected

Local extra buffer, used if the intermediate buffer is too small.

◆ extraBufferPointer_

uint8_t* Ocean::IO::Scanner::extraBufferPointer_ = nullptr

protected

Pointer inside the extra buffer.

◆ extraBufferSize_

size_t Ocean::IO::Scanner::extraBufferSize_ = 0

protected

Number of remaining characters inside the extra buffer.

◆ filename_

std::string Ocean::IO::Scanner::filename_

protected

The name of the input file, if the input is a file.

◆ firstCharTable_

CharTable Ocean::IO::Scanner::firstCharTable_

protected

Table holding the definition of allowed first characters.

◆ followingCharTable_

CharTable Ocean::IO::Scanner::followingCharTable_

protected

Table holding the definition of allowed following characters.

◆ intermediateBuffer_

Memory Ocean::IO::Scanner::intermediateBuffer_

protected

Local intermediate buffer.

◆ intermediateBufferPointer_

uint8_t* Ocean::IO::Scanner::intermediateBufferPointer_ = nullptr

protected

The current pointer inside the intermediate buffer.

◆ intermediateBufferSize_

size_t Ocean::IO::Scanner::intermediateBufferSize_ = 0

protected

Number of remaining characters in the intermediate buffer.

◆ invalidCharTable_

CharTable Ocean::IO::Scanner::invalidCharTable_

protected

Table holding the definition of not allowed following characters.

◆ invalidId

constexpr uint32_t Ocean::IO::Scanner::invalidId = uint32_t(-1)

staticconstexpr

Definition of an invalid keyword or symbol id.

◆ keywordMap_

IdMap Ocean::IO::Scanner::keywordMap_

protected

Map mapping keyword strings to identifier ids.

◆ keywordsAreCaseSensitive_

bool Ocean::IO::Scanner::keywordsAreCaseSensitive_ = true

protected

Determines whether all keywords are case sensitive.

◆ line_

size_t Ocean::IO::Scanner::line_ = 1

protected

Holds the current line.

◆ lineRemarks_

LineRemarks Ocean::IO::Scanner::lineRemarks_

protected

Registered line remarks.

◆ maxBufferSize_

constexpr size_t Ocean::IO::Scanner::maxBufferSize_ = 8192

staticconstexprprotected

Definition of the maximum intermediate buffer size.

◆ maximalLengthLineRemarks_

size_t Ocean::IO::Scanner::maximalLengthLineRemarks_ = 0

protected

Length of the maximal line remark.

◆ maximalLengthScopeRemarks_

size_t Ocean::IO::Scanner::maximalLengthScopeRemarks_ = 0

protected

Length of the maximal scope remarks.

◆ minBufferSize_

constexpr size_t Ocean::IO::Scanner::minBufferSize_ = 2048

staticconstexprprotected

Definition of the minimum intermediate buffer size.

◆ nextToken_

Token Ocean::IO::Scanner::nextToken_

protected

Next token.

◆ position_

size_t Ocean::IO::Scanner::position_ = 0

protected

Holds the current position of the scanner.

◆ progress_

float* Ocean::IO::Scanner::progress_ = nullptr

protected

The scanner's progress in percent, with range [0, 1].

◆ recentToken_

Token Ocean::IO::Scanner::recentToken_

protected

Recent token.

◆ scopeRemarks_

ScopeRemarks Ocean::IO::Scanner::scopeRemarks_

protected

Scope remarks.

◆ stream_

std::shared_ptr<std::istream> Ocean::IO::Scanner::stream_

protected

The input stream from which the scanner receives the data.

◆ symbolMap_

IdMap Ocean::IO::Scanner::symbolMap_

protected

Map mapping symbol strings to symbol ids.

The documentation for this class was generated from the following file:

Scanner.h

Data Structures

Public Member Functions

Static Public Member Functions

Static Public Attributes

Protected Types

Protected Member Functions

Protected Attributes

Static Protected Attributes

Private Member Functions

Static Private Member Functions

Detailed Description

Member Typedef Documentation

◆ CharTable

◆ IdMap

◆ LineRemarks

◆ ScopeRemarks

Member Enumeration Documentation

◆ FirstChar

Constructor & Destructor Documentation

◆ Scanner() [1/4]

◆ Scanner() [2/4]

◆ Scanner() [3/4]

◆ ~Scanner()

◆ Scanner() [4/4]

Member Function Documentation

◆ column()

◆ consume()

◆ createInputStream() [1/2]

◆ createInputStream() [2/2]

◆ data() [1/2]

◆ data() [2/2]

◆ discardNonWhiteSpace()

◆ filename()

◆ findNextToken() [1/2]

◆ findNextToken() [2/2]

◆ get()

◆ getExtra()

◆ isValid()

◆ isWhitespace()

◆ keywordId()

◆ line()

◆ lineToken()

◆ look()

◆ pop()

◆ position()

◆ readCharacter()

◆ readIdentifier()

◆ readInteger()

◆ readKeyword()

◆ readLine()

◆ readLineRemark()

◆ readNumber()

◆ readRemark()

◆ readScopeRemark()

◆ readString()

◆ readSymbol()

◆ readToken()

◆ readWhiteSpace()

◆ refillExtraBuffer()

◆ refillIntermediateBuffer()

◆ registerKeyword()

◆ registerLineRemark()

◆ registerScopeRemark()

◆ registerSymbol()

◆ registerWhiteSpaceCharacter()

◆ setKeywordProperty()

◆ size()

◆ symbolId()

◆ token()

◆ tokenPop()

Field Documentation

◆ cancel_

◆ column_

◆ extraBuffer_

◆ extraBufferPointer_

◆ extraBufferSize_

◆ filename_

◆ firstCharTable_

◆ followingCharTable_

◆ intermediateBuffer_