Ocean::IO::Scanner::Token Class Reference

This class implements a token for the scanner. More...

#include <Scanner.h>

Public Types
enum	Type : uint32_t {   TOKEN_INVALID = 0u , TOKEN_CHARACTER , TOKEN_END_OF_FILE , TOKEN_IDENTIFIER ,   TOKEN_LINE , TOKEN_NUMBER , TOKEN_INTEGER , TOKEN_KEYWORD ,   TOKEN_STRING , TOKEN_SYMBOL }
	Definition of different token types. More...

Public Member Functions
	Token ()=default
	Creates an invalid token.
	Token (std::string &&data, const Type type)
	Creates a new token with given data and type.
	Token (const std::string &data, const Type type)
	Creates a new token with given data and type.
	Token (std::string &&data, const uint32_t id, const Type type)
	Creates a new token with given keyword or symbol id.
	Token (const std::string &data, const uint32_t id, const Type type)
	Creates a new token with given keyword or symbol id.
Type	type () const
	Returns the type of the token.
bool	isType (const Type type) const
	Returns whether this token is of a specific type.
bool	isCharacter () const
	Returns whether this token holds a character.
bool	isIdentifier () const
	Returns whether this token holds an identifier.
bool	isInteger () const
	Returns whether this token holds an integer.
bool	isLine () const
	Returns whether this token holds a remaining line.
bool	isNumber () const
	Returns whether this token holds a number.
bool	isIntegerOrNumber () const
	Returns whether this token holds an integer or a number.
bool	isKeyword () const
	Returns whether this token holds a keyword.
bool	isKeyword (const uint32_t keyword) const
	Returns whether this token holds a special keyword.
bool	isString () const
	Returns whether this token holds a string.
bool	isSymbol () const
	Returns whether this token holds a symbol.
bool	isSymbol (const uint32_t symbol) const
	Returns whether this token holds a special symbol.
bool	isEndOfFile () const
	Returns whether this token holds a end of file.
uint8_t	character () const
	Returns the character value of this token.
const std::string &	identifier () const
	Returns the identifier value of this token.
int	integer () const
	Returns the integer value of this token.
Scalar	number () const
	Returns the float value of this token.
Scalar	integerOrNumber () const
	Returns the integer or float value of this token.
const std::string &	line () const
	Returns the remaining line of this token.
uint32_t	keyword () const
	Returns the id of the keyword of this token.
uint32_t	symbol () const
	Returns the id of the symbol of this token.
const std::string &	string () const
	Returns the string value of this token.
std::string	moveString ()
	Returns the string value of this token and invalidates the token.
const std::string &	raw () const
	Returns the raw data of the token.
	operator bool () const
	Returns whether the token is valid.

Protected Attributes
Type	type_ = TOKEN_INVALID
	Holds the type of the token.
uint32_t	id_ = invalidId
	Holds the id of the keyword or symbol, if any.
std::string	data_
	Holds the data of the token.

Detailed Description

This class implements a token for the scanner.

Member Enumeration Documentation

Type

enum Ocean::IO::Scanner::Token::Type : uint32_t

Definition of different token types.

Enumerator
TOKEN_INVALID	Invalid token.
TOKEN_CHARACTER	Character token.
TOKEN_END_OF_FILE	End of file token.
TOKEN_IDENTIFIER	Identifier token.
TOKEN_LINE	Line token.
TOKEN_NUMBER	Number token.
TOKEN_INTEGER	Integer token.
TOKEN_KEYWORD	Keyword token.
TOKEN_STRING	String token.
TOKEN_SYMBOL	Symbol token.

Constructor & Destructor Documentation

Token() [1/5]

Ocean::IO::Scanner::Token::Token ( )

default

Creates an invalid token.

Token() [2/5]

Ocean::IO::Scanner::Token::Token	(	std::string &&	data,
		const Type	type
	)

Creates a new token with given data and type.

Parameters

data	Token data
type	Token type

Token() [3/5]

Ocean::IO::Scanner::Token::Token	(	const std::string &	data,
		const Type	type
	)

Creates a new token with given data and type.

Parameters

data	Token data
type	Token type

Token() [4/5]

Ocean::IO::Scanner::Token::Token	(	std::string &&	data,
		const uint32_t	id,
		const Type	type
	)

Creates a new token with given keyword or symbol id.

Parameters

data	Raw token data
id	Keyword or symbol id of the token
type	Token type

Token() [5/5]

Ocean::IO::Scanner::Token::Token	(	const std::string &	data,
		const uint32_t	id,
		const Type	type
	)

Creates a new token with given keyword or symbol id.

Parameters

data	Raw token data
id	Keyword or symbol id of the token
type	Token type

Member Function Documentation

character()

uint8_t Ocean::IO::Scanner::Token::character

(

)

const

Returns the character value of this token.

Returns

Character value

identifier()

const std::string & Ocean::IO::Scanner::Token::identifier

(

)

const

Returns the identifier value of this token.

Returns

Identifier value

integer()

int Ocean::IO::Scanner::Token::integer

(

)

const

Returns the integer value of this token.

Returns

Integer value

integerOrNumber()

Scalar Ocean::IO::Scanner::Token::integerOrNumber

(

)

const

Returns the integer or float value of this token.

Returns

Integer or float value as float value

isCharacter()

bool Ocean::IO::Scanner::Token::isCharacter ( ) const

inline

Returns whether this token holds a character.

Returns

True, if so

isEndOfFile()

bool Ocean::IO::Scanner::Token::isEndOfFile ( ) const

inline

Returns whether this token holds a end of file.

Returns

True, if so

isIdentifier()

bool Ocean::IO::Scanner::Token::isIdentifier ( ) const

inline

Returns whether this token holds an identifier.

Returns

True, if so

isInteger()

bool Ocean::IO::Scanner::Token::isInteger ( ) const

inline

Returns whether this token holds an integer.

Returns

True, if so

isIntegerOrNumber()

bool Ocean::IO::Scanner::Token::isIntegerOrNumber ( ) const

inline

Returns whether this token holds an integer or a number.

Returns

True, if so

isKeyword() [1/2]

bool Ocean::IO::Scanner::Token::isKeyword ( ) const

inline

Returns whether this token holds a keyword.

Returns

True, if so

isKeyword() [2/2]

bool Ocean::IO::Scanner::Token::isKeyword

(

const uint32_t

keyword

)

const

Returns whether this token holds a special keyword.

Parameters

keyword

The keyword id to check

Returns

True, if so

isLine()

bool Ocean::IO::Scanner::Token::isLine ( ) const

inline

Returns whether this token holds a remaining line.

Returns

True, if so

isNumber()

bool Ocean::IO::Scanner::Token::isNumber ( ) const

inline

Returns whether this token holds a number.

Returns

True, if so

isString()

bool Ocean::IO::Scanner::Token::isString ( ) const

inline

Returns whether this token holds a string.

Returns

True, if so

isSymbol() [1/2]

bool Ocean::IO::Scanner::Token::isSymbol ( ) const

inline

Returns whether this token holds a symbol.

Returns

True, if so

isSymbol() [2/2]

bool Ocean::IO::Scanner::Token::isSymbol

(

const uint32_t

symbol

)

const

Returns whether this token holds a special symbol.

Parameters

symbol

The symbol id to check

Returns

True, if so

isType()

bool Ocean::IO::Scanner::Token::isType ( const Type  type ) const

inline

Returns whether this token is of a specific type.

Parameters

type	The type to check

Returns

True, if so

keyword()

uint32_t Ocean::IO::Scanner::Token::keyword

(

)

const

Returns the id of the keyword of this token.

Returns

Keyword id

line()

const std::string & Ocean::IO::Scanner::Token::line

(

)

const

Returns the remaining line of this token.

Returns

Line value

moveString()

std::string Ocean::IO::Scanner::Token::moveString

(

)

Returns the string value of this token and invalidates the token.

Returns

The token's string value

number()

Scalar Ocean::IO::Scanner::Token::number

(

)

const

Returns the float value of this token.

Returns

Float value

operator bool()

Ocean::IO::Scanner::Token::operator bool ( ) const

inlineexplicit

Returns whether the token is valid.

Returns

True, if so

raw()

const std::string & Ocean::IO::Scanner::Token::raw ( ) const

inline

Returns the raw data of the token.

Returns

Raw data

string()

const std::string & Ocean::IO::Scanner::Token::string

(

)

const

Returns the string value of this token.

Returns

The token's string value

symbol()

uint32_t Ocean::IO::Scanner::Token::symbol

(

)

const

Returns the id of the symbol of this token.

Returns

Symbol id

type()

Scanner::Token::Type Ocean::IO::Scanner::Token::type ( ) const

inline

Returns the type of the token.

Returns

Token type

Field Documentation

data_

std::string Ocean::IO::Scanner::Token::data_

protected

Holds the data of the token.

id_

uint32_t Ocean::IO::Scanner::Token::id_ = invalidId

protected

Holds the id of the keyword or symbol, if any.

type_

Type Ocean::IO::Scanner::Token::type_ = TOKEN_INVALID

protected

Holds the type of the token.

---

The documentation for this class was generated from the following file:

• Scanner.h