DocToken
struct DocToken
Token.
A token is only valid for the lifetime of the Tokenizer that produced it, as it contains a reference to a substring of the source file.
Variables
| Name | Description |
|---|---|
| type | Token type. |
| value | Iterator to first character. |
Functions
| Name | Description |
|---|---|
| DocToken | Constructor. |
| view | Get token as string. |
| substr | Get substring of the token as a string. |
Variable Details
type
DocTokenType type
Token type.
value
TextLineCursor value
Iterator to first character.
Function Details
DocToken
DocToken(const DocTokenType type = NONE, TextLineCursor value = TextLineCursor())
Constructor.
type- Token type.
value- Token value.
substr
TextLineCursor substr(size_t pos = 0) const
Get substring of the token as a string.
pos- Position of the first character.
view
std::string_view view() const
Get token as string.