| Modifiers | Name | Description |
|---|---|---|
static byte |
COMMENT1 |
Comment 1 token id. |
static byte |
COMMENT2 |
Comment 2 token id. |
static byte |
END |
The token type, that along with a length of 0 marks the end of the token list. |
static byte |
ID_COUNT |
The total number of defined token ids. |
static byte |
INTERNAL_FIRST |
The first id that can be used for internal state in a token marker. |
static byte |
INTERNAL_LAST |
The last id that can be used for internal state in a token marker. |
static byte |
INVALID |
Invalid token id. |
static byte |
KEYWORD1 |
Keyword 1 token id. |
static byte |
KEYWORD2 |
Keyword 2 token id. |
static byte |
KEYWORD3 |
Keyword 3 token id. |
static byte |
LABEL |
Label token id. |
static byte |
LITERAL1 |
Literal 1 token id. |
static byte |
LITERAL2 |
Literal 2 token id. |
static byte |
NULL |
Normal text token id. |
static byte |
OPERATOR |
Operator token id. |
byte |
id |
The id of this token. |
int |
length |
The length of this token. |
Token |
next |
The next token in the linked list. |
| Constructor and description |
|---|
Token
(int length, byte id)Creates a new token. |
| Type Params | Return Type | Name and description |
|---|---|---|
|
java.lang.String |
toString()Returns a string representation of this token. |
| Methods inherited from class | Name |
|---|---|
class java.lang.Object |
java.lang.Object#wait(long), java.lang.Object#wait(long, int), java.lang.Object#wait(), java.lang.Object#equals(java.lang.Object), java.lang.Object#toString(), java.lang.Object#hashCode(), java.lang.Object#getClass(), java.lang.Object#notify(), java.lang.Object#notifyAll() |
Comment 1 token id. This can be used to mark a comment.
Comment 2 token id. This can be used to mark a comment.
The token type, that along with a length of 0 marks the end of the token list.
The total number of defined token ids.
The first id that can be used for internal state in a token marker.
The last id that can be used for internal state in a token marker.
Invalid token id. This can be used to mark invalid or incomplete tokens, so the user can easily spot syntax errors.
Keyword 1 token id. This can be used to mark a keyword. This should be used for general language constructs.
Keyword 2 token id. This can be used to mark a keyword. This should be used for preprocessor commands, or variables.
Keyword 3 token id. This can be used to mark a keyword. This should be used for data types.
Label token id. This can be used to mark labels (eg, C mode uses this to mark ...: sequences)
Literal 1 token id. This can be used to mark a string literal (eg, C mode uses this to mark "..." literals)
Literal 2 token id. This can be used to mark an object literal (eg, Java mode uses this to mark true, false, etc)
Normal text token id. This should be used to mark normal text.
Operator token id. This can be used to mark an operator. (eg, SQL mode marks +, -, etc with this token type)
The id of this token.
The length of this token.
The next token in the linked list.