![]() |
TokenBlock |
Header: Script.h |
struct TokenBlock { Ptr source; SInt32 sourceLength; Ptr tokenList; SInt32 tokenLength; SInt32 tokenCount; Ptr stringList; SInt32 stringLength; SInt32 stringCount; Boolean doString; Boolean doAppend; Boolean doAlphanumeric; Boolean doNest; ScriptTokenType leftDelims[2]; ScriptTokenType rightDelims[2]; ScriptTokenType leftComment[4]; ScriptTokenType rightComment[4]; ScriptTokenType escapeCode; ScriptTokenType decimalCode; Handle itlResource; SInt32 reserved[8]; }; typedef TokenBlock TokenBlockPtr;
A pointer to a stream of characters.
The length of the source stream.
A pointer to an array of tokens.
The maximum length of TokenList
The number of tokens generated by the tokenizer.
A pointer to a stream of identifiers.
The length of the string list.
The number of bytes currently used.
A Boolean value indicating, on input, whether the IntlTokenize function should create a Pascal string representing the contents of each token it generates.
A Boolean value indicating, on input, whether to append tokens and strings it generates to the current token list and string list.
A Boolean value indicating, on input, whether to interpret numeric characters as alphabetic when mixed with alphabetic characters.
A Boolean value indicating, on input, whether to allow nested comments (to any depth of nesting).
An array of two integers, each of which contains the token code of the symbol that may be used as an opening delimiter for a quoted literal. If only one opening delimiter is needed, the other must be specified to be delimPad.
An array of two integers, each of which contains the token code of the symbol that may be used as the matching closing delimiter for the corresponding opening delimiter in the leftDelims field.
An array of two pairs of integers, each pair of which contains codes for the two token types that may be used as opening delimiters for comments.
An array of two pairs of integers, each pair of which contains codes for the two token types that may be used as closing delimiters for comments.
A single integer that contains the token code for the symbol that may be an escape character within a quoted literal.
A single integer that contains the token type of the symbol to be used for a decimal point
A handle to the tokens ('itl4') resource of the script system under which the source text was created.
An 8-byte array of type LongInt.On input, this must be set to 0.
The token block structure is a parameter block used to pass information to the IntlTokenize function and to retrieve results from it.
© 2000 Apple Computer, Inc. (Last Updated 6/30/2000)