Element Configurable Tokenizer Config $Token Type

Name

Type

Required

Description

Token Characters

String

No

A set of characters which are acceptable as consecutive characters within the same token (i.e. these character are acceptable values which can follow the beginning of the token).

Token Start Characters

String

No

A set of characters which, when found in the input string, represent the beginning of a new token.


iWay Software