PARSETOK (Parse Tokens)

Syntax:

PARSETOK(string,stemname[,nbd][,blankopt][,dropopt])

Arguments:

string

the string to be parsed into tokens. The string may be of any size and may contain any number of tokens (up to the storage limits of the machine).

stemname

the name of the stem array that will contain the parsed tokens. The combined lengths of the stemname argument and the largest subscript (including any periods) cannot exceed the REXX maximum length for symbols (250 characters). If stemname is to be a true REXX stem, code a period (.) as the last character. If you do not specify the period, the subscripts will abut the stem name without an intervening period. For example, if you specify a stemname of "ABC.", PARSETOK will create variables of the form "ABC.1", "ABC.2", "ABC.3", etc. If you specify a stemname of "ABC" (no period), PARSETOK will create variables of the form "ABC1", "ABC2", "ABC3", etc.

nbd

a string containing the non-blank delimiters to be used in the parsing process. Each byte in nbd is interpreted as a delimiter. Multi-byte delimiters are not supported. The characters of the nbd argument are treated as tokens, and are included in the stemname-specified array. If you specify a character more than once in nbd, only the first occurrence is used.

For the purpose of finding tokens, blanks are always treated as delimiters. However, unless you specify BLANKS (see blankopt below), blanks will not appear in the stemname-specified array. Furthermore, specifying the blank character in the nbd string has no effect. Thus, if you want to improve the readability of the nbd string, you can separate the delimiter characters with blanks.

blankopt

an option string (only the first character of which is recognized) used to indicate whether blanks themselves are to be treated as tokens and returned as elements of the stemname array. The valid values are:

Blanks: specifies that blanks are to be treated as tokens.
Noblanks: specifies that blanks are not to be treated as tokens. This is the default.

dropopt

an option string (only the first character of which is recognized) used to indicate whether the stem given in stemname should be dropped (all existing values are unassigned) prior to parsing the string. The valid values are:

Drop: specifies that stemname is to be dropped prior to parsing. All stemname variables will be restored to their original un-initialized state.
Nodrop: specifies that stemname is not to be dropped prior to parsing. Any stemname variables that are not overwritten by the parsing process will be preserved. This is the default.

Module Name:

SWXPTOK

Service Description:

The PARSETOK function is used to parse strings into tokens where the location and types of delimiters are difficult to predict. For example, consider the following string which contains keywords to be parsed:

keywords = 'DSN(ABC.DATA(LIST)) NOPARENKEYWRD'

Parsing this string with WORD or PARSE would be difficult because:

not all of the interesting tokens are delimited by blanks, and
delimiter tokens that we would like to use in breaking the string down (like the parentheses) are either nested or not present.

Using the PARSETOK function you can break the string down into its constituent tokens and parse it by successively examining each token.

Returned Information:

The PARSETOK function returns the number of stemname variables created. If you CALL the PARSETOK function, the returned value is contained in the RESULT special variable. If tokens are found, they will be returned in stemname-named variables. The zeroth element of the stemname array contains the number of stemname variables created. The RC special variable is unchanged.

Examples:

Call the PARSETOK function to parse STRING into words and place the resulting tokens into variables that begin with 'TOK.':
```
string = 'The rain in Spain'
call parsetok string, 'tok.'
/* tok.0 = 4; tok.1 = 'The'; tok.2 = 'rain';
tok.3 = 'in'; tok.4 = 'Spain' */
```
Note that while blanks are used as delimiters in this example (as always), they are not returned in the TOK. array since the default value for the blankopt argument is 'NOBLANKS'.
Call the PARSETOK function to parse KEYWORDS into tokens and place the resulting tokens into variables that begin with 'TOK.'. In addition to blanks, parentheses are to be used as delimiters. Blanks should be returned in the TOK. array:
```
keywords = 'dsn(abc) noname'
call parsetok keywords, 'tok.', '()', 'blanks'
/* tok.0 = 6; tok.1 = 'dsn'; tok.2 = '(';
tok.3 = 'abc'; tok.4 = ')'; tok.5 = ' ';
tok.6 = 'noname' */
```