BPAM Functions

BPAM, which stands for Basic Partitioned Access Method, is used to process Partitioned Data Sets (PDS). Partitioned Data Set Extended (PDSE) data sets may also be accessed with BPAM. Unlike QSAM, BPAM permits multiple PDS members to be processed with just one OPEN/CLOSE sequence. Because of this, and because BPAM has functions for managing PDS/PDSE directories, it is often the preferred method for processing partitioned data sets.

BPAM Basics Record Formats
Programming for BPAM Allocating Data Sets
Opening and Closing Data Sets Reading, Writing, and Updating Records
Sharing PDSE Members Sharing PDS Members
BPAM Return Codes Service Descriptions

BPAM Basics

Using BPAM you can access a partitioned data set (PDS) or a partitioned data set extended (PDSE). A partitioned data set is similar to a physical sequential data set, however, the data is partitioned into discrete segments called members. A portion of a PDS's space is reserved for a directory containing information that is used by the operating system to locate a member's storage. A member's location is recorded in TTR format, where "TT" is 2 bytes identifying the track and "R" is a single byte identifying the block.

When a member is added to a PDS, an entry for the member is inserted into the directory. Entries in the directory are ordered by member name in ascending sequence. Only one user at a time can add a member to a PDS.

A member's records can be updated in place. However, the only way to extend a member is to completely rewrite it (i.e., DISP=MOD will not cause records to be added to the end of a member as it does with sequential files). The same holds true for selective deletion of records: you must re-write the member while excluding those records that you no longer want.

When a member is deleted from a partitioned data set, its entry is removed from the directory. However, the space previously occupied by the member's records cannot be reused until the data set is "compressed" by a utility such as IEBCOPY.

PDSE data sets, from a usage standpoint, are identical to partitioned data sets, but with additional capabilities. Most importantly, PDSEs permit multiple users to write to a data set simultaneously (though each user must be writing a different member). PDSEs also reuse storage that is freed by member deletions, and thus, do not require periodic compression.

When using BPAM to read a partitioned data set member or members, you must allocate the data set without specifying a member name. A special function, FINDM, is used to position to a member prior to reading it. When creating a new member, the STOWM function is used to create a directory entry after all of the member's records have been written.

Partitioned and partitioned extended data sets can reside only on DASD. In addition, PDSEs must be managed by DFSMS.

Record Formats

The records in partitioned data sets can take several forms:

The records can all be same length or they can be varying lengths.
The records can be grouped into blocks. The system uses the blocks when performing physical input/output operations to improve performance. If the records are not blocked, each physical record contains exactly one logical record.
If desired, printer control characters can be placed in the first byte of each record.

The REXXTOOLS BPAM functions support the following formats (RECFMs):

F FA FB FBA FBM FM
V VA VB VBA VBM VM
U UA UM
The individual letters of RECFM indicate the following:

F: Fixed length records.
V: Variable length records.
U: Undefined length records. From a logical record perspective, U format is similar to variable length records, but the records cannot be blocked.
B: Blocked records.
A: ANSI printer control characters are in the first byte of each record.
M: Machine printer control characters are in the first byte of each record.

For information regarding the selection of an appropriate data set organization and record format refer to the IBM publication Using Data Sets for your system's level of DFP or DFSMS.

Note: PDS and PDSE data sets cannot be defined with RECFM=FS or RECFM=FBS (spanned records).

Programming For BPAM

The basic flow of a program that processes a data set using BPAM is as follows:

Allocate the file. A Data Definition Name (DDNAME) is associated with the data set you wish to process. Do not specify a member name for the data set. Use a status of NEW or OLD if you are creating or updating members. Use a status of SHR if you are reading members.
Note: You can concatenate multiple partitioned data sets to a single ddname for reading only. When searching for a member using FINDM, the first occurrence of the member name in the concatenation of partitioned data sets is the one that will be located.
Open the file for processing using the OPEN function. The access method makes a logical connection between your program and the data set to be accessed. If you are reading the file use the INPUT option on OPEN. If you are writing the file, use the OUTPUT option. If you are updating records in place, use the UPDATE option.
If you are reading or updating a member, use the FINDM function to position to the member, and optionally, to retrieve the member's directory entry. If you want to process all of the members in a PDS, use the REXXTOOLS LISTM function to obtain a list of member names.
Read a record from the file using GET or write a record to the file using PUT. When reading, the GET function returns the record's contents. You can assign the record to a REXX variable (or variables if you use PARSE VALUE). If the data set is blocked, the access method handles deblocking. Your program processes logical records only (not physical records). A return code of 8 indicates that there are no more records to be read.
When writing records, the PUT function accepts the record to be written as an argument. PUT handles any blocking that may be required, and writes physical blocks to the file as required.
If you are reading records, determine what processing applies to the record (if any) and perform the processing. If the data set was opened for update, the record can be replaced. Record deletion is not possible. To delete a record, or to add new records, the entire PDS member must be rewritten. For information regarding the processing of record fields, please see "Working with Record Fields".
If appropriate, return to step 4 to process other records.
If you are writing or re-writing a member, use the STOWM function to update the directory with the member name and its location (the operating system determines the location). You also may use STOWM to store user information in the directory entry. If any physical writes are pending when you issue STOWM, these are completed prior to updating the directory.
If appropriate, return to step 3 to process other members.
Close the file using the CLOSE function. Any pending physical writes are completed, and the logical connection between the program and the data set is terminated.
Free the file. The DDNAME is disassociated with the data set.

Allocating Data Sets

The JCL Data Definition (DD) statement may be used to allocate the data set. If dynamic allocation is required, the REXXTOOLS ALLOCATE and FREE commands can be used.

Opening and Closing Data Sets

To associate your program with the data set you want to process, you use the OPEN function. The CLOSE function performs the opposite action by disassociating your program with the data set.

Shadow REXXTOOLS maintains information about open BPAM files in a data structure associated with the task under which the OPEN function was executed. The information is maintained by ddname. Files remain registered with REXXTOOLS and open until they are explicitly closed, or until the task that opened them terminates.

All REXX programs under a MVS task share the same REXXTOOLS BPAM data structures. Thus if program A calls program B (REXX CALL or function call), any ddname that is opened by either program A or program B is known to the other. File sharing also extends to directly subtasked REXX programs. As a consequence, if Program A attaches program B, the files opened by program A are known by program B. However, files opened by program B will not be known to A when control is returned. This is because task termination will close all files opened by program B.

Notes:

A PDS supports only one writer at a time. If you need to support multiple simultaneous writers, you must use a PDSE.

OPEN Options

When opening a data set using BPAM, you may specify two types of options which affect processing. These are:

Type of Access

The REXXTOOLS BPAM interface supports 3 access options:

INPUT: indicates that you will be reading records from the data set. Data sets that are open for input can be accessed with the FINDM and GET functions only. The data set can be allocated with a status of OLD or SHR (preferred).
OUTPUT: indicates that you will writing records to the data set. Data sets that are open for output can be accessed with the PUT and STOWM functions only. The data set should be allocated DISP=OLD.
UPDATE: indicates that you will be reading the data set, and that you may be re-writing none, some, or all records. A record must be read before it can be re-written. When open for update, a data set can be accessed with the FINDM, GET, PUT, and STOWM functions. The data set should be allocated DISP=OLD.

DCB Parameters

Record format (RECFM), logical record length (LRECL), block size (BLKSIZE), and number of buffers (BUFNO) can be specified on OPEN. However, with rare exception, you are much better off letting the operating system derive these values for you. For new data sets, the information can be derived from allocation information that is kept in memory. For existing data sets, the information is taken from the Volume Table of Contents (VTOC).

Reading, Writing, and Updating Records

The Shadow REXXTOOLS BPAM functions for access to records and directory entries are:

FINDM: The FINDM function is used to locate a member's position on DASD for reading or updating records. Optionally, the member's directory entry user field may be retrieved.
GET: The GET function is used to retrieve records sequentially. GET returns a logical record as its value (or a null string if an error was encountered).
PUT: The PUT function is used to sequentially write or re-write records. PUT accepts a record as one of its arguments. If the PUT follows a GET where the file has been opened for update, the record that was retrieved by the GET will be replaced. In all other cases, PUT is interpreted as a request to add a new record.
STOWM: The STOWM function is used to add, replace, or remove a member's directory entry. STOWM also can be used to add, replace, or remove a member alias. Optionally, the member's directory entry user field can be created or updated.

Input Processing

Using the INPUT option of OPEN, you can read some or all of a member's records. In the following example, a PDS member is read:

/* REXX */
address rexxtool
"alloc fi(indd) da(user.data) shr"
if open('bpam','indd','input') <> 0 then do
  say 'OPEN failed with RC='rc 'REASON='reason
  exit 8
end
if findm('indd','timecard') <> 0 then do
  say 'TIMECARD member not found.'
  exit 8
end
timerec = get('indd')
do while rc = 0
  parse var timerec lname +10 fname +10 timein +8 timeout +8
  timerec = get('indd')
end
call close 'indd'
"free fi(indd)"

Output Processing

Using the OUTPUT option of OPEN, you can create one or more members. In the following example, multiple members may be created:

/* REXX */
address rexxtool
"alloc fi(outdd) da(user.data) new sp(1 1) track",
  "dsorg(po) recfm(v b) lrecl(255) dir(10) unit(sysda)"
call open 'bpam', 'outdd', 'output'
say 'Enter member name or ENTER to quit.'; parse pull member
do while member <> ''
  say 'Enter records. Terminate with a null line.'
  parse pull record
  do while record <> ''
    call put 'outdd', record
    parse pull record
  end
  call stowm 'outdd', member
  say 'Enter member name or ENTER to quit.'; parse pull member
end
call close 'outdd'
"free fi(outdd)"

Update Processing

Using the UPDATE option of OPEN, you can replace some or all of a member's records. You may not, however, add new records or remove existing records. The interface ensures that replacement records are the same size as the original record. If the replacement record is smaller than the original, the record is padded on the right with blanks. If it is larger than the original record, the replacement record is truncated on the right. No error indication is given in either case.

In the following example, all of a PDS's members are updated. A 2-byte packed decimal date field in YY format is converted to a 2-byte binary date that contains the century.

/* REXX */
dsn = "'crude.royalty.data'"
parse value listm(dsn) with rc mc mlist
address rexxtool "alloc fi(iodd) da("dsn") old"
call open 'bpam', 'iodd', 'update'
do i = 1 to mc
  parse var mlist member mlist
  call findm 'iodd', member
  parse value get('iodd') with type +1 1 record
  do while rc = 0
    if type = 7 then do
      parse var record first +25 date +2 last
      call put 'iodd', first||d2c(p2d(date)+1900,2)||last
    end
    parse value get('iodd') with type +1 1 record
  end
end
call close 'iodd'
address rexxtool "free fi(iodd)"
exit

Notes:

LISTM is a REXXTOOLS function for retrieving member names. P2D is a REXXTOOLS function for converting packed decimal data to REXX decimal.
For more information on record parsing refer to "Working with Record Fields".

Extend Processing

Unlike sequential files, partitioned data sets can not be extended using DISP=MOD. To add records to a member (or remove records), you must completely re-write the member and use STOWM to update the directory, as the following example demonstrates:

/* REXX */
 
/* First, allocate and open the file 2 times,
   once for input and once for output */
"alloc fi(indd)  da(user.pds) shr reu"
"alloc fi(outdd) da(user.pds) old reu"
call open 'bpam', 'indd',  'input'
call open 'bpam', 'outdd', 'output'
 
/* Next, copy the existing records */
call findm 'indd', 'member1'
record = get('indd')
do while rc = 0
  call put 'outdd', record
  record = get('indd')
end
 
/* Now, add some new records */
do i = 1 to 10
  call put 'outdd', 'New record' i
end
 
/* Last, replace the directory entry */
call stowm 'outdd', 'member1', 'r'
call close 'indd'
call close 'outdd'
"free fi(indd outdd)"
exit

Note: Using this method you can add records anywhere in the member, not just on the end. You can also omit records on the copy step, thereby deleting them.

Sharing PDSE Members

A PDSE permits multiple tasks to write to and read from the data set at the same time. To use PDSE sharing, ensure that all accessors allocate the PDSE using DISP=SHR (If a component of your application requires exclusive use of the entire data set, you can use DISP=OLD to lock-out other users).

A "connection" to a member is established whenever you use the FINDM function. The length of the connection is determined as follows:

If the M function is used, the FINDM function executes a FIND by name. This establishes a connection that lasts until a subsequent FINDM (for the same ddname) is executed or the data set is closed.
If the T function is used, the FINDM function executes a FIND by TTR. The connection established lasts until the the data set is closed.
If the S or U functions are used, the FINDM function executes a BLDL for the member, followed by a FIND by name. Because the BLDL is issued without the NOCONNECT option, a connection is established that lasts until the data set is closed.

Once a program has connected to a member, a temporary version of that member is created and remains in existence until the connection is severed, even though another user may re-write or delete the member.

For a detailed description of PDSE member sharing refer to the Using Data Sets publication for your system's level of DFP or SMS.

Sharing PDS Members

If your application requires many users to access a partitioned data set concurrently, you should use a PDSE instead of a PDS. As discussed in the previous section, PDSEs provide built-in support for sharing members. If for some reason your application cannot use a PDSE, it is possible to share PDS members, although a little more programming is required.

One way of protecting a PDS while maintaining a relatively high degree of concurrency is to use the REXXTOOLS ENQ and DEQ functions to implement an access protocol. A simple, but effective, protocol follows:

All parties agree to allocate the PDS using DISP=SHR. Certain tasks that require exclusive control of the data set (for example, data set compression jobs or 3rd shift batch update jobs) must use DISP=OLD. Any task that uses DISP=OLD does not need to follow the rest of the protocol.
Prior to reading a member, a shared enqueue on the data set (not just the member) must be obtained. Any queue name (qname) can be chosen, but all parties must use this name when obtaining the enqueue. The rname should be the data set name without a member name. When the member has been read, the enqueue is released. Ideally, the ENQ should precede the FINDM for the member, and the DEQ should follow the last GET.
Note: If you are using the REXXTOOLS LISTM function to read the directory, its operation also must be shielded with a shared enqueue.
Prior to writing a member, an exclusive enqueue on the data set must be obtained. When the member has been written, the enqueue is released. Again, the qnames and rnames must be used consistantly by all parties. The ENQ should precede the first PUT (or the first GET if the member is being updated) and the DEQ should follow the call to STOWM, if applicable.
In all cases, the reading and writing of members should proceed expeditiously to ensure a high degree of concurrency.

Note that whenever an application has the capacity to change a PDS directory (i.e., the application contains STOWM function calls) or opens a PDS for output, an enqueue for the entire data set must cover every input/output operation. If a data set-wide enqueue is not obtained for all I/O operations, multiple tasks can write to the data set's free space or to the directory at the same time, irreparably damaging your data. In addition, program abends can occur when a task attempts to read a directory while it is being written to by another task.

If your application is structured so that new members are not created and existing members are not extended (that is, the PDS is pre-formatted, and the only type of write operation performed is an update-in-place), you can increase concurrency by narrowing the scope of your enqueues to include specific member names. However, if your application uses STOWM to update the user field of directory entries, you still must use data set-wide enqueues.

If your application requires a "browsing" function, where information from one or more members is displayed to a user for the purpose of subsequent update, the protocol must be modified slightly in order to maintain a high degree of concurrency while protecting the data set:

As before, all parties use DISP=SHR.
As before, all read operations are bracketed with a shared enqueue.
As before, all write operations are bracketed with an exclusive enqueue.
No enqueue is held while the data is being browsed. This prevents, for example, other users from being locked-out while a user who is browsing is away from his terminal.
Prior to applying modifications to browsed data, the application obtains an exclusive enqueue and re-reads the data. A comparison is made of the re-read data and an un-modified copy of the data as it was originally read. If the comparison indicates that the data has not been modified by another user during the period in which it was browsed, the changes are applied, and the exclusive enqueue is released. Otherwise -- if the comparison indicates that another user has modified the data -- the enqueue is released and the re-read data is displayed, along with a notification that the user's changes were not applied.

Again, the scope of the enqueues used depends, entirely, on whether or not the directory can be updated by the application. If the directory can be updated by the application, data set-wide enqueues must be used. If not, member-specific enqueues can be used.

BPAM Return Codes

Upon completion, all Shadow REXXTOOLS BPAM functions provide a return code and a reason code. The return code is placed in the standard RC variable, while the reason code is placed in a special variable named REASON.

In addition, except for the GET function which returns a record, all BPAM functions return the return code as their value. For example, the following code will display the return code 2 times, once as OPEN's returned value, and once as the value of the RC variable

say "RC="open('bpam','indd','input') "RC="rc "REASON="reason

Unless otherwise noted, the values for RC and REASON are taken directly from the underlying BPAM macro return and reason codes. In all cases, a return code of zero indicates success.

In the case of GET and PUT, the underlying BPAM macros (READ, WRITE, and CHECK) do not produce return codes. The REXXTOOLS GET function returns RC=8 when end-of-file is reached. All other non-zero GET and PUT return codes are either ABEND codes (which are accompanied by an "IEC" message) or synchronous error exit codes (which are accompanied by a "RXT" message).

Very Important: The complete list of BPAM return and reason codes can be found in the IBM publication Macro Instructions for Data Sets for your system's level of DFP or DFSMS. ABEND codes and "IEC" messages are documented in one or more MVS messages and code publications.

Note: The return and reason codes (including ABEND codes) produced by the BPAM functions are all decimal (not hexadecimal) values. Service Descriptions The sections that follow describe the syntax and operation of the BPAM-related functions.