File Handler Configuration

Sorting Files

Chapter 7: File Handler API

7.1 Overview

You can make explicit calls to the File Handler from within your program using the following syntax:

call "EXTFH" using opcode fcd

where the parameters are:

You must follow these steps before the first call:

1. Allocate data areas for the File Control Description (FCD), the Record Area, the Filename Area and the Key Definition Block (if the file is indexed).
   
   
2. Initialize all the data areas to binary zeros to ensure that the File Handler does not receive invalid values.
   
   
3. Set the pointers in the FCD to point to the:
   
   
   • Record Area
     
     
   • Filename Area
     
     
   • Key Definition Block (indexed files only)

You must then follow these steps for each File Handler operation:

1. Fill in the appropriate fields in the FCD for the selected operation code.
   
   
2. Call the File Handler.
   
   
3. Determine the success of the file I/O operation by checking the file status.
   
   
4. Process the FCD fields and/or any data in the Record Area.

7.2 Data Structures

The File Handler uses four data structures during file I/O operations:

• File Control Description

  The FCD is a data area containing information about the file in use.

• Record Area

  The record area is a data area into which records are read, and from which records are written.

• Filename Area

  The Filename Area is a data area which contains the name of the file in use, as recognized by the operating system.

• Key Definition Block

  The Key Definition Block is used by the File Handler to hold index key information during operations on indexed files.

7.2.1 File Control Description (FCD)

To use the File Handler your program must set up an FCD, complete the appropriate fields (these vary according to the operation code) and then call the File Handler. The File Handler returns information to your program by writing it to the appropriate fields in the FCD.

All unused or reserved areas of the FCD must be set to binary zeros.

The FCD used on the call to open a file must be the one used on all subsequent accesses to that file. You can perform multiple open operations on a file but you must use a different FCD for each open.

The FCD includes pointers (that is, USAGE POINTER data items) which point to the record area, filename area and key definition block.

WRITE AFTER ADVANCING line-count LINES

7.2.2 Record Area

The record area is a data area into which records are read, and from which records are written.

The size of the record area must be four bytes larger than the largest record in the file.

The following example shows how the FCD is set up to point to the record area:

01 RECORD-AREA      PIC X(85).
...
   SET FCD-RECORD-ADDRESS TO ADDRESS OF RECORD-AREA
   ...

7.2.3 Filename Area

The filename area is a data area which contains the name of the file in use, as recognized by the operating system.

It can contain drive and/or path information as well as the actual name of the file. The name must be terminated by a space.

This filename area must be filled before the first operation on the file.

The following example shows how the FCD is set up to point to the filename area:

01 FILENAME-AREA    PIC X(65) VALUE "master.dat".
...
   MOVE 65 TO FCD-NAME-LENGTH
   SET FCD-FILENAME-ADDRESS TO ADDRESS OF FILENAME-AREA
   ...

7.2.4 Key Definition Block

The Key Definition Block is used to pass index key information to the File Handler when opening an indexed file. It consists of three data areas:

• Global Information Area
  
  
• Key Definition Area
  
  
• Component Definition Area

7.2.4.1 Global Information Area

The size of the Key Definition Area and the number of keys that are in the file are held in the Global Information Area.

All unused or reserved areas must be initialized to binary zeros.

7.2.4.2 Key Definition Area

The Key Definition Area describes the keys used in the indexed file.

The Key Definition Area follows the Global Information Area and contains one key definition for each key in the file.

You must define all keys before their components.

The order in which keys are defined is important as the ordinal position of the key is used to identify it. For example, if you have an indexed file with a prime key and two alternate keys, the Key Definition Area contains three key definitions: the prime key is key 0, the first alternate is key 1, and the second alternate is key 2.

The layout of the Key Definition Area is shown below. All unused or reserved areas must be initialized to binary zeros.

7.2.4.3 Component Definition Area

The Component Definition Area follows the Key Definition Area.

It contains a definition for each key component. Each key consists of one component, unless defined as a split key, when each component of the key requires its own component definition.

The component definitions define the position and length of the key component.

For numeric keys, you can use the Component Definition Area to specify the numeric type. You can then use the IXNUMKEY Compiler directive to specify that keys are ordered according to numeric type (the default situation is that keys are ordered according to their alphanumeric value).

The layout of the Component Definition Area is shown below. All unused or reserved areas must be initialized to binary zeros.

7.3 Accessing the FCD

When file I/O operations are performed by your program, using ordinary COBOL syntax, a File Control Description (FCD) is created automatically for each file. You can access this automatically created FCD by compiling your program with the FCDREG Compiler directive. This enables you to read or change the FCD or to use it to make explicit calls to the File Handler.

To access the FCD, you must set up an FCD definition in the Linkage Section of your program. An example FCD definition is supplied in the file, XFHFCD.CPY (which can be located using Infomgr). This definition can be mapped onto the FCD you want to read or alter using the following SET statement in your program:

set address of fcd to address of fh--fcd of file

where the parameters are:

Following this SET operation, you can access the file's FCD by accessing the data items in the FCD which you set up in your Linkage Section.

Similarly, you can access the Key Definition Block as follows:

set address of kdb to address of fh--keydef of file

An example of accessing the FCD is given below.

Example of Accessing the FCD

The following program turns on key compression in a file by setting the appropriate bits in the file's Key Definition Block.

$set fcdreg
     select masterfile 
         assign to ... 
         organization is indexed 
         record key is rec-key 
         alternate key is m-alt-key-1 with duplicates 
         alternate key is m-alt-key-2 
         alternate key is m-alt-key-3 with duplicates. 
     select indexfile 
         assign to ... 
         organization is indexed 
         record key is rec-key 
         alternate key is t-alt-key-1 with duplicates. 
   ... 
 working-storage section. 
   ... 
 linkage section. 
 01 key-def-block. 
    03 filler pic x(6). 
    03 key-count pic 9(4) comp-x. 
    03 filler pic x(6). 
    03 key-def occurs 1 to 4 times depending on key-count. 
       05 filler pic x(5). 
       05 key-compression pic 9(2) comp-x. 
       05 filler pic x(10). 
  ... 
 procedure division. 
     set address of key-def-block 
                      to address of fh--keydef of masterfile
     move 6 to key-compression (1) 
     move 7 to key-compression (2) 
     move 6 to key-compression (3) 
     move 7 to key-compression (4) 
     open I-O masterfile 
  ... 
     set address of key-def-block 
                       to address of fh--keydef of indexfile
     move 4 to key-compression (1) 
     move 1 to key-compression (2) 
     open input indexfile

7.4 Operation Codes

Each file operation which can be performed by the File Handler is identified by a two-byte operation code.

There are two types of operation code - standard and special.

• Standard operation codes

  Standard operation codes contain x"FA" in the most significant byte. The least significant byte indicates the specific operation.

• Special operation codes

  Special operation codes contain x"00" in the most significant byte. The least significant byte indicates the specific operation.

7.4.1 Standard Operation Codes

Standard operation codes contain x"FA" in the most significant byte. The least significant byte indicates the specific operation.

• Close - terminates the processing of files.
  
  
• Commit - releases all record locks in all files held by a run unit.
  
  
• Delete - logically removes a record from a file.
  
  
• Delete file - physically removes the specified file from the physical devices on which it resides.
  
  
• Open - initiates the processing of files. It also performs checking and/or writing of labels and other I/O operations.
  
  
• Read - for sequential access, READ makes available the next or previous logical record from a file. For random access, READ makes available a specified record from a disk file.
  
  
• Rollback - releases all record locks in all files held by a run unit.
  
  
• Start - provides a basis for logical positioning in a relative or indexed file, for subsequent retrieval of records.
  
  
• Step - steps to the next physical record.
  
  
• Unlock - releases all record locks held by a run unit on a named file.
  
  
• Write/rewrite - releases a logical record for an output or input-output file.

Not all operation codes are appropriate with all file organizations. Each operation code description includes a list of the file organizations for which it is valid, using the following key:

L    Line sequential
S    Record sequential (including vS)
R    Relative (including vR)
I    Indexed

vS    Variable format sequential only

vR    Variable format relative only

Close

Terminates the processing of files and reels or units with optional rewind and locking where appropriate. For record sequential files, the terms "reel" and "unit" are interchangeable; the treatment of sequential disk files is logically equivalent to the treatment of a file on tape or similar sequential media.

Input Fields

None

Output Fields

Commit

Releases all record locks in all files held by this run unit.

Input Fields

Always requires FCD used by the file.

Output Fields

None

Delete

Logically removes a record from a disk file. It can be specified only for files with relative or indexed organization.

Input Fields

Output Fields

Delete File

Physically removes the specified file from the physical devices on which it resides.

Input Fields

Filename area

Output Fields

Open

Initiates the processing of files. It also performs checking and/or writing of labels and other I/O operations.

Input Fields

Filename area

Key Definition Block (I)

Output Fields

Opening a File More Than Once

You can assign several FCDs to the same physical file and have them all open at the same time. The operating system counts it as just one open file. The physical file is not closed until every logical file assigned to it has been closed.

Read

For sequential access, READ makes available the next or previous logical record from a file. For random access, READ makes available a specified record from a disk file.

Input Fields

Output Fields

Rollback

Releases all record locks in all files held by this run unit.

Input Fields

None

Output Fields

None

Start

Provides a basis for logical positioning in a relative or indexed file, for subsequent retrieval of records.

Input Fields

Output Fields

Step

Steps to the next physical record. This operation enables very fast access to a record.

Input Fields

Output Fields

Comments

STEP operations are a method of sequentially retrieving records in a relative or indexed file without having to read via a key or relative record number. Generally, this is a faster method of accessing data in the file.

A STEP NEXT operation returns the record that is physically stored immediately after the one that was last retrieved in the file (either by a STEP or READ operation).

If you are using relative byte addressing and you have specified that the current record pointer should be updated to the record you have just accessed, STEP operations are relative to this new position.

The relative byte address is returned on every STEP operation, so if you need to update the current record pointer to the record just retrieved, you can use the READ (direct) operation using the address returned.

Note: The current record pointer is not affected by STEP operations.

Unlock

Releases all record locks held by the run unit on a named file.

Input Fields

None

Output Fields

Write

Writes a new record to a file.

For sequential files, it can also be used for vertical positioning of lines in a logical page.

The REWRITE operation replaces an existing record.

Input Fields

Output Fields

7.4.2 Special Operation Codes

Special operation codes contain x"00" in the most significant byte. The least significant byte indicates the specific operation.

• File information - returns general file information for all file organizations and, for indexed files, information on the file keys in the form of the Key Definition Block.
  
  
• Create index file - creates a new .idx file, containing only header information for the file to allow new indexes to be added.
  
  
• Get next record - gets the next physical record from the data file part of an indexed file.
  
  
• Add key value - adds a key value to the index file part of an indexed file.
  
  
• Re-index file - re-indexes the file using a supplied key definition block.
  
  
• Flush file - flushes all data for a specific file to disk.
  
  
• Unlock record - unlocks a specific record in a file.
  
  

File Information

Returns general file information for all file organizations and, for indexed files, information on the file keys in the form of the Key Definition Block.

Input Fields

Output Fields

Key Definition Block (I)

It is your responsibility to ensure that you have enough space allocated for the Key Definition Block to hold all the information returned. Failure to allocate enough space causes corruption of some data areas.

For variable-length sequential files and variable-length relative files, no key definition information exists.

If the organization byte of the FCD is set to x"FF", this tries to determine the file type and return with the relevant information set. This mode of operation is not recommended if fixed-length sequential files are involved as it might be impossible to determine the difference between the first record of a fixed-length sequential file and the header of a variable-length sequential file. When the file type is determined as incompatible, the error code 9/161 is returned.

Create Index File

Creates a new .idx file, containing only header information for the file to allow new indexes to be added.

Input Fields

As OPEN.

Output Fields

As OPEN

Get Next Record

Gets the next physical record from the data file part of an indexed file.

Input Fields

Output Fields

Add Key Value

Adds a key value to the index file part of an indexed file.

Input Fields

Output Fields

Re-index File

Re-indexes the file using a supplied Key Definition Block.

Input Fields

Output Fields

None

Flush File

Flushes all data for a specific file to disk.

Input Fields

None

Output Fields

None

Unlock Record

Unlocks a specific record in a file.

Input Fields

Output Fields

None

7.5 Relative Byte Addressing

The relative byte address of a record is placed in fcd-reladdr-offset in the FCD (or fcd-reladdr-big if bit 4 of fcd-config-flags is set) by all File Handler operations that involve specific records. To use the relative byte address of a record, simply save the contents of this field after a READ operation.

Using the relative byte address is a fast method of accessing records, but has limitations of which you should be aware:

• A relative byte address operation does not affect the current record pointer. Therefore, a READ (sequential) after a relative byte address operation returns the record after the last one accessed using normal access methods. It does not return the record after the one you accessed using the relative byte address. It is possible, however, to update the current record pointer to point to the record you accessed using relative byte addressing by setting bit 5 of fcd-config-flags in the FCD before calling the File Handler to perform the relative byte address operation.
  
  
• Relative byte address operations update the index when a record is rewritten or deleted.
  
  
• If a record has been deleted since the relative byte address for it was obtained, a read or write using its relative byte address usually fails. However, it is possible for it to have been replaced by a different record and for this to go undetected.

Record locking is supported with relative byte address operations.

Once you have obtained the relative byte address of a record, you can use it to perform the following operations:

• Read a record
  
  
• Rewrite a record
  
  
• Delete a record

7.5.1 Read a Record

You can read a specific record from a file in two ways using the relative byte address:

• Put the address in fcd-reladdr-offset or fcd-reladdr-big in the FCD and set bit 6 of fcd-config-flags in the FCD. If you want to update the current record pointer, set bit 5 of fcd-config-flags .
  
  
• Perform a READ (random) WITH NO LOCK, READ (random) WITH LOCK, READ (random) WITH KEPT LOCK or READ (random) operation on the file.

The READ (direct) operations work in the same way, but do not require the bits in the FCD to be set. The READ (direct) operations always return the record at the address given in the relative byte address field of the FCD and update the current record pointer.

Both of the above methods provide a way of switching the current key-of-reference to a different key. For example, if READ (sequential) operations are performed via the prime key, you can start reading via the first alternate key from the current record using the following steps:

1. Read the next record in the file (the address of this record is in the FCD relative byte address field).
   
   
2. Put the new key-of-reference in fcd-key-id in the FCD.
   
   
3. Perform a READ (direct) operation to return the record at this address. The key-of-reference is changed to the new one.
   
   
4. Read the next record in the file. This is the next record in the new key-of-reference index.

7.5.2 Rewrite a Record

You can rewrite a record to a specific address by putting that address in fcd-reladdr-offset or fcd-reladdr-big in the FCD and setting bit 6 of fcd-config-flags in the FCD.

If you want to update the current record pointer, set bit 5 of fcd-config-flags in the FCD.

You then perform a REWRITE operation.

7.5.3 Delete a Record

You can delete a record at a specific address by putting that address in fcd-reladdr-offset or fcd-reladdr-big in the FCD and setting bit 6 of fcd-config-flags in the FCD.

You then perform a DELETE operation.

7.6 Creating Your Own File Handler

You can create your own, customised, file handler and use this is place of the File Handler supplied with Server Express. Alternatively, you can create file handlers that handle files of a specific type, such as relative files.

7.6.1 Using Your Own File Handler

To force programs to use your file handler for processing COBOL I/O syntax, use the CALLFH Compiler directive. For example, if you have a file handler called USERFH, compile the program with the directive:

CALLFH"USERFH"

This causes all COBOL I/O operations to be compiled into calls to USERFH.

Note: If you use CBL_EXIT_PROC, you must set the priority to 200, so that when the application calling your file handler terminates, your file handler shuts down properly.

7.6.2 Using Your Own File-type Specific File Handler

You can create your own file handler for a specific file type, and use this in place of the default file handler.

The default file handlers for the various types of file organization and record format are:

The file handlers you link to your application instead of any of the default file handlers shown here must conform to the format of the call interface described in this chapter.

To link your own file handlers to your application, you must include the -m option on the cob command line. See the chapter Descriptions of cob Flags in your Server Express User's Guide for details of this option.

For example, to redirect all fixed length sequential file handling to your own file handler (myseqfh) you should use the following command line:

cob -x -msqfile=myseqfh myapp.cbl myseqfh.o

7.7 Creating a New Index File

You can use special operation codes to recreate an index file from the data file part of an existing indexed file as follows:

1. Create the new index file using Create Index File (special operation code x"07").
   
   
2. Read each record from the data file using Get Next Record (special operation code x"08").
   
   
3. For each key value within the record, add the key value to the index file using Add Key Value (special operation code x"09").

The following example shows you how to create a new index file from an existing data file:

 78 close-file                  value x"fa80"
 78 open-new-index              value x"0007".
 78 get-next-rec                value x"0008".
 78 add-key-value               value x"0009".
 ...
     move open-new-index to fh-opcode
     perform extfh-op
     move get-next-rec to fh-opcode
     perform extfh-op
     perform until fcd-status (1:1) not = "0"
         perform varying fcd-key-id from 0 by 1
                   until fcd-key-id = key-count 
                      or fcd-status (1:1) not = "0"  
             move add-key-value to fh-opcode
             perform extfh-op
         end-perform
         move get-next-rec to fh-opcode
         perform extfh-op
     end-perform
     move close-file to fh-opcode
     perform extfh-op
     ...
 extfh-op.
     call "EXTFH" using fh-opcode, fcd
     if fcd-status of fcd (1:1) = "1"
         move 1 to return-code
     end-if.

7.8 Compression Routines

The compression routines that the File Handler uses to compress data are standalone modules. This means that you can:

• Use them in your own applications
  
  
• Make the File Handler use your own compression routines instead

There can be up to 127 Micro Focus compression routines, and up to 127 user-supplied compression routines.

7.8.1 Micro Focus Compression Routines

Micro Focus compression routines are stored in modules called CBLDCnnn, where nnn is within the range 001 to 127.

To use a Micro Focus compression routine, set fcd-data compress in the FCD to a value of 001 to 127, as appropriate.

7.8.1.1 CBLDC001

The Micro Focus compression routine, CBLDC001, uses a form of run-length encoding. This is a method of compression that detects strings (runs) of the same character and reduces them to an identifier, a count and one occurrence of the character.

Note: This routine is not effective for use with files that contain significant occurrences of double-byte characters, including double-byte spaces, as these are not compressed.

CBLDC001 puts special emphasis on runs of spaces, binary zeros and character zeros (that can be reduced to a single character) and printable characters (that are reduced to two characters consisting of a count followed by the repeated character).

In the compressed file, bytes have the following meanings (hex values shown):

7.8.1.2 CBLDC003

Like CBLDC001, this routine uses run length encoding, but detects strings (runs) of single- or double-byte characters. This routine is therefore suitable for DBCS characters, but can also be used in place of CBLDC001.

The format of the compression is two header bytes followed by one or more characters. The bits in the header bytes indicate:

The length of the character string depends on the header bits:

7.8.1.3 Calling a Micro Focus Compression Routine

For data file compression the File Handler calls the compression routine that you specify in the DATACOMPRESS Compiler directive.

If you want to use the compression routines for compressing your own data, you can call them directly from your program:

        call "CBLDCnnn" using input-buffer,
			      input-buffer-size,
			      output-buffer,
			      output-buffer-size,
			      compression-type

where the parameters are:

The RETURN-CODE special register indicates whether the operation succeeded or not. Compression or decompression fails only if the output buffer is too small to accept the results.

7.8.2 User-supplied Compression Routines

User-supplied compression routines must be stored in modules called USRDnnn where nnn is within the range 128 through 255.

When creating compression routines, you should bear the following in mind:

• Your compression routines must not make any calls to the File Handler, as this would result in an endless loop.
  
  
• Once you have enabled data compression for a file, you must always subsequently specify the same type of compression for that file. If you do not, you will receive a run-time system error when you open the file.
  
  
• Data compression has no effect on files other than indexed or record sequential files.

File Handler Configuration

Sorting Files