PreviousSharing Files Callable File Handler and Callable Sort APIsNext

Chapter 6: File Handler Configuration

Certain aspects of the File Handler's behaviour can be configured. You can alter these aspects by entering values in the File Handler configuration file.

6.1 Configuration File

The default filename for the File Handler configuration file is extfh.cfg but you can use a different filename by setting the EXTFH environment variable. For example:

set EXTFH=c:\mydir\test.cfg

sets the filename to c:\mydir\test.cfg.

The File Handler configuration file enables you to alter File Handler parameters for individual files, or for all files. Settings which apply to all files are listed under the tag:

[XFH-DEFAULT]

while settings which apply to an individual file are listed under the individual filename, for example:

[TEST.DAT]

The settings for an individual file override the global settings.

6.2 Configuration Options

The following table lists the parameters that you can set via the File Handler configuration file.

Parameter
Meaning
COMMITFLUSH In the context of a program that is not using FILESHARE, specifies whether the COMMIT and ROLLBACK statements cause all file updates to be flushed to disk, as well as releasing record locks.
CONVERTSTATUS Specifies the name of a program that is called, after the File Handler has finished processing, to map returned status values for emulation purposes.
DATACOMPRESS Specifies whether data compression is in effect and the type of compression.
DATAFILE Maps the file name, passed to an OPEN statement, onto another name.
EXPANDPOSITIONING When using the WRITE AFTER POSITIONING statement (OS/VS COBOL compatibility), determines whether the written record includes carriage control information.
EXPANDTAB Determines whether to expand TAB characters, encountered in line sequential or line advancing files during a READ operation, into the equivalent number of spaces.
FASTREAD When reading an indexed file, specifies whether or not the File Handler performs extra checking to ensure data integrity.
FHREDIR Specifies whether files can be handled by a remote server using FILESHARE.
FILEMAXSIZE1 Specifies the size in bytes to use for holding offsets used in locking calls, allowing access to very large files.
FILEPOINTERSIZE For format 8 indexed files, specifies the length in bytes to use to store file pointers.
IDXDATBUF Determines the size of buffer used when accessing the data portion of a file with organization INDEXED.
IDXFORMAT Specifies the format to use when creating indexed files.
IDXNAMETYPE Specifies the format of the filename type for both the data file and, if present, the index file.
IGNORELOCK Specifies whether to ignore locks when reading files open for input.
INDEXCOUNT Specifies the number of index nodes to be cached for an indexed file.
INSERTNULL When performing WRITE or REWRITE operations on line sequential or line advancing files, determines whether to insert NULL (x"00") characters before non-printable characters. Also determines whether to strip out those NULL characters during READ operations.
INSERTTAB When performing WRITE or REWRITE operations on line sequential or line advancing files, determines whether to replace TAB characters with sequences of spaces.
KEYCHECK Specifies whether the File Handler checks that the key definition defined in your application matches that of the indexed file you are opening.
KEYCOMPRESS Specifies the type of key compression in use.
LOADONTOHEAP Specifies whether the file is to be loaded into memory at open time
LOCKTYPE Specifies the type of record locking in use.
MAINFRAMEPRINT For those files using WRITE AFTER ADVANCING or WRITE BEFORE ADVANCING, determines whether to use mainframe printer format (filetype(11)).
NAMEOPTIONS Specifies whether the name you pass to the OPEN statement can contain File Handler directives enclosed in square brackets. This parameter is provided for compatibility with older MERANT products - you should only use it if you require compatibility with Micro Focus Workbench V4.0.
NFSFILELOCK Enables applications to detect record locks and file locks on UNIX NFS file systems.
NODESIZE Specifies the size of the index nodes to use for an indexed file.
OPENINPUTSHARED Specifies whether files that you open for input, and for which you do not specify a LOCK MODE clause, can be shared by other users.
OSVSREWRITE Specifies whether to permit the WRITE statement for sequential files that are open for output. If so, these WRITE statements behave exactly like REWRITE statements.
READSEMA Specifies whether the system tries to gain a semaphore for shared files when I/O operations are performed that do not modify the file.
RELDATBUF Determines the size of buffer used when accessing a file with organization RELATIVE.
RETRYLOCK When an operation tries to access locked records, indicates whether you wish to retry the operation.
RETRYOPEN When an operation tries to access locked files, indicates whether you wish to retry the operation.
RETRYTIME Specifies whether the integer given in the RETRYLOCK or the RETRYOPEN parameter specifies the number of attempts or the number of seconds.
RUNITLOCKDETECT Specifies whether you can detect that a record has been locked by a separate OPEN statement issued from inside the same run unit.
SEQDATBUF Determines the size of buffer to use when accessing a file with organization SEQUENTIAL.
SKIPLOCK When a locked record is encountered during a sequential READ, specifies whether to advance the current record pointer.
SPACEFILL When performing READ operations on line sequential or line advancing files, determines whether to fill with spaces that part of the record area beyond the data that is read.
STRIPSPACE When performing WRITE or REWRITE operations on line sequential or line advancing files, determines whether to remove trailing space characters.
SUPPRESSADV For record sequential files, determines whether to ignore the ADVANCING phrase of the WRITE statement.
TRACE Specifies whether Xfhtrace is enabled.
TRACEFILENAME When the TRACE option is active, specifies the name of the trace file to produce.
WRITETHRU Determines whether to write file modifications out to disk immediately. Alternatively, the modifications may be buffered internally, either by the COBOL system or by the operating system, and flushed to disk at a later stage.

1 See the warning below.

For full details on valid settings and the default value of each of these options, see the Net Express online help. (Click Help Topics on the Help menu. Then, on the Contents tab, click Reference, File Handling, File Handler, Configuration File Parameters.)

The Net Express online help also contains an example configuration file (click Programming, File Handling, Examples, File Handler, Configuration file).


Warning:

The FILEMAXSIZE option specifies the number of bytes used internally to store file offsets. This also affects internal locking mechanisms. Programs sharing the same file all need to use the same FILEMAXSIZE settings to ensure semaphores and record locks are handled correctly, thereby avoiding potential file corruption in shared files.

The FILEMAXSIZE option defaults to 4, limiting file addressibility to 32 bit values which is compatible with earlier versions of Micro Focus systems. The other setting for the FILEMAXSIZE option is 8, which allows up to 64 bit values for file addressing on Windows NT platforms when accessing the NTFS file system, but changes the underlying record locking mechanism.

Do not set FILEMAXSIZE to 8 under any of the following circumstances:


6.3 Configurable Behaviour

Those aspects of the File Handler's behaviour which can be configured via the File Handler configuration file and require some further explanation than is given in the online help file, are detailed below.

6.3.1 Large Indexed Files

By default, the File Handler can handle files that are up to a maximum of 2 Gigabytes in size. However, if you need to create larger indexed files, you can set the IDXFORMAT parameter in the File Handler configuration file to 8 to create IDXFORMAT(8) files. These files comprise a single file (as opposed to the usual .idx and .dat files) and can be up to 32 Terabytes in size.

Many operating systems have a limit on file size (2 Gigabytes on Windows 95, for example). To enable you to create large files on these systems, a striping option is provided. Striping splits a large logical file into as many physical files as are required (up to a limit of 256).

6.3.2 File Striping

The file striping option enables you to create logical files that are up to 512 Gigabytes in size. These logical files can comprise up to 256 physical files.

As these large logical files comprise multiple physical files, you cannot enable file striping on a global basis as this could lead to the operating system running out of file handles. You need to decide, therefore, the files on which you are going to use striping.

You can enable striping for a particular file by setting the STRIPING=ON parameter for that file in the File Handler configuration file.

6.3.2.1 Headers in Striped Files

In a striped file, the header of an individual stripe is 128 bytes long. The header contains an ASCII string of the form:

Stripe number nnn of file: filename

where nnn is a 3 digit number containing the stripe number, and filename is the name of the associated file.

The string is padded with spaces up to the end of the 128 byte header.

6.3.2.2 Options

Striping options are not case sensitive.


Notes:


STRIPING=[ON/OFF]

Specifies whether a file is a striped file. The default setting is OFF.

STRIPENAMETYPE=[0/1]

This option specifies the file naming convention used to name the stripes. The default setting is 0.

If StripeNameType is set to 0, the stripe files use the same filename as the base file, but with a stripe number appended to the basename, before the file extension. For example, if the file test.dat has 3 stripes, the base file and its stripes are called:

File
Description
test.dat base file
test01.dat stripe number 1
test02.dat stripe number 2
test03.dat stripe number 3

If the basename is too long to accommodate the stripe file number, characters are removed from the right of the basename. For example, if the file testfile.dat has 3 stripes, the file and its stripes are called:

File
Description
testfile.dat base file
testfi01.dat stripe number 1
testfi02.dat stripe number 2
testfi03.dat stripe number 3

If StripeNameType is set to 1, the stripe files use the same filename as the base file but with a stripe number appended to the filename, after the file extension. Note that the base file becomes stripe 0 and is renamed. For example, if the file test.dat has 3 stripes, the base file and its stripes are called:

test.dat.00
test.dat.01
test.dat.02

MAXSTRIPESIZE=n

Specifies the maximum size in bytes of an individual stripe. The default for n is 1 Gigabyte (1073741824 bytes). The minimum value is 65536 bytes. The maximum value is currently 2 Gigabytes (2147483648 bytes).

MAXSTRIPEFILES=n

Specifies the number of stripe files, not including the base file. By default, 17 files are created, the base file and 16 stripes. The minimum value of n is 1 and the maximum value is 255.

MAXSTRIPEDIGITS=n

Specifies the number of digits to be appended to the name of the base file when naming the stripes. For example, 2 would give test01.dat and 5 would give test00001.dat or tes0001.dat depending on the filename limitations of the operating system. The default value of n is 2. The minimum value is 1 and the maximum value is 5.

STRIPE-X=path[,n]

Specifies the path and size for a stripe file where x is the stripe number, path is the location at which this stripe can be found and n is the size of the stripe. If n is omitted, the size is the same as that of the previous stripe.

Once set, the new path and size apply to all stripes until changed by another STRIPE-X option. The number following Stripe- is the stripe number, with 0 indicating the base file. For stripes other than the base file, the number must not start with a 0 character.

6.3.2.3 Naming Conventions

With file striping, a file comprises the base file (the filename specified by the program) and a number of stripe files. The stripe files use the same filename as the base file, but with a stripe number appended. For example, if the file test.dat has 3 stripes, the base file and its stripes are called:

File
Description
test.dat base file
test01.dat stripe number 1
test02.dat stripe number 2
test03.dat stripe number 3

If the basename is too long to accommodate the stripe file number, characters are removed from the right of the basename. For example, if the file testfile.dat has 3 stripes, the file and its stripes are called:

File
Description
testfile.dat base file
testfi01.dat stripe number 1
testfi02.dat stripe number 2
testfi03.dat stripe number 3

6.3.2.4 Examples

Below are several examples of how to specify file striping.

Example 1

The file test.dat has 2 stripes, each of the same size:

[test.dat]
Striping=on
Stripe-1=dir2
Stripe-2=dir3

This causes test.dat to be in the current directory, test01.dat to be in dir2 and test02.dat to be in dir3.

Example 2

The file test.dat has 5 stripes:

[test.dat]
Striping=on
Stripe-0=dir1
Stripe-4=dir2

This causes test.dat to be in dir1, test01.dat to be in dir1, test02.dat to be in dir1, test03.dat to be in dir1, test04.dat to be in dir2 and test05.dat to be in dir2.

Example 3

The file test.dat has 3 stripes:

[test.dat]
Striping=on
Stripe-0=dir1,100000000
Stripe-1=dir2,200000000

This causes test.dat to be in dir1 with a file size of 100000000 bytes and test01.dat to be in dir2 with a file size of 200000000 bytes.

6.3.3 Data Compression

Data compression enables you to compress the data in a record sequential or indexed file to save disk space.

Specifying data compression for a fixed format sequential file changes it into a variable format sequential file. This does not affect your program unless the program uses the REWRITE statement. A REWRITE of a record in a compressed sequential file will fail if the record length changes as a result of the compression.

You can specify data compression for a file using the DATACOMPRESS parameter in the File Handler configuration file. Alternatively, you can use the DATACOMPRESS Compiler directive when compiling the program.

The compression used by a file is determined by the last processed DATACOMPRESS directive when the SELECT statement for the file is processed. So you can specify data compression for an individual file by using a line of the form:

$SET DATACOMPRESS

immediately before its SELECT statement in your program. You must not forget to turn it off with a $SET NODATACOMPRESS before any other files are processed.

6.3.4 Key Compression

Key compression enables you to compress the keys of an indexed file to save disk space. Four types of key compression are available:

Key compression is specified using the KEYCOMPRESS parameter in the File Handler configuration file using the following integers to indicate which type of compression you need:

1 Duplicate key compression
2 Leading character compression
4 Trailing space compression
8 Trailing null compression

You can add these numbers together to specify a combination of compression types (with the exception of trailing nulls and trailing spaces which are mutually exclusive).

Alternatively, you can use the KEYCOMPRESS Compiler directive when compiling the program.

The key compression used by a file is determined by the last processed KEYCOMPRESS directive when the SELECT statement for the file is processed so you can set key compression for an individual file by using a line of the form:

$SET KEYCOMPRESS"8"

immediately before its SELECT statement in your program. You can turn it off again by specifying $SET NOKEYCOMPRESS before any other files are processed.

6.3.4.1 Trailing Nulls

When a key is defined with compression of trailing nulls, trailing nulls in a key value are not stored in the file.

For example, assume you have a primary or alternate key that is 30 characters long, and that you write a record in which only the first 10 characters of the key are used, the rest being nulls. Without compression, all 30 characters of the key are stored requiring 30 bytes. With compression of trailing nulls, only 11 bytes are required (10 bytes for the 10 characters of the key and 1 byte which is used to maintain a count of the trailing nulls).

6.3.4.2 Trailing Spaces

When a key is defined with compression of trailing spaces, trailing spaces in a key value are not stored in the file.

For example, assume you have a primary or alternate key that is 30 characters long, and that you write a record in which only the first 10 characters of the key are used, the rest being spaces. Without compression, all 30 characters of the key are stored requiring 30 bytes. With compression of trailing spaces, only 11 bytes are required (10 bytes for the 10 characters of the key and 1 byte which is used to maintain a count of the trailing spaces).

6.3.4.3 Leading Characters

When a key is defined with compression of leading characters, all leading characters that match leading characters in the preceding key are not stored in the file.

For example, assume that records are written with the following key values:

AXYZBBB BBCDEFG BBCXYZA BBCXYEF BEFGHIJ CABCDEF

With compression of leading characters, the keys actually stored in the index file are:

AXYZBBB BBCDEFG XYZA EF EFGHIJ CABCDEF

A count field is used to indicate the number of leading characters which are the same as in the previous key.

6.3.4.4 Duplicate Keys

When an alternate key is defined with compression of duplicates, only the first duplicate key is stored in the file.

For example, suppose you write a record with an alternate key value of "ABC". If you have enabled compression of duplicate keys, and you write another record with the same key value, the duplicate key value is not physically stored in the file.

6.3.5 Performance

You can take various steps to improve the performance of the File Handler. These all involve the use of various configuration options. Details are as follows:


Copyright © 2000 MERANT International Limited. All rights reserved.
This document and the proprietary marks and names used herein are protected by international law.

PreviousSharing Files Callable File Handler and Callable Sort APIsNext