This chapter describes all of those Fileshare features which enable you to safeguard the integrity of your database:
Note: In Mainframe Express, Fileshare is only supported for use by the components of Mainframe Express. Information in this chapter that describes using Fileshare in your own applications is not supported.
Transaction processing enables you to group a set of data file updates together into one logical unit of work, known as a transaction. The changes you make as part of a transaction can be:
A transaction can include multiple updates to multiple data files on multiple Fileshare Servers.
A transaction starts when your program performs the first update operation (WRITE, REWRITE or DELETE) on a file which has the WITH ROLLBACK clause defined in its SELECT statement after:
A transaction terminates when:
Before using transaction processing you are strongly advised to do the following to ensure the highest level of integrity for your data:
Record locks are implicitly obtained on every record that is updated in a transaction. If a transaction is temporarily suspended, for example, if your program requests input and the user is not responding, record locks can remain in the file for a considerable length of time. If another Fileshare Client tries to access the locked records, and a specified length of time has passed, the Fileshare Server aborts (rolls back) the transaction. When this happens, the transaction is said to be timed out by the Fileshare Server. For more information, see Automatic Record Lock Timeout below.
Note: By keeping the number of updates performed in a transaction to a minimum, the number of record locks obtained is also kept to a minimum. This reduces the number of record locking conflicts between Fileshare Clients sharing the data, and increases the speed of any subsequent COMMIT or ROLLBACK operations.
In a multi-user environment where files are being shared, programs use record locks to control concurrent access to individual records. In addition, Fileshare automatically obtains a record lock on any record that it updates within a transaction. The larger the number of record locks being held, however, the less concurrent access to data is possible. It is beneficial, therefore, for any one program to keep the number of record locks that it holds at any one time to a minimum.
If a Fileshare Client obtains a lock on a record (either explicitly or because it updated the record within a transaction) and a second Fileshare Client tries to read the locked record, the Fileshare Server checks to see how long it is since the Fileshare Client holding the record lock last contacted the Fileshare Server. If this period is greater than the timeout period, then:
If this period is less than the timeout period, then:
The default timeout period is 60 seconds. You can change this value using the /t option in the Fileshare Server configuration file. Specifying a value of 0 disables the automatic record lock timeout such that record locks will only be released under application program control.
A program that is timed out receives error 9/125 on its next I/O operation. You should treat this as a warning message and repeat the operation on which the error was returned. The Fileshare Server will roll back any active transaction and release all the record locks held by the program that timed out.
For each data file on which you want transaction processing active, use the WITH ROLLBACK clause when you define the data file using the SELECT statement in your COBOL program.
select test-file assign to "test.dat" organization indexed record key prime-key lock mode manual with rollback ...
This sample code specifies that transaction processing is active on the file test.dat.
In your program, use the COMMIT verb to make the updates in a transaction permanent. Use the ROLLBACK verb to remove the updates in a transaction from the data files.
The COMMIT and ROLLBACK verbs work on all data files involved in a transaction. This means that you cannot check the status of the COMMIT or ROLLBACK operation using the file status of any one data file. Instead, you must call FHRedir to check the status of the COMMIT or ROLLBACK. The entry point to call in the FHRedir module is fs_status. The format of the call is:
call "fs_status" returning t-status end-call
where t-status is defined as:
01 t-status pic x(2) comp-x.
A non-zero value returned in t-status indicates that the COMMIT or ROLLBACK operation failed.
1 SELECT test-file ASSIGN TO "test.dat" 2 LOCK MODE MANUAL WITH ROLLBACK 3 ... 4 OPEN I-O test-file 5 ... 6 move 1 to prime-key 7 WRITE test-file-record 8 ... 9 COMMIT 10 call "fs_status" 11 returning t-status 12 end-call 13 if t-status < > 0 14 display "Warning - Commit operation failed" 15 end-if 16 move 2 to prime-key 17 WRITE test-file-record 18 ... 19 ROLLBACK 20 call "fs_status" 21 returning t-status 22 end-call 23 if t-status < > 0 24 display "Warning - Rollback operation failed" 25 end-if 26 WRITE test-file-record 27 ...
Lines 1 - 2:
SELECT test-file ASSIGN TO "test.dat" LOCK MODE MANUAL WITH ROLLBACK
Specifies transaction processing for this data file.
Starts a new transaction.
Makes the updates to test-file permanent and terminates the transaction.
Lines 10 - 15:
call "fs_status" returning t-status end-call if t-status < > 0 display "Warning - Commit operation failed" end-if
Checks the status of the COMMIT operation.
Starts a new transaction.
Removes the updates made to test-file and terminates the transaction.
Lines 20 - 25:
call "fs_status" returning t-status end-call if t-status < > 0 display "Warning - Rollback operation failed" end-if
Checks the status of the ROLLBACK operation.
Starts a new transaction.
You do not need to change the configuration of the Fileshare Client to enable transaction processing.
You do not need to change the configuration of the Fileshare Server to enable transaction processing. However, if you recover a data file using the Rollforward Recovery Utility (see below), any outstanding transactions at the end of the recovery process are implicitly rolled back. For more information, see Rollforward Recovery Utility.
A system failure can result in loss of data in either of the following circumstances:
Rollforward recovery logging is the method used by Fileshare to store all the updates made to your data files in one or more log files. If your system fails, resulting in loss of data, the Rollforward Recovery Utility together with the log file(s) can be used to re-apply the updates to a back-up copy of the original data files. When the recovery is complete, the files contain the data they contained at the time of the system failure (with any outstanding transactions rolled back).
You do not need to make any changes to the Fileshare Client in order to use rollforward recovery logging.
To log updates to one or more files you need to create a database reference file containing the name of the file(s) for which updates are to be logged, and the name of the log file. See Database Reference File Maintenance below for details on creating database reference files.
You do not need to include an extension when you specify the log file name as this is generated automatically by the Fileshare Server. If you do specify an extension, the following message is displayed:
FS180-W The log filename specified has an extension. The name of the log file has been changed to "filename".
During the logging process the Fileshare Server creates one or more recovery log files. The first recovery log file is called a "starter" log file, and subsequent log files are called "continuation" log files. The Fileshare Server creates a continuation log file whenever the current log file is backed up.
The current recovery log file always has an extension beginning with the character "L". A recovery log file that has been backed up has an extension beginning with the character "B". Following either of these letters is a two digit number indicating the recovery log file number, starting at 01. These automatically generated extensions indicate the order in which the log files were written, and hence the order in which they should be applied if a rollforward recovery is performed. For example, suppose you specify a log file called RECOVER. When the Fileshare Server starts up it creates a log file called RECOVER.L01 and logs updates in this file. If the recovery log file is backed up, the Fileshare Server renames the current log file RECOVER.B01 and creates a new one called RECOVER.L02. If the Fileshare Server is shut down, there would be two recovery log files: RECOVER.B01 is called the "starter" log file and RECOVER.L02 is a "continuation" log file. There is only ever one starter log file, but there can be any number of continuation log files.
When performing a recovery operation, the Rollforward Recovery Utility processes the starter log file first and then the continuation log files in the order in which they were created.
Updates to data files with entries in the database reference file are logged unless you explicitly turn logging off. A data file entry is added to the database reference file using the /f option of the Database Reference File Maintenance Utility. To turn logging off for a particular data file, use the /o l (omit logging) option when you add the filename entry to the database reference file.
Due to the caching used by some operating systems, not all of the information written to the log file is physically written to disk. The Fileshare Server periodically calls the operating system to force the log file updates to be written. By default, the log file is flushed after every 100 records are written to the log file. You can configure this interval using the /lc (log count) option when you add the log file name to the database reference file. If you specify a log count of zero, the Fileshare Server does not make any calls to the operating system to flush the updates to disk. Instead, the operating system controls the flushing of the log file to disk.
The log file updates are also flushed to disk after every COMMIT operation.
If a transaction was pending at the time of a system failure, the Rollforward Recovery Utility implicitly rolls back the transaction when the files are recovered.
Rollforward recovery logging can severely affect performance. You can use the /o l option in the database reference file to switch logging off for those files for which this feature is not essential. Alternatively, you can omit entries for these files from the database reference file altogether.
The recommended method of using recovery logging is:
Note: You can have the Fileshare Server automatically perform this sequence of operations for you by specifying a backup directory in the database reference file. See the section Automatic Database Backup and Rollforward Recovery below.
If you have a large number of updates being logged, the log file can grow very large. If the disk where the log file resides were to become full, the ability to write records to the log file could be lost. This would result in an incomplete log file forcing the Fileshare Server to close down. A warning is displayed when the amount of free disk space for the log file drops below a certain percentage of the total disk. You can configure this percentage using the /p database reference file option. When you see this warning, close down the Fileshare Server and take a new back up of your data files. This enables you to discard the log file and continue with a new one when the Fileshare Server is restarted.
If you ignore the warning message and the log file continues to grow larger, the Fileshare Server suspends operation when the free disk space available for the log file drops below 200,000 bytes. It displays error message FS030 and waits for the old log file to be backed up remotely or removed manually before continuing with a new log file.
The Rollforward Recovery Utility enables you to recover updates to data files after a system failure.
Note: The Rollforward Recovery Utility can only be used to recover a database if the updates were logged to a recovery log file. See Rollforward Recovery Logging above for details.
Before attempting the rollforward recovery process:
The Rollforward Recovery Utility is invoked using the /r (recover) option. You must specify a database reference file when invoking the Rollforward Recovery Utility, for example:
fs /r dbase.ref [options]
Here dbase.ref is the name of the database reference file used by the Rollforward Recovery Utility. This must be the same as the database reference file used by the Fileshare Server while it was running. The Rollforward Recovery Utility reads the database reference file to determine the location and name of the rollforward recovery log file(s) and the name(s) of the data file(s) to recover.
Note: You can select the data files you want to restore by erasing other entries from the database reference file using the Database Reference File Maintenance Utility (/e option) before invoking the Rollforward Recovery Utility.
The following options can be specified with the Rollforward Recovery Utility:
Specifies that the Rollforward Recovery Utility restores the updates made to the data files up to the time specified. The format of date-time is YYMMDDHHMMSS (year, month, day, hour, minutes, seconds) specified numerically. When this option is not specified, the Rollforward Recovery Utility applies all of the updates that are stored in the rollforward recovery log file.
Causes the Rollforward Recovery Utility to display additional information about which recovery log files it is searching for and in which directories.
fs /r dbase.ref /dt 931231235959
invokes the Rollforward Recovery Utility using the database reference file dbase.ref. Only those updates that were made up to the end of 31 December 1993 are restored.
Updates can be recorded in multiple log files (see Rollforward Recovery Logging above for details). The Rollforward Recovery Utility scans the log file directory for the starter log file. If none is found, the directory is scanned for a backed up starter log file. If a starter log file still cannot be found the same procedure is repeated with the searches being performed in the backup directory if one is specified in the database reference file. If the Rollforward Recovery Utility is able to locate the starter log file then the recovery procedure commences; if not an error is displayed and the recovery does not start.
Once the starter log file has been processed, the Rollforward Recovery Utility searches for a continuation log file. If no backup of the recovery log file was performed, there will be no continuation log file and the recovery process terminates. The Rollforward Recovery Utility searches for the continuation log file in the log file directory and then in the log file backup directory. The recovery process continues using the continuation log file(s) until all of the updates have been recovered and applied to the data files.
Database sequence numbering provides a method of linking groups of data files so that they can be kept in step.
It is also used to ensure that data files and their corresponding log file are kept in step. In order for the Rollforward Recovery Utility to work, all data files must have sequence numbers that match the log file. The log file contains a sequence number that is written to the header of each file for which sequence number checking is active at the time that the Fileshare Server is shut down.
A sequence number is automatically stored in the header of each data file for which you have specified sequence numbering. Fileshare increments these sequence numbers every time you invoke the Fileshare Server. If the sequence numbers in the data files do not match, the Fileshare Server fails to start after displaying this message:
FS038-S A matching sequence number not found in <filename>
The possible causes for this error are:
All data files listed in the database reference file have sequence number checking active by default. You can turn off sequence number checking using the database reference file option /o s (omit sequence numbering). If sequence number checking is not used on a file, your system administrator must make sure that the correct version of the files is installed in the Fileshare System.
Sequence numbering is only available on data files with a variable length record file header. The following file organizations do not contain headers and so sequence number checking is not possible:
If you have fixed length record data files that you want to use sequence numbering on, create them as variable length record data files. You can do this by changing the file definition in your COBOL program before creating the file.
Data file definition specifying fixed length records:
FD account-file. 01 account-file-record. 03 account-file-prime pic x(4).
Equivalent data file definition specifying variable length records:
FD account-file recording mode is variable. 01 account-file-record. 03 account-file-prime pic x(4).
You do not have to change the configuration of the Fileshare Client or your program to make use of database sequence numbers.
For data files that you entered in the database reference file, sequence number checking is on by default. If you do not need sequence number checking for a particular data file, you can disable the feature by using the /o s (omit sequence numbers) option for that data file's entry in the database reference file.
fs /d dbase.ref /l \logdir\log.dat
fs /d dbase.ref /f test1.dat
fs /d dbase.ref /f test2.dat /o l
fs /d dbase.ref /f test3.dat /o s
fs -d dbase.ref -l /logdir/log.dat
fs -d dbase.ref -f test1.dat
fs -d dbase.ref -f test2.dat -o l
fs -d dbase.ref -f test3.dat -o s
This example specifies:
The recommended method of using Fileshare's update logging for rollforward recovery is:
See the section Rollforward Recovery Logging above for more details.
You can get Fileshare to do this for you automatically by specifiying a backup directory name in the database reference file using the /backup option. For example, the command:
fs /d dbase.ref /backup c:\backdir
adds an entry to the database reference file called dbase.ref which specifies the backup directory as c:\backdir. A full path can be specified for the backup directory, otherwise paths which are not fully qualified will be taken to be relative to the current directory of the Fileshare Server.
Setting a backup directory in this way causes the Fileshare Server to copy all the data files specified in the database reference file to the directory c:\backdir on start-up. The Fileshare Server will also delete any recovery log file(s) that it finds since this data is obsolete once the database files have been backed up.
While the Fileshare Server is active, all updates to files which have logging switched on for them will be recorded in the log file as normal. The next time the Fileshare Server is restarted, it will again backup the database files and discard any log file(s). However, if it detects that the previous Fileshare Server session did not shut down successfully and a valid terminating record had not been written to the log file (for example due to a system crash), then Fileshare will automatically trigger the Rollforward Recovery Utility. This will automatically restore the previous backup of the database files and recover all updates to them as recorded in the log file(s). If the recovery completes succesfully, the Fileshare Server will then start up as normal.
It is still possible, when a backup directory is specified in the database reference file, to run the Rollforward Recovery Utility manually, before starting up the Fileshare Server. The Rollforward Recovery Utility will automatically restore the previous backup before initiating the recovery.
Note: The purpose of the automatic database backup facility is to help database administrators follow the recommended steps for rollforward recovery, thus ensuring that the latest Fileshare log file contains only the updates made since the last database backup. If database files are added or replaced manually, Fileshare may not detect this and could backup or restore the incorrect version of a file, compromising the integrity of your data. It is recommended, therefore, that you do not attempt to replace any of the database files manually and only ever use Fileshare operations such as closing down the server to back up the entire database.
If you have enabled automatic database backup by specifying a backup directory, you cannot use the /o l option to switch logging off for any of the data files declared in the database reference file. By having logging active for every file specified in the database reference file, Fileshare can guarantee that it can recover updates to them all when it copies backed up versions from the backup directory. If there are some files for which you do not require automatic backup, omit them from the database reference file.
If you need to back up your Fileshare files, for example as part of a system-wide backup overnight, you can use a Fileshare Manager API function to request Fileshare to shut down, back up all the database files and then restart with a fresh log file. Having done this, it is recommended that you then make a permanent hard copy of the Fileshare backup directory. See the section Fileshare Manager in the chapter Advanced Operation for more information.
fs /d dbase.ref /l logfile fs /d dbase.ref /backup c:\backdir fs /d dbase.ref /f c:\accounts\customer.dat fs /d dbase.ref /f c:\finance\history.dat
On start-up, the Fileshare Server will automatically back up these data files in the directory c:\backdir. If the log file is backed up while the Fileshare Server is active, backup files will also be placed in this directory. If the Rollforward Recovery Utility is invoked, the backup files in c:\backdir will be restored to their original directories and any updates recorded in the log file(s) will be re-applied.
The database reference file contains information that the Fileshare Server uses to determine which Fileshare features you want to use on which files. This section explains how to create and update a database reference file in order to make use of the features described earlier in this chapter.
The Database Reference File Maintenance Utility is invoked using the /d (database) option, for example:
fs /d dbase.ref [options]
|is the name of the database reference file. If the database reference file does not exist, it is created.
|you specify the operation to be carried out on the database reference file by specifying options. Each initiation of the Database Reference File Maintenance Utility adds a record to or removes a record from the database reference file specified.
You can add four types of record to the database reference file:
These are described below.
You specify the name of the recovery log file using the /l option, for example:
fs /d dbase.ref /l logfilename
You can only have one recovery log file record in a database reference file.
You can specify the following for a recovery log file record:
You can specify the point at which the Fileshare Server displays a warning when the drive containing the log file is becoming full using the /p option:
fs /d dbase.ref /l logfilename /p nnn
where nnn specifies the percentage disk space remaining, below which the Fileshare Server displays a warning message. The default value is 5, that is, the Fileshare Server displays the warning message when the drive containing the log file contains less than 5% free space.
You can specify how often the log file is flushed to disk using the /lc option:
fs /d dbase.ref /l logfilename /lc nnn
where nnn specifies the number of records that are written to the log file before the Fileshare Server calls the operating system to flush the log file buffers to disk. The default value is 100, that is, the Fileshare Server flushes the log file to disk after each set of 100 records are written to it. In addition, the Fileshare Server automatically flushes the log file after every successful COMMIT operation.
The Fileshare Server is unable to flush a log file if the log file is not on the machine where the Fileshare Server is running or if the operating system does not allow Fileshare to force the log file to be flushed. In this situation, the /lc 1 option forces the Fileshare Server to specify writethrough when opening the log file. The log file is then written to disk if the operating system supports writethrough on files.
You can erase a recovery log file record using the /e option:
fs /d dbase.ref /l logfilename /e
This is the only way to remove a record once you have created the database reference file.
fs /d dbase.ref /l \log\log.dat /p 10 /lc 200
Specifies the log file as log.dat in the directory log. Any data files that are specified in this database reference file have all updates recorded in this log file, unless logging is explicitly disabled for a data file. The /p option specifies that the Fileshare Server displays a warning message when less than 10 percent of the disk space remains free on the drive where the log file resides. The /lc option specifies that the Fileshare Server flushes the log file to disk after every 200 records are written to it.
You specify a filename record using the /f option, for example:
fs /d dbase.ref /f filename
specifies a data file that you want to include in the database reference file.
Any file that you specify in the database reference file has logging and, for those file organizations that support it, database sequence numbering, enabled by default.
You can specify the following for a filename record:
You can specify an alternative filename for a filename record using the /af option:
fs /d dbase.ref /f filename /af alternate-filename
such that whenever the Fileshare Server is asked to access the data file filename, it accesses alternate-filename instead.
You can specify a virtual file handler for a filename record using the /ap option:
fs /d dbase.ref /f filename /ap program-name
where program-name specifies the virtual file handler that the Fileshare Server calls to process all I/O requests on the data file. See the section Virtual File Handler in the chapter, Advanced Operation.
The filename definition in the /f entry (corresponding to an /ap entry) can include a trailing wildcard to specify that groups of files are to have their requests passed to the specified virtual file handler.
You can specify data and key compression for a filename record using the /k option, for example:
fs -d dbase.ref /f filename /k DnnnIx
where nnn specifies the data compression routine number and x specifies the key compression number. Valid values for nnn and x are the same as those that can be specified for the Compiler directives DATACOMPRESS and KEYCOMPRESS.
You can disable logging for a file using the /o l option:
fs /d dbase.ref /f filename /o l
disables logging for filename.
Note: You cannot disable logging for a file if you have enabled automatic database backup by specifying a backup directory record.
You can disable database sequence numbering for a file using the /o s option:
fs /d dbase.ref /f filename /o s
disables database sequence numbering for filename.
You can disable transaction processing for a file using the /o t option:
fs /d dbase.ref /f filename /o t
disables transaction processing for filename. The WITH ROLLBACK clause in a SELECT statement for this data file is ignored. This option in the database reference file overrides any transaction processing specified in your program.
You can erase a filename record using the /e option, for example:
fs /d dbase.ref /f filename /e
erases the record from the database reference file. This is the only way to remove a record once the database reference file has been created.
fs /d dbase.ref /f \accounts\nomlgr.dat /k d001i7
specifies that the data file nomlgr.dat in the directory accounts has data compression and key compression activated if it is created by the Fileshare Server. The d001 option specifies the data compression routine you want to use is the same as if you compiled your program with the DATACOMPRESS"1" Compiler directive. The i7 option specifies the key compression to use is the same as if you compiled your program with the KEYCOMPRESS"7" Compiler directive. This overrides any data or key compression specified in the program.
fs /d dbase.ref /f \accounts\salesday.dat /o tl
specifies that any transaction processing specified in your program for the data file salesday.dat in the directory accounts is ignored. Also, logging is disabled for this file.
fs /d dbase.ref /f \ref\clients.dat /af \oldref\oldclnts.dat /o s
specifies that all I/O requests for the data file clients.dat in the directory ref are redirected to the data file oldclnts.dat in the directory oldref. In addition, database sequence numbering is disabled for this file.
fs /d dbase.ref /f \accounts\saleslgr.dat
enables logging and database sequence numbering for the data file saleslgr.dat in the directory accounts. By default, logging and database sequence numbering are enabled for every data file listed in the database reference file.
fs /d dbase.ref /f \orders\week10.dat
enables logging and database sequence numbering for the data file week10.dat in the directory orders. By default, logging and database sequence numbering are enabled for every data file listed in the database reference file.
You specify a file string record using the /fs option, for example:
fs /d dbase.ref /fs string1 /af string2
specifies that for any data file with a filename starting with string1, string1 is replaced by the character string string2.
You can erase a file string record using the /e option:
fs /d dbase.ref /fs string /e
erases the record from the database reference file. This is the only way to remove a record once the database reference file has been created.
fs /d dbase.ref /fs c:\accounts\ /af c:\lastyear\accounts\
specifies that the character string c:\accounts\ at the front of any filename is replaced by the character string c:\lastyear\accounts\. Note that the string specified by the /fs option is not expanded relative to the Fileshare Server's current directory.
You specify an automatic backup directory using the /backup option, for example:
fs /d dbase.ref /backup c:\backup
specifies that on start-up the Fileshare Server automatically copies all of the files named in the database reference file into the directory c:\backup.
Copyright © 1998 Micro Focus Limited. All rights reserved.
This document and the proprietary marks and names used herein are protected by international law.