| Rebuilding Indexed Files |
This chapter describes the following utilities that Net Express provides to help you move files between the mainframe and the PC:
| File Conversion Utility |
Use to: |
|---|---|
| DFCONV | Convert downloaded data files to the required format on the PC. |
| VRECGEN | Modify mainframe files into the format required for download from the mainframe to the PC. |
| VRECGEN2 | Modify files uploaded from the PC into the format required on the mainframe. |
The DFCONV utility, also known as the Data File Converter, converts data files. The utility provides a batch and called program interface to enable conversion of file formats, organizations and data formats and reorganization of indexed files using data file conversion profiles.
Use the Data File Converter to:
Notes:
The following sections describe the two Data File Converter operations:
The command line is:
run dfconv profile-fname [input-fname] [output-fname]
where the parameters are:
profile-fname |
The name of the profile file that contains the details of the task you need to perform (see the section Defining File Conversion with a Profile File later in this chapter). |
input-fname |
The name of the input file (optional). |
output-fname |
The name of the output file (optional). |
The input and output file-names are optional as they can be extracted from the profile file. If you specify file-names on the command line as well as in the profile file, those on the command line take precedence.
After using RUN DFCONV, you should manually update the English language messages by typing the following four lines at a new Net Express command prompt:
CD ..\BIN RUN LIBRARY DFCV.LNG MFLANGDF.LBR = MFLANGDF.LBR RUN LIBRARY DFCV.LNG MFLANG01.LBR = MFLANG01.LBR RUN LIBRARY DFCV.LNG MFLANG05.LBR = MFLANG05.LBR
After conversion, the operating system error level (ERRORLEVEL) is set to non-zero if an error occurred during conversion. If you are running the conversion from a batch file, you can interrogate this to determine what action to take after the conversion.
If a profile for a conversion exists, you can invoke the Data File Converter to do the conversion using the following interface:
call dfconv using dfconv-params
where dfconv-params comprises:
01 dfconv-params.
03 profile-filename PIC X(65).
03 input-filename PIC X(65).
03 output-filename PIC X(65).
and where the parameters are:
profile-filename
|
The name of the profile file containing the details of the data file conversion (see the section Defining File Conversion with a Profile File later in this chapter). |
input-filename |
The name of the input file. |
output-filename
|
The name of the output file. |
If you specify file-names in the parameter fields as well as in the profile file, those in the parameter fields take precedence.
On return from the Data File Converter, RETURN-CODE is set to non-zero
if an error occurred, and the error number is returned in the profile-filename
field.
The following sections describe how to use the Data File Converter to convert data files and rebuild file indexes.
Converting files from one format to another enables data to be restructured from one format to another. For example, you can convert from sequential organization to indexed organization. You can also add extra secondary indices as the needs of an application change. When using the Data File Converter, you do not need to recreate data already produced or write special conversion programs.
Although many file transfer packages convert files from one format to another, they are restricted to converting the whole file to the other data format. Typical COBOL data files and database records have complex record structures with a mix of text and binary data that must be translated accordingly. The GUI version of the Data File Converter provides an easy method for data conversion using record layout descriptions. For more information on how to create a record layout file, see the section Creating a Record Layout File in the chapter Maintaining and Creating Data Files in your Getting Started.
The Data File Converter can also convert internal floating-point formats between IEEE format and S/370 format.
A mainframe file should be transferred to the PC in binary format. If, initially, this file is a variable-length indexed file, use the mainframe program VRECGEN to perform an initial reformat of the file on the mainframe. The Data File Converter can then download and process the file.
A mainframe report format file is either a file downloaded to the PC from the mainframe or a file created by an application, downloaded to the PC and run on the PC.
A mainframe report format contains carriage-control characters which are converted by the Data File Converter to PC printable formats. When defining the mainframe report files, the record length includes the carriage-control character and the records must be fixed length. In PC Print files, the record length is solely the data length.
You can print the PC print format files on any standard ANSI printer capable of printing 132 characters per line. As some ANSI printers print only 80 characters per line, you might have to print your files in compressed mode to achieve this.
If you specify the input file as a mainframe report format file, but it was created on the PC using a downloaded mainframe application, you may receive error message 139 ("Input file not "real" mainframe report format"). This can occur if the printer output has been created in PC format rather than the mainframe report format. Check directives used to control the compilation and execution of the program.
The Data File Converter provides specific support for conversion between Mainframe Report files and PC Print files. Conversion to or from these formats to other formats is not supported.
As indexed files are updated by adding, deleting and altering records, the index and data structures become disjointed, making processing less efficient. In addition, space left by deleted records is not always reused, making the file bigger than it needs to be. After a large number of changes have been made, it is worth reorganizing an indexed file. By ensuring that the input and output file settings are identical, the data and indices can be rebuilt in an ordered sequential manner, reclaiming any free space, and so providing optimum file performance and integrity.
This section describes the entries required in the profile file to convert a file, or to restructure a file from one organization to another. For example, you might need to do one of the following:
DFS Profile File Version V09.B01.07
Batch-Process: Convert-file
***************************************************
* (Twin) (Single) *
* Options: Convert-File Rebuild-index *
* Rebuild-index-and-data None *
***************************************************
Strt-file: drive:\path\filename.STR
Edit-Mode: Quick
* Options: Quick/Full *
--------------------------------------------------------------------------------
Input-File drive:\path\inputfile.dat
Format: Micro-Focus
* Options: Micro-Focus IDXFORMAT(4) *
* Btrieve C-ISAM *
* LEVEL-II ESDS *
Organization: VRECGEN
* Options: Sequential Line-Sequential *
* Indexed Mainframe-Report-ANSI *
* Relative Mainframe-Report-mach *
* VRECGEN PC-Print *
Record-Format: Variable
* Options: Fixed/Variable *
Character-Set: EBCDIC
* Options: ASCII/EBCDIC *
Floating-Point: 370
* Options: IEEE/370 *
Compression: Off
* Options: On/Off *
Min-Rec-Length: 352
Max-Rec-Length: 633
--------------------------------------------------------------------------------
Output-File drive:\path\outputfile.dat
Format: Micro-Focus
Organization: Indexed
* Options: Sequential Line-Sequential *
* Indexed Mainframe-Report-ANSI *
* Relative PC-Print *
* VRECGEN2 *
Record-Format: Variable
Character-Set: EBCDIC
Floating-Point: 370
Compression: Off
Min-Rec-Length: 352
Max-Rec-Length: 633
* Key to settings *
* d= allow duplicates *
* s= sparse character set *
* Compression *
* cd= on duplicates *
* cl= on leading spaces *
* ct= on trailing spaces *
Prime-key : 1:1, 2:6
Alt-Key 1: 8:1
d,cl
Alt-Key 2: 9:1
d,cd,ct
Alt-Key 3: 10:5
The first record in the file identifies the format of the profile file. The profiles used by DFCONV are the same format as used with earlier Workbench.
The operation to be performed is chosen as a parameter to the Batch-Process option.
| Batch-Process | The operation to be performed on the defined file(s):
Convert-file |
An Input-File is defined for all operations and an Output-File is defined for Convert-file and Rebuild-index-and-data. The Rebuild-index operation recreates the file index for the input data file so an Output-File description is not required.
| Strt-file | The file containing the Record Layout descriptions for
the input file.
No-structure |
If the data file is being converted between EBCDIC and ANSI and the records contain non-text data items, a record layout description is required to define the data items requiring conversion and those to be left unconverted (for example, binary values). The record layouts are created with the Record Layout Editor and stored in files with an .str file extension. For more information on how to create a record layout file, see the section Creating a Record Layout File in the chapter Maintaining and Creating Data Files in your Getting Started.
| Edit-Mode | The edit mode if this profile is read by the Data File
Editor
Quick |
This parameter is not used by DFCONV.
First of all, choose an input file:
| Input-File | The input file name for the defined operation. |
Then, use parameters to describe the input file:
If the file being created is indexed, the keys are defined using the following records.
First of all, choose an output file:
| Output-File | The output file name for the defined operation. |
Then, use parameters to describe the output file:
If the file being created is indexed the keys are defined using the following records.
Alternative keys are defined following the prime key by using "Alt-Key 1:" as the key word for the first alternate key and "Alt-Key 2:" for the second alternate key. The format of the two records in the profile file defining the alternate keys are as defined for the Prime key.
You are allowed to define up to 63 alternate record keys by specifying, for each, the POSITION and LENGTH in the record and whether duplicate values are permitted. Each key can be split and you can define overlapping keys.
Split keys are defined by including multiple POSITION and LENGTH definitions in the key definition record separated by comma and then a space (for example, 1:12, 20:5).
You should be aware of the following points when defining the input and output files:
The record length information is extracted from the file header and replaces the record length information defined for the input file in the profile file
You should set the input record lengths and the output record lengths so that the record length of the mainframe Print Format file is one character longer to hold the carriage control characters
The Data File Converter cannot interpret the profiles sent to it; the File Converter applies only to DOS and ANSI based programs. To ensure correct processing, use an ANSI-based program to call the Data File Converter.
The output file type you specify should not be line sequential. Extra bytes (X'00') are inserted into the ANSI line sequential record whenever a byte contains a value of X'1B' or less, this may result in unnecessarily large files. The extra bytes are include to provide NULL protection to data held in the line sequential file, these bytes would otherwise be interpreted as control characters used in this type of file.
The interface to Rebuild from the Data File Converter supports the rebuilding of corrupt indices.
DFS Profile File Version V08.U04.01
Batch-Process: Rebuild-index
***************************************************
* (Twin) (Single) *
* Options: Convert-File Rebuild-index *
* Rebuild-index-and-data None *
***************************************************
Strt-file: No-structure
Edit-Mode: Quick
* Options: Quick/Full *
--------------------------------------------------------------------------------
Input-File e:\test\testidx.dat
Format: Micro-Focus
* Options: Micro-Focus IDXFORMAT(4) *
* Btrieve C-ISAM *
* LEVEL-II ESDS *
Organization: Indexed
* Options: Sequential Line-Sequential *
* Indexed Mainframe-Report-ANSI *
* Relative Mainframe-Report-mach *
* VRECGEN PC-Print *
Record-Format: Variable
* Options: Fixed/Variable *
Character-Set: ASCII
* Options: ASCII/EBCDIC *
Floating-Point: IEEE
* Options: IEEE/370 *
Compression: Off
* Options: On/Off *
Min-Rec-Length: 352
Max-Rec-Length: 633
* Key to settings *
* d= allow duplicates *
* s= sparse character set *
* Compression *
* cd= on duplicates *
* cl= on leading spaces *
* ct= on trailing spaces *
Prime-key : 1:1, 2:6
Alt-Key 1: 8:1
d
Alt-Key 2: 9:1
d
The Rebuild function can be used for rebuilding index sequential files only. To rebuild other file formats, use the file conversion function with identical settings for the input and output files.
Note: If you are using a profile file, as described above, any offsets you define in the Prime-key field start at 1. However, if you use the GUI version of the Data File Converter, key offsets start at zero from the Key Information dialog box. To find out more about the Data File Converter, see the section Converting Data Files in the chapter GUI Data Tools.
Indexed files that have a corrupt index can be recovered using this option. Indices can become corrupt if, for example, a power loss occurs when a program has opened and changed an index file, but has not yet closed the file. A flag, indicating that this has occurred, is detected in the file and reported by subsequent accesses to that file. No further operations can be done on the file until its integrity has been recovered.
Rebuild rebuilds just the index, which is all that is required for the recovery. If the index is already in existence, Rebuild tries to recover the index using the information in that file.
You should be aware of the following points when using Rebuild:
The error messages are described in the Net Express online help. (Click Help Topics on the Help menu. Then, on the Contents tab, click Reference, Error Messages, Data File Converter messages.)
VRECGEN is a mainframe utility. Use VRECGEN to create a copy of a variable-length mainframe file. That copy embeds record length information in the data records for download from the mainframe to the PC. Micro Focus provides the source code for the VRECGEN utility (vrecgen.cbl). However, you may need to modify the source code before use, to specify options not catered for by the supplied version vrecgen.cbl.
The file created by VRECGEN has a two-byte record length field preceding each of the variable length data records. The record length field contains a value which is the two-byte record length field plus the length of the variable length data record. This is necessary because most file transfer software (for example, IBM SEND/RECEIVE) works with logical records and strips off the "Record Descriptor Word" which contains the length for each record. Without this information, it is impossible to separate the records to rebuild the data set on the PC.
The file created by VRECGEN is in a special download format. You should use the standard Transfer utility to transfer this file to the PC in binary format. The file can then be processed through the Data File Converter.
Note: The source code to VRECGEN (vrecgen.cbl) is provided by Micro Focus primarily as a model, rather than as a final utility that meets the needs of all sites without modification. You should edit your version of vrecgen.cbl so that it contains the specific values that meet your application or site requirements. A typical reason for editing vrecgen.cbl is to increase the maximum record length to be more than 32,000 bytes.
If you plan to download variable-length test files, you must first upload and compile the VRECGEN program in your mainframe environment. This program is provided as a sample COBOL program for converting mainframe variable-length format files.
Note: Each site should review the provided source code and determine maximum and minimum lengths for its variable-length files. These values (plus 4 for the MVS "Record Descriptor Word") should be used in the OCCURS DEPENDING ON clause of the record descriptions in the sample FDs.
To install VRECGEN:
Note: If you use VS COBOL II, Release 3 or above, and receive a file status of 39, recompile VRECGEN using the CMPR2 compile option.
You need to supply the appropriate JCL information before running the VRECGEN on the mainframe. Enter the following JCL skeleton pointing to the correct STEPLIB, input data set and output data set:
//MYJOB JOB (ACCOUNTING INFORMATION) //STP1 EXEC PGM=VRECGEN //STEPLIB DD DSN=STEPLIB,DISP=SHR //SYSOUT DD SYSOUT=* //INPDD DD DSN=INPUT.OS.DATA.SET.DISP=SHR //OUTDD DD DSN=OUTPUT.OS.DATA.SET, // DISP=(NEW,CATLG,DELETE), // DCB=(RECFM=VB,LRECL=LLL,BLKSIZE=BBB), // SPACE=(TRK,(X,Y),,RLSE), // UNIT=SYSDA
Notes:
VRECGEN2 is a mainframe utility that converts files in Micro Focus variable-length record format to mainframe format. Thus it enables you to upload these files and use them on the mainframe.
If you plan to convert files in Micro Focus variable-length record format to mainframe format, you must first upload and compile the VRECGEN2 program in your mainframe environment:
The following steps show you how to upload a variable length data file whose records are between 100 and 400 bytes long:
FORMAT = VB
RECLEN = 406
BLKLEN = 410
A valid variable length file should result as the output. The following shows the format of a VRECGEN2 format file, using a record length of 100 bytes as an example:
| Bytes |
Contents |
|---|---|
| 1-4 | Record Descriptor Word. |
| 5-6 | Record length. |
| 7-106 | Record data. The record is padded with null bytes if it is shorter than the maximum record size. |
Note: The VRECGEN and VRECGEN2 file formats are incompatible in that VRECGEN format is used to download files from the mainframe and VRECGEN2 format is used to upload Micro Focus format files to the mainframe.
Copyright © 2000 MERANT International Limited. All rights reserved.
This document and the proprietary marks and names
used herein are protected by international law.
| Rebuilding Indexed Files |