Chapter 12: Scan64

This chapter describes the Scan64 utility, which enables you to check programs for pointer violations.

12.1 Introduction

Use Scan64 to help you migrate applications from 32-bit to 64-bit COBOL systems. If you have developed COBOL programs using the Server Express 32-bit development environment that you want migrate to the Server Express 64-bit development enviroment, you should run Scan64 before compiling and running them with the 64-bit system development environment.

A POINTER in COBOL is a data item of unknown size that can only be operated on as a pointer. Direct or indirect treatment of POINTER data-items as non-pointer data-types is a violation of the rules for using pointers. You must detect and correct these violations before migrating your COBOL source code to 64-bit COBOL systems. The Scan64 utlity enables you to detect pointer violations.

Note: Scan64 cannot detect invalidly-sized data items provided to prototyped CALL statements that require pointers.

12.2 Preparing to Use Scan64

You can only use Scan64 on applications that have been compiled to intermediate code. You create intermediate code using the -a cob flag.

For example, the command:

cob -a myapp.cbl

creates the file myapp.int.

12.3 Starting Scan64

You start Scan64 from the command line. The format of the command is:

cobscan64 [-v] [-m] [-ttfile [-ltlist]] filename

where the parameters are:

-v	Internal addressing information is displayed. This can be useful to MERANT Micro Focus support representatives.
-m	Include modifier information.
-ttfile	The COBOL header file that contains COBOL CALL prototypes for entry points called within this program.
-ltlist	The resulting template listing file with any diagnostics from parsing the COBOL header file.
filename	The intermediate code file to be analyzed. You can specify the filename with or without the .int filename extension. If you omit the filename extension, .int is assumed.

If you enter cobscan64 without parameters, the available options are listed.

12.4 Examples of Using Scan64

The following sections give you examples of using Scan64. You are advised to look at each demonstration program that relates to each example. The demonstration programs are stored in the directory $COBDIR/demos/scan64. Use Infomgr to look at the demonstration programs and move them to a working directory.

12.4.1 Modification of Pointers by Implicit Redefinition

This example shows you how you can use Scan64 to identify pointers that are modified by implicit redefinition. demo1.cbl is a short COBOL program that contains such pointer operations.

Create the intermediate code file:

cob -a demo1.cbl

Run Scan64 on this program by entering the command:

cobscan64 demo1

This produces the following report:

Processing file 'demo1.int'
**Unsafe modification of pointer at (file: demo1.cbl, line: 9)
  Pointer : PP (file: demo1.cbl, def: 6)

Finished file 'demo1.int' - pointer problems could exist

Look at the source for demo1.cbl. Line 9 of the program contains the following:

move low-values to ppg

This modification of the group item containing the pointer pp is unsafe, as the memory occupied by the pointer pp is treated as an alphanumeric data item. In this case, since low-values is being moved, no changes are necessary. However, it would be better practice to change the code at line 9 to:

set pp to null

It can be difficult in COBOL to identify the point at which a pointer is modified. Use the Scan64 -m option to help you identify code that modifies pointers.

For example:

cobscan64 -m demo1

produces:

Processing file 'demo1.int'
**Unsafe modification of pointer at (file: demo1.cbl, line: 9)
  Modifier : PPG (file: demo1.cbl, def: 5)
  Pointer : PP (file: demo1.cbl, def: 6)
  Finished file 'demo1.int' - pointer problems could exist

12.4.2 Possible Pointer Modification Via CALL Statements and Prototypes

This example shows you how you can use Scan64 to identify pointers that are modified by CALL statements. Scan64 can use CALL prototypes to validate the pointer parameters passed on CALL statements; this enables it to avoid reporting problems on parameters that comply with the prototype.

For this example, use the demonstration file demo2.cbl. Create the intermediate code file:

cob -a demo2.cbl

This example contains calls to the library routines CBL_ALLOC_MEM or CBL_FREE_MEM; without CALL prototypes any pointers provided to these routines are effectively redefined to alphanumeric. You can see this if you run Scan64 without parameters:

cobscan64 demo2

This produces the report:

Processing file 'demo2.int'
**Possible unsafe modification of pointer in a 'BY REFERENCE' parameter 
**in CALL 'CBL_ALLOC_MEM' statement at (file: demo2.cbl, line: 8)
  Pointer : PTR (file: demo2.cbl, def: 5)
**Possible unsafe reference of pointer in a 'BY VALUE' parameter
**in CALL 'CBL_FREE_MEM' statement at (file:  demo2.cbl, line: 11)
  Pointer : PTR (file: demo2.cbl, def: 5)
  Finished file 'demo1.int' - pointer problems could exist

However, you can specify that Scan64 should use the prototype file demoproto.cpy which defines the APIs CBL_ALLOC_MEM and CBL_FREE_MEM:

cobscan64 -t demoproto.cpy demo2

This produces the report:

Processing file 'demo2.int'
Finished file 'demo2.int' - pointer usage is OK

In this case, no messages are generated because Scan64 knows that CBL_ALLOC_MEM and CBL_FREE_MEM expect pointers to be passed into them and there is no danger of them being treated as fixed size alphanumeric.

12.4.3 Explicit and Implicit Definitions of Pointers in CALL Prototypes

This example shows you how you can use Scan64 to identify pointers that are modified by implicit redefinition. The demonstration program demo3.cbl shows how prototypes are used to create implicit definitions of pointers within scanned programs, and how incompatible parameters are reported.

Create the intermediate code file:

cob -a demo3.cbl

The parameter to which you should pay attention in this program is the ptrnum-g group item. Without prototypes for the various calls used within this program this source file passs through Scan64 without problems. For example, entering:

cobscan64 demo3

gives:

Processing file 'demo3.int'
Finished file 'demo3.int' - pointer usage is OK

However, if you specify a prototype, the program no longer passes through Scan64 without problems. For example, specifying:

cobscan64 -m -t demoproto.cpy demo3

gives:

Processing file 'demo3.int'
**Possible unsafe modification of 'BY REFERENCE' pointer parameter
**in CALL 'CBL_ALLOC_MEM' statement at (file: demo3.cbl, line: 10)
  Modifier : PTRNUM-G (file: demo3.cbl, def: 5)
  Pointer : MEM-PTR a formal parm
            (file: demoproto.cpy, def: 17, ref: 29, no: 1)
          in CALL prototype of CBL_FREE_MEM
            (file: demoproto.cpy, def:28)
          as implicitly def'd by CALL at (file: demo3.cbl, line: 10)

**Possible unsafe modification of 'BY VALUE' pointer parameter
**in CALL 'CBL_FREE_MEM' statement at (file: demo3.cbl, line: 13)
  Modifier : PTRNUM-G (file: demo3.cbl, def: 5)
  Pointer : MEM-PTR a formal parm
            (file: demoproto.cpy, def: 17, ref: 29, no: 1)
          in CALL prototype of CBL_FREE_MEM
            (file: demoproto.cpy, def:28)
          as implicitly def'd by CALL at (file: demo3.cbl, line: 13)

**Unsafe modification of pointer at (file: demo3.cbl, line: 17)
  Modifier : PTRNUM (file: demo3.cbl, def: 6)
  Pointer : MEM-DUMMY-PTR
            (file: demoproto.cpy, def: 19)
          of formal parm MEM-DUMMY-PTR-G
            (file: demoproto.cpy, def: 18, ref: 32, no: 1)
          in CALL prototype of CBL_DUMMY_ALLOC_MEM
            (file: demoproto.cpy, def:31)
          as implicitly def'd by CALL at (file: demo3.cbl, line: 15)

Finished file 'demo3.int' - pointer problems could exist

This report contains two kinds of warning:

At lines 10 and 13, the calls to CBL_ALLOC_MEM and CBL_FREE_MEM expect a pointer directly; instead they receive a group item containing a data item defined as pic x(4). This redefines the formal parameter pointer MEM-PTR as an alphanumeric data item, and is a source code problem. If pointers are expected by an API, then pointers must be provided.
At line 17, while the problem is more complicated, the reason is roughly the same as that given above. The reported problem stems from the call to CBL_DUMMY_ALLOC_MEM at line 15; this routine expects a group item containing a pointer. The calling program, however, passes a group item containing a data item defined as pic x(4). Because of the complex nature of group definitions and the redefinitions within them, this is not reported directly; Scan64 instead creates an implicit pointer definition at the data location in the program at which the call prototype expected a pointer to be defined. Any operations on the memory occupied by this implicitly defined pointer that are not consistent with pointer usage are then flagged as needing attention. The statement at line 17 is, then, an invalid modification of the memory location occupied by the implicitly defined pointer.

12.4.4 Constant and Variable Indexing

This example shows how indexing is detected and reported, and how static indexing compares with dynamic and variable indexing.

Create the intermediate code file:

cob -a demo4.cbl

In this program, the record structure is such that:

item1-1(2) and item1-3(2) are redefined by ptr2-notmod-1 and ptr2-notmod-2 respectively
item1-2(2,1) and item1-2(2,2) are redefined by ptr2-mod-1 and ptr2-mod-2

Enter the command:

cobscan64 -m demo4

This produces the report

Processing file 'demo4.int'
**Unsafe modification of pointer at (file: demo4.cbl, line: 26)
  Modifier : ITEM1-2 (file: demo4.cbl, def: 13)
  Pointer : PTR2-MOD-1 (file: demo4.cbl, def: 18)

**Unsafe modification of pointer at (file: demo4.cbl, line: 27)
  Modifier : ITEM1-2 (file: demo4.cbl, def: 13)
  Pointer : PTR2-MOD-1 (file: demo4.cbl, def: 18)

**Unsafe modification of pointer at (file: demo4.cbl, line: 27)
  Modifier : ITEM1-2 (file: demo4.cbl, def: 13)
  Pointer : PTR2-MOD-2 (file: demo4.cbl, def: 19)

Finished file 'demo4.int' - pointer problems could exist

No information is generated for prt2-notmod-[12]; however, the static indexing for item1-2 at line 26 produces a warning of unsafe modification for ptr2-mod-1, and the run-time indexing of item1-2 at line 27 produces warnings for both ptr2-mod-1 and ptr2-mod-2. As the program uses variable indexing, Scan64 cannot determine statically which pointers are really affected.

12.4.5 Constant and Variable Reference Modification

This example shows how static and dynamic reference modification is detected and reported.

Create the intermediate code file:

cob -a demo5.cbl

In this program the record p1 is accessed via a reference modification, both statically and dynamically.

Enter the command:

cobscan64 -m demo5

This produces the report:

Processing file 'demo5.int'
	**Unsafe modification of pointer at (file: demo5.cbl, line: 12)
	  Modifier : P1 (file: demo5.cbl, def: 6)
	  Pointer  : PTR2-1 (file: demo5.cbl, def: 8)

	**Unsafe modification of pointer at (file: demo5.cbl, line: 13)
	  Modifier : P1 (file: demo5.cbl, def: 6)
	  Pointer  : PTR2-1 (file: demo5.cbl, def: 8)

	**Unsafe modification of pointer at (file: demo5.cbl, line: 13)
	  Modifier : P1 (file: demo5.cbl, def: 6)
	  Pointer  : PTR2-2 (file: demo5.cbl, def: 9)

	**Unsafe modification of pointer at (file: demo5.cbl, line: 14)
	  Modifier : P1 (file: demo5.cbl, def: 6)
	  Pointer  : PTR2-1 (file: demo5.cbl, def: 8)

	**Unsafe modification of pointer at (file: demo5.cbl, line: 14)
	  Modifier : P1 (file: demo5.cbl, def: 6)
	  Pointer  : PTR2-2 (file: demo5.cbl, def: 9)

	Finished   file 'demo5.int' - pointer problems could exist

The first static reference modification at line 11 is correctly not reported; however the second static reference modification at line 12 will overwrite the first byte of ptr2-1, so is reported as a violation.

At line 13 and line 14, at syntax-check the Compiler has no idea what the contents of the variable refm might be, so any pointer within the record p1 could be changed; Scan64 therefore reports possible violations of the two pointers ptr2-1 and ptr2-2.