Chapter 19: Overview of Testing

This chapter provides an overview of TouchPoint, the enhanced testing-specific feature of Mainframe Express. See your Getting Started for a more general introduction to TouchPoint (including the terminology used in this chapter) and two tutorials that show you, step-by-step, how to use it to create regression and conformance tests and to perform Year 2000 testing.

19.1 Overview

If you need to make changes to a COBOL application, and you want to be confident that those changes have been fully tested before you release the application to production, you can use the testing features of Mainframe Express to create regression and conformance tests that both simplify and automate the testing process. You can create and edit tests for the programs within a Mainframe Express project. The tests can then be run, and the results analyzed, from within the Mainframe Express IDE.

This chapter provides an overview of those features and outlines the recommended process for using them.

Two new menus include the main TouchPoint functions - Fragment and Test Case.

In addition, a Tests tab is added to the Project View, as illustrated in Figure 19-1.

Figure 19-1: The Tests Tab

Within the Tests view, the left-hand pane represents the structure of a project's test suite:

Project
All the tests that have been created for a project are displayed in the right-hand pane when Project is selected in the left-hand pane. The figure in parentheses indicates the total number of tests that exist for the project.
Open Code Fragments
All the application tests that have been created for a project are displayed in the right-hand pane when Open Code Fragments is selected in the left-hand pane. The figure in parentheses indicates the total number of application tests for the project.

In addition, in the left-hand pane, any open fragments that you create are listed under this heading, together with the module that they are defined in.
Closed Code Fragments
All the fragment tests that have been created for a project are displayed in the right-hand pane when Closed Code Fragments is selected in the left-hand pane. The figure in parentheses indicates the total number of fragment tests for the project.

In addition, in the left-hand pane, any closed fragments that you create are listed under this heading.

19.2 Testing Your Applications

Regression tests ensure that your code has not been broken by any fixes you have made. Running a test both before and after a fix should produce the same results.

You can use TouchPoint to perform regression testing by creating and recording a test before you make any changes to your code. Recording a test creates a baseline, against which you can test. After you have made the fixes to your code, you simply run the same test(s) again. TouchPoint automatically records the output variable values and compares the new results to the baseline. The regression test fails if any values do not match. Discrepancies are highlighted when you view the results using TouchPoint's analysis options.

If you need to test whether your application conforms to a particular set of criteria, you can use TouchPoint to run conformance tests on your application. Conformance tests ensure that the application works correctly when new data is supplied to it. For example, to test your application for year 2000 conformance, you would supply it with dates after the year 2000 and monitor the results. It is important to realise that, unlike regression testing, the expected results for conformance testing cannot be generated automatically. (Recording a conformance test before you make the changes to your program would produce invalid results.) The expected, or correct, results can only be defined by someone who knows the program well, as he or she is able to use the semantics of the application to understand what results should be expected from a given set of inputs.

Conformance tests, then, work by enabling you to define expected results against which a piece of code is tested. There are two methods of doing this:

Specifying a COBOL condition
Any valid COBOL condition can be used as a validation criterion. For example, you can specify a value that a data item must be equal to, or less than. If a data item does not meet the validation criteria that you have specified, the conformance test fails and this fact is highlighted when you view the results using TouchPoint's analysis options.
Editing baseline values
When you create a fragment test, a baseline is generated which contains default values for all the output variables defined for the fragment. You can edit these values directly to create a set of expected results.

The process for using TouchPoint differs depending on whether you are performing regression testing or conformance testing and whether you create fragment or application tests. This chapter details each process.

19.3 Regression Testing

The major tasks that you are likely to perform within any general development cycle in order to run regression tests on your application are outlined below:

Create a project for the application and build the application into an executable state.
Identify the key program routines that you will need to test after you have made your changes, and define them as code fragments.
If you are going to use fragment testing, record initial values for the code fragments.
Create your tests.
Record a baseline.
Make the changes to your application.
Run the tests again and compare the results with the baseline to catch regressions.
Analyze the test results.

The details of this process differ slightly depending on whether you are using fragment or application tests.

19.3.1 Fragment Tests

If the code fragments that you have created are closed, then the most effective way of testing them is to create fragment tests. (Note that you cannot create fragment tests for open fragments.) The process that we recommend for creating fragment tests is outlined below:

Define the code fragment(s)
You define a code fragment using a TouchPoint wizard. This wizard:
- Enables you to select the group of lines that comprise the code fragment.
- Identifies any statements within the code fragment that force the fragment to be defined as open. (You cannot create fragment tests for an open code fragment.)
- Presents a list of all the data items that are used by the fragment so that you can specify which of these are input and output variables.
- Offers you the opportunity to create a set of initial values (although this can be deferred until later, if you wish).
  To perform fragment testing an initial value is needed for every data item used by the fragment. Initial values can be generated:
  - By running the application and recording the value of each data item as the fragment is encountered. (A value is only recorded the first time the fragment is encountered, before it is actually executed.)
  - Manually, either by editing the values directly, by specifying default values for numeric and alphanumeric data items or by specifying that the VALUE clause should be used.
  It may be useful to record more than one set of initial values for a fragment using a different system date, for example, or a different application scenario.
Once defined, all code fragment information is stored within the TouchPoint database. The TouchPoint database file is stored in the project directory and is named project.mvt.
Create the fragment test
Fragment tests are created using a TouchPoint wizard. You must specify a fragment name but entering a description is optional. You must nominate the fragment to be tested and the initial values to be used (if none are available you will be given the opportunity to import a set of initial values). In addition, you must specify values for the input variables. This can be done as:
- A set of individual values
- A range of values
- A series of values
- A date or range of dates
- By importing values from a file
- By reading values from a file when the test is run
As values are entered, the resulting iterations are displayed in the form of a table.

As this is a regression test, you must also specify that the test results are to be compared to a recorded baseline.
Record a baseline
Before changes are made to the code, the test should be recorded so that output values from a successful run (a baseline) are stored for later comparison against subsequent runs.

When you record baseline values the program is loaded, TouchPoint positions execution to the start of the fragment, sets the required initial values and then repeatedly executes the fragment, recording the contents of the output variables at the end of each iteration. It repeats this process until all input values have been used. For each iteration, the execution point is reset to the start of the fragment and the required initial values are reset. The fragment is run in the background and the Mainframe Express Output window displays status information (although the amount of information displayed is configurable). The baseline values are stored in a separate file with an .itr extension in tests\fragment\record).
Make the changes to your code.
Run the fragment test
Once you have made the changes to your code, the tests should be run again. When you run a test the results are stored in playback files (these have an .itp extension and are located in tests\fragment\run). At the end of the run, the Last Run and Status fields (displayed in the project view when you click on the Tests tab) are updated.
Analyze the results
Once you have run your tests, you can use TouchPoint to analyze the results. A dialog box lists the baseline values and those generated by running the test. Any data item values that do not match are flagged.

19.3.2 Application Tests

Application tests can be created for one or more fragments, open or closed. (Note that for open fragments, you cannot use fragment tests). The process that we recommend for creating application tests is outlined below:

Define the code fragment(s)
Code fragments are created using a TouchPoint wizard. This wizard:
- Enables you to select the group of lines that comprise the code fragment.
- Identifies any statements within the code fragment that force the fragment to be defined as open.
- Presents a list of all the data items that are used by the fragment so that you can specify which of these are input and output variables.
If the fragment is closed, (but not if it is open), offers you the opportunity to create a set of initial values (although this can be deferred until later, if you wish). Note, however, that initial values are not used by application tests.
Once defined, all code fragment information is stored within the TouchPoint database. The TouchPoint database file is stored in the project directory and is named project.mvt
Create an application test
Application tests are created using a TouchPoint wizard. This wizard prompts you for:
- A name and description.
- Application start parameters.
- The name(s) of the fragment(s) to be tested.
You can choose anything from one fragment to all the fragments in the application.

Note that if you have WinRunner 2000 installed, you can specify a WinRunner 2000 script at this point that will be used to drive the application test.
Record a baseline
This function records the values of the output variables as each code fragment is encountered. (The output variable value is recorded each time the fragment is executed.) These baseline values can then be used for comparison after you have made your application changes.
Make the changes to your code.
Run the test
This function records the values of the output variables as each code fragment is encountered.

At the end of the run the Last Run and Status fields are updated (these are displayed in the project view when you click on the Tests tab).
Analyze the results
A list of code fragments encountered by the test is shown. Any differences between the baseline values recorded for the output variables and the values generated when the test was run are highlighted.

19.4 Conformance Testing

Because you can only specify COBOL conditions or edit baseline values when creating fragment tests, you must create a fragment test in order to run conformance tests on your program.

The major tasks that you are likely to perform within any general development cycle in order to run conformance tests on your application are outlined below:

Create a project for the application and build the application into an executable state.
Identify the key program routines that you will need to test for conformance, and define them as code fragments.
Generate initial values for the code fragments.
Create fragment tests that test your application for conformance. You can do this either by specifying COBOL conditions which act as validation criteria, or by editing the default baseline to provide a set of "expected" results.
Run the test cases to highlight any non-conformance.
Analyze the test results.

19.4.1 Fragment Tests

The process that we recommend for creating fragment tests is outlined below:

Define the code fragment(s)
You define a code fragment using a TouchPoint wizard. This wizard:
- Enables you to select the group of lines that comprise the code fragment.
- Identifies any statements within the code fragment that force the fragment to be defined as open and will therefore prevent fragment testing.
- Presents a list of all the data items that are used by the fragment so that you can specify which of these you want to define as input and output variables.
Offers you the opportunity to create a set of initial values (although this can be deferred until later, if you wish).
To perform fragment testing an initial value is needed for every data item used by the fragment. Initial values can be generated:
- By running the application and recording the value of each data item as the fragment is encountered. (A value is only recorded the first time the fragment is encountered, before it is actually executed.)
- Manually, either by editing the values directly, by specifying default values for numeric and alphanumeric data items or by specifying that the VALUE clause should be used.
It may be useful to record more than one set of initial values for a fragment using a different system date, for example, or a different application scenario.

Once defined, all code fragment information is stored within the TouchPoint database. The TouchPoint database file is stored in the project directory and is named project.mvt.
Create the fragment test
Tests are created using a TouchPoint wizard. You must specify the fragment and the initial values to be used (if none are available you will be given the opportunity to import a set of initial values). In addition, you must specify values for the input variables. This can be done as:
- A set of individual values
- A range of values
- A series of values
- A date or range of dates
- By importing values from a file
- By reading values from a file when the test is run
As values are entered, the resulting iterations are displayed in the form of a table.

For conformance testing, you must also either:
- Specify the validation criteria for each iteration.
  For example, you can specify a value that a data item must be equal to or less than. In fact, you can specify any COBOL condition to be verified against any single iteration of the fragment test. For example, you can specify that the value of one data item must be greater than that of another data item. If the COBOL condition is not met for a particular iteration, the test status is marked as "fail" and the failure is highlighted at the analysis stage.
- Edit the default baseline.
  When you create a fragment test, each output variable is allocated a default value. Together, these values comprise the default baseline. You can elect to edit this default baseline in order to create a set of "expected" results. If you use this method, you must ensure that baseline comparison is switched on.
Run a test
Once you have created a fragment test and either specified the validation criteria to be used or edited the default baseline to create a set of "expected" results, you can run the test to validate each iteration.
Analyze the results
Once you have run your tests, you can use TouchPoint to analyze the results. A dialog box lists the input and output variables and displays a status of "pass" or "fail" for each iteration. If you have used validation criteria, an iteration is deemed to have failed if it does not satisfy the COBOL condition specified. If you have edited the default baseline to create a set of "expected" results, an iteration is deemed to have failed if the baseline value does not match the value recorded when the test was run.

19.5 TouchPoint and SmartFind-Fix/2000

If you have purchased the Micro Focus Revolve/2000 + SmartFind-Fix/2000 product and used SmartFind-Fix/2000 to investigate your application's potential year 2000 problems, you can import the information from the SmartFind-Fix/2000 report (.mdb file) into TouchPoint and automatically create a code fragment for each point of interest.

Note: The fragments that are created contain the original, not the fixed, source code.

To import the SmartFind-Fix/2000 points of interest, select Import Points Of Interest from the Fragment menu. A file selection dialog box is displayed, enabling you to select the .mdb file created by SmartFind-Fix/2000. Select the file and click Open. The Points of Interest dialog box is displayed, as illustrated in Figure 19-2.

Figure 19-2: Importing Points of Interest

Select each point of interest that you want to create a code fragment for (or press Select All if you want to create a code fragment for every point of interest) and then click on OK.

Once the code fragments have been created, you are prompted to create initial values for the fragments that you have just created. Click Yes to generate initial values by running the application. Click No if you want to generate the initial values later, or define them manually.