Finding Dates and Reviewing the Worksheet

Analyzing Data Items and Their Year Types

Chapter 7: Analyzing Statements and Categorizing Them

The tutorial in this chapter shows how to analyze a statement to determine whether it is genuinely a problem, and how to set the category accordingly.

7.1 Overview

The worksheet now contains all the potentially date-related problems. These are the points of interest. Each point of interest has one or more categories assigned to it, indicating the reason why it is of interest.

The next stage in the process is to systematically examine each statement in the worksheet, exploring the related source code and the data items involved. When you understand the statement fully, you make comments and add reminders in the Notes column, as necessary, and you categorize the statement according to whether you are verifying the application or intending to fix it:

• If you are verifying, categorizing is a fundamental part of the process. Here your objective is to record the level of compliance. In this case, you might categorize the statement according to what issues have been fixed correctly, what fixes have been missed, and what issues need further investigation.

• If you intend to fix, categorizing is not essential but can be a useful aid for focussing on the problems to fix. You might categorize statements according to whether they definitely need fixing, might need fixing, or definitely do not need fixing.

You can categorize points of interest in any way that suits your organization or the application. For example, your organization might need to know whether statements are windowed or fixed using data expansion. SmartFind Plus comes with a set of categories, but your organization can set up its own. The facility to do this is reserved as an administration task, since we recommend that you provide one defined set of categories for your work group to ensure that everyone categorizes consistently.

During this analysis process, you can also examine the data items used in the statements, and possibly adjust the year types and categories assigned to them. To simplify the learning process, examining data items is left until the next tutorial. With a real application, however, you would probably examine and make decisions on the relevant data items at the same time as the statements.

The aim in this tutorial is to categorize every statement according to whether it is a problem or not. The A-Yes category is used to indicate that the statement is logically unsound and needs fixing. A-No is used to to indicate that the statement is OK and does not need fixing, perhaps because it has already been fixed correctly.

In this tutorial, you analyze each statement in the worksheet and assign it an A- category using:

• Display Filters to show only one category of statement in the worksheet

• View Source Code to examine a statement in the context of the source code

• Show contained data items to examine the data items used in a statement

• The Analyze a statement analysis tool to help to view all the data items and statements related to the statement in question

• Select Category to assign a category

This tutorial takes about 30 minutes.

7.2 Preparation

This section explains how to run the tutorial for the first time and how to reset everything so that you can restart the tutorial.

7.2.1 Running the Tutorial for the First Time

If you are continuing immediately from the last tutorial, you can skip this Preparation section and go straight to next main section in this chapter, Reviewing the False Positives.

If you stopped following the tutorials, open the Tour project and the default worksheet, as before, and check that your options are set correctly for the tutorials. For details, see the sections Setting the Analysis Tools Options and Setting the Worksheet Options in the chapter Finding and Adding Dates to the Worksheet.

The worksheet should contain 53 data items and 119 statements.

7.2.2 Repeating the Tutorial

If you have already started this tutorial and now want to restart it, you need to reopen the backup worksheet that you made at the end of the previous tutorial. To do this:

1. At the worksheet, click Load from File and select the backup worksheet from the previous chapter Tour\EndofChap6.mdb, or whatever name you used for your latest backup.

   Depending on how you have Windows set up, the worksheet names are displayed with or without the .mdb extension.

2. Click No to overwrite the existing project worksheet that you no longer want, and click Yes to confirm overwriting.

   The title bar of the worksheet now shows Worksheet - Tour, and you can work on this leaving the backup EndofChap6.mdb intact in case you need it again.

3. Check that your options are set correctly for the tutorials, as described in the sections Setting the Analysis Tools Options and Setting the Worksheet Options in the chapter Finding and Adding Dates to the Worksheet.

The worksheet should contain 53 data items and 119 statements.

7.3 Reviewing the False Positives

In the previous tutorial, the Verify - all research analysis tool searched for all the potentially date-related problems. Despite the specialized analysis criteria used to identify potential date problems, some of the identified statements do not require remediation for the year 2000. These are known as false positives.

You now need to examine the false positives to see if you can disregard any of them, to help focus on the statements that are genuinely a problem. The tool started this process, by running the component tool Verify - false positives categorization, which identified any likely false positives in the worksheet and categorized them as follows:

• R-BadFormat. This applies to data items whose PICTUREs cannot contain dates and also applies to the statements that use those data items. For example, a data item defined with a decimal point, a debit/credit or a currency symbol is unlikely to hold a date.

• R-NonYearType. This applies to data items whose date formats do not represent years or periods that span the millennium, but represent months or days within one year or are not dates at all. This covers data items with year types such as NOTDATE, MM9, DDX, MMDD9 and CC9. For example, a data item with a year type MM9 is a 2-digit numeric item representing a month, and since it does not span a year (or millennium) is unlikely to be a problem. Contrast this with a 3-digit item MMM, which might be used to hold the number of months since the beginning of 1900 and so would span the millennium.

• R-NonDateName. This applies to the data items whose names look unlikely to hold dates, such as a data item ending in WEIGHT or LENGTH or COUNT.

In this section, you review the false positives in the context of the source code.

1. Filter the worksheet to show just the statements that are categorized as potentially false positives. To do this:
   • Go to the Statements tab of the worksheet.
   • Click Display Filters.
   • Click Default values.
   • Click Categories.
   • Scroll down the list of categories and check R-BadFormat, R-NonDateName, and R-NonYearType.
   • Click OK twice.

   The worksheet is now filtered and shows 16 statements. Notice that the information bar towards the top of the worksheet shows Filtered (16) .

2. Sort the statements into order. You now work through these statements one by one, examining each one to determine whether it is a false positive, as described in the next steps.

3. COMPUTE MONTHS-ELAPSED-SINCE-JAN-1900 = ...
   • Right click the above statement in the worksheet.
   • Click Show contained data items.
   • Widen the Categories column, if necessary, to show all the categories.
   • Which data item is categorized as not a year?

   Only MM has the category R-NonYearType. Since some of the data items represent years, this means that the statement is not a false positive, and so you must leave this statement intact for now.

4. COMPUTE MMM-CONTRACT-TERM-TO-DATE = ...
COMPUTE SUB2 = (YYMMDD-N + 9999) / 10000

   Go back to the Statements tab. Review these other COMPUTE statements in the same way. They are not false positives and you can skip over them.

5. IF P-FIRST-DD = 0 OR > 31 OR
   • Click the statement.
   • Click View Source Code.
   • Does the statement refer to years?

   Since it refers to only days and months, it is not a problem. It is a false positive. Categorize the statement as not a problem as follows:

   • Select the statement in the worksheet at the top.
   • Click Select Category.
   • Click A-No.
   • Click Add.
   • Check that the new category is added to the Category column.

6. IF P-LAST-DD = 0 OR > 31 OR

   This statement is not a problem in the same way. Categorize it as A-No in the same way.

7. PERFORM VARYING SUB2
   • Click the statement so that it shows in the Source View at the bottom.
   • How is the data item SUB2 used?
   • It is used as a counter and is not a date.
   • Click P-CONTRACT-LENGTH in the Source View.
   • Does this data item hold a date?
   • Click Previous Usage twice.
   • Notice that the value moved into the data item is not a date.
   • Categorize the PERFORM statement as A-No and not a problem, in the same way as before.

8. Normally, you would review all the statements in a similar way. In this case, the other statements might contain some data items that hold dates and so they might be of interest. They might not be false positives and so you do not need to do anything with them yet.

9. Close the Source View, by clicking View Source Code again.

7.4 Analyzing the SmartFind Statements

In the previous tutorial, the tool Verify - SmartFind research ran automatically, and found the majority of points of interest, adding them to the worksheet with the category R-SF Wizard.

The next stage is to systematically examine each of these statements to decide whether or not it is a problem before you finally assign the appropriate category. During this process, you can add notes to the statement and its data items and potentially adjust the year types of the data items, but for now, year types are left until the next tutorial.

In this section, you examine the statements with the category R-SF Wizard and assign them either A-Yes or A-No.

1. Filter the worksheet to show just the statements that are categorized as R-SF Wizard, using Display Filters in the same way as before.

   The worksheet now shows 83 filtered statements.

2. Sort the statements into order, and examine the following statements as suggested..

3. ADD 1900 P-START-YEAR GIVING TMP-YYYY
   • Click the statement and click View Source Code to see the statement in context of the source code. This statement appears to be adding a century to a 2-digit year to make a 4-digit year, and is not windowed.

   • Display the data items in the statement by clicking the statement in the worksheet at the top and pressing F3.

   • Click P-START-YEAR in the worksheet at the top.
     • What is its data definition shown in the source at the bottom?
     • Does this data item genuinely hold a 2-digit year?
     • Click Next Usage several times.

   • Click TMP-YYYY in the worksheet.
     • What is its data definition shown in the source at the bottom?
     • Does the data item hold 4-digit year or a 6-digit date?
     • Click Next Usage several times.
     • Click in the data item's Notes column.
     • Enter the note "sometimes 4-digit year and sometimes 6-digit date".
     • Double click in the TMP-YYYY.
     • Select MANUAL as the year type, which is explained in the next tutorial.

   • Go back to the Statements tab and set the statement's category to A-Yes, in the same way as before.

4. ADD 1900 TO YYYY
ADD 2000 TO YYYY
IF YYYY > 50
   • Click the first of these statements in the worksheet at the top, and review its source in the code at the bottom. This statement is part of windowing code that is fixing the year 2000 problem.

   • Work through all the similar statements, by clicking the statement in the worksheet at the top, pressing down arrow and watching the corresponding source code at the bottom.

   • Select these statements and set their category to A-No.

5. IF MMMM-CONTRACT-TERM-TO-DATE <= 6

   From the source code, this statement appears to be referring to a duration rather than a date and is not a problem. Set the statement's category to A-No.

6. IF P-FIRST-DD = 0 OR > 31 OR
IF P-LAST-DD = 0 OR > 31 OR

   You already categorized these statements as A-No, because they did not involve any year-related data items, and so you can skip these statements.

7. IF P-YEARS-TO-REPORT = 0 OR > 10 OR

   This statement involves durations rather than dates and is not a problem. Set the statement's category to A-No.

8. IF RD-END-YY < P-START-YEAR OR
IF YYMMDD <= RD-END-DATE

   You can see from the source that these statements are comparing years and so are definitely a problem. Set the statements' categories to A-Yes.

9. MOVE "190427" TO AC-CONTRACT-END-DATE

   This statement contains "19" but this is the year not the century and so is not a problem. Set the statement's category to A-No. You could also add a note saying "Literal is a year not a century".

10. Normally you would go on through every statement methodically, categorizing and adding notes as appropriate. In the interests of speed, assign the remaining statements with these categories.

    Statement that starts with	Category
    PERFORM READ-CONTRACT	A-Yes
    PERFORM VARYING TMP-YY	A-Yes
    SUBTRACT 20000	A-Yes
    SUBTRACT 5	A-Yes
    SUBTRACT P-START-YEAR	A-Yes
    SUBTRACT RD-END-DATE-N	A-Yes
    SUBTRACT MMMM	A-Yes
    PERFORM VARYING SUB* (3 statements)	A-No
    READ and START (many statements)	A-No
    SUBTRACT CN-CONTRACT-TERM (many statements)	A-No
    SUBTRACT SUB2	A-No
    WRITE (many statements)	A-No

    All the statements with the category R-SF Wizard are now categorized as either A-Yes or A-No.

11. Use Display Filters to display the statements with the A-Yes category. There should be 10 statements.

There are 10 statements with the A-Yes category.

7.5 Categorizing the Remaining Statements

In the previous tutorial, the tool Verify - supplementary research ran automatically, and found the more obscure points of interest, adding them to the worksheet with a category indicating why it is of interest, such as R-AddNegLitl for adding a negative literal.

The next stage is to systematically examine each of these statements to decide whether or not it is a problem, in much the same way as before, adding notes and categorizing as appropriate.

In this section, you examine the statements that you have not yet examined and categorize them.

1. Hide all the statements with A categories, so that you can easily focus on the remaining statements to be examined.
   • Go to the Statements tab of the worksheet.
   • Click Display Filters.
   • Click Default values.
   • Click Hide items.
   • Click Categories.
   • Check A-Yes and A-No.
   • Click OK twice.

   The worksheet now shows 33 filtered statements.

2. Set the option to dynamically refresh the displayed worksheet, so that as you assign an A category, that statement is hidden from view. To do this:
   • Click Options.
   • Check Enable active filtering.
   • Click OK.

3. Sort the worksheet into statement order.

4. Normally you would systematically go through the statements, examining them individually and assigning the appropriate category. In the interests of speed, assign the statements with these categories in the following table.

   Statements	Category
   ADD -*0000 TMP-YYYY (3 statements)	A-Yes
   ADD P-START-YEAR (1 statement)	A-Yes
   COMPUTE (3 statements)	A-Yes
   ACCEPT (2 statements)	A-No
   ADD 1 (4 statements)	A-No
   IF (2 statements)	A-No
   MOVE (all statements)	A-No

   The filtered worksheet should now be empty.

5. Uncheck filtered to show the full worksheet again. Notice that all the statements have an A category.

6. Display just the A-Yes category, using Display Filters. Remember to click Show instead of Hide.

   There are 17 filtered statements. These are the statements that need fixing.

All the statements in the worksheet are now categorized as A-Yes or A-No.

7.6 Analyzing a Statement in Depth

Many of the statements are relatively straight forward to understand and establish whether they are a problem or not. Sometimes, however, a statement has wider implications because of the data items it uses and the way those data items are used elsewhere. To help in these situations, the Analyze a statement analysis tool produces a set of all the data items and statements associated with the statement in question.

This section examines one PERFORM statement in depth, using Analyze a statement.

1. Make sure that you can see both the Analysis Tools window and the worksheet. If necessary, drag the worksheet to one side.

2. Click the statement PERFORM READ-CONTRACT-REC.

   Drag the statement and point to the name Analyze a statement in the Analysis Tools window.

3. When the cursor changes to to indicate a set operation, drop the set onto Analyze a statement.

4. Click OK.

   This produces a set of 11 points of interest and displays them in a Set View, as shown in Figure 7-1.

Figure 7-1: Set View

5. The left pane lists the points of interest in the set. In this case, these are the data items in the original PERFORM statement together with the statements involving those data items.

   Some set entries are in gray and these are in the worksheet because they are potentially a problem. The entries in black are not in the worksheet, because they are not a problem in their own right. The black entries are useful here to help in analyzing the PERFORM statement in question.

   The right pane contains a history of the set, and shows how the set was created. In this case, this shows that the set originated from a statement in the worksheet and used the Analyze a statement tool.

   The status line at the bottom shows that there are 11 points of interest in the set.

6. You can drag the border of the Set View, to enlarge or shrink it, and you can also drag the divider between the two panes to suit. For now, lengthen the Set View by dragging its top or bottom border.

7. Use View Source Code in the Set View to show the full PERFORM READ-CONTRACT-REC statement.

8. Use Definition of Usage and Next Usage to examine the statement and its data items. From the source, you can see that:
   • The WS*DATE data items hold dates in the format year-month-day

   • The data items CONTRACT-ENDED and CONTRACT-EOF do not hold dates

   • The statement is a problem since the dates do not include any century information

9. Close the Set View and do not save the set.

Since the PERFORM statement already has the category A-Yes you do not need to do anything further.

7.7 Before Continuing

The worksheet should now contain 53 data items and 119 statements. 17 of the statements have the category A-Yes.

The worksheet is persistent and is continually saved. However, it is good practice to save a backup of the worksheet from time to time.

1. At the worksheet, click Save to File.

2. Specify the filename, EndofChap7, for the backup worksheet for this chapter.

   If you already have a backup with this name, give a new name such as EndofChap7-1.

3. When asked whether to make the saved worksheet your current worksheet, reply No, because the saved worksheet is just a backup and you need to stay working in the project worksheet.

You can close SmartFind Plus, if you want to stop for now. You can then continue with the next tutorial some other time.

Finding Dates and Reviewing the Worksheet

Analyzing Data Items and Their Year Types