Analyzing Statements and Categorizing Them

Reporting on the Date Issues

Chapter 8: Analyzing Data Items and Their Year Types

The tutorial in this chapter explains year types and shows how to adjust them.

8.1 Overview

Every statement in the worksheet is now categorized according to whether or not it is a problem. Each statement has either the A-Yes or A-No category.

The next stage is to examine the data items that are used in the statements that are a problem. You need to check the year types of those data items to make sure they are assigned correctly, and that they are used consistently. If they are not, you need to adjust them. Your focus of attention depends on whether you are verifying applications or you intend to fix them, as follows:

• If you are verifying, this analysis provides a more in depth investigation and completes a thorough verification. Although a statement might appear logically correct, it might not be if the data items hold dates in different formats from those that the statement is assuming. To thoroughly verify a fix, you need to ensure that the statement handles the data items appropriately for the date formats held in those data items.

• If you intend to fix, your main focus here is to ensure that the date issues in the worksheet are complete and consistent. In particular, all the date data items that are problems must be in the worksheet and they must have the correct year types so that the fixes are generated correctly.

The aim of this tutorial is to finalize the year types of the data items in the worksheet, so that you are ready for running final reports and fixing.

In this tutorial, you:

• Use Display Filters and Show contained data items to deduce which data items are of interest

• Use Select Category to assign categories to the data items

• Use Select Year Type to adjust the year type of a data item

• Use Reports to run the Consistency checks and check for any statements containing data items with different year types

• Use the Worksheet channel analysis tool to categorize all the inconsistent statements as S-CheckMismatch

This tutorial takes about 30 minutes.

8.2 Preparation

This section explains how to run the tutorial for the first time and how to reset everything so that you can restart the tutorial.

8.2.1 Running the Tutorial for the First Time

If you are continuing immediately from the last tutorial, you can skip this Preparation section and go straight to next main section in this chapter, Deducing Categories from the Statements.

If you stopped following the tutorials, open the Tour project and the default worksheet, as before, and check that your options are set correctly for the tutorials. For details, see the sections Setting the Analysis Tools Options and Setting the Worksheet Options in the chapter Finding and Adding Dates to the Worksheet.

The worksheet should contain 53 data items and 119 statements.

8.2.2 Repeating the Tutorial

If you have already started this tutorial and now want to restart it, you need to reopen the backup worksheet that you made at the end of the previous tutorial. To do this:

1. At the worksheet, click Load from File and select the backup worksheet from the previous chapter Tour\EndofChap7.mdb, or whatever name you used for your latest backup.

   Depending on how you have Windows set up, the worksheet names are displayed with or without the .mdb extension.

2. Click No to overwrite the existing project worksheet that you no longer want, and click Yes to confirm overwriting.

   The title bar of the worksheet now shows Worksheet - Tour, and you can work on this leaving the backup EndofChap7.mdb intact in case you need it again.

3. Check that your options are set correctly for the tutorials, as described in the sections Setting the Analysis Tools Options and Setting the Worksheet Options in the chapter Finding and Adding Dates to the Worksheet.

The worksheet contains 53 data items and 119 statements.

8.3 Deducing Categories from the Statements

Now that you have decided which statements are of interest, you can conclude that the data items in those statements are accordingly of interest.

In this section, you take the statements of interest with the A-Yes category and apply that category to the data items used in those statements.

1. Go to the Statements tab.

2. Use Display Filters to show only the statements with the A-Yes category.

3. Select all the 17 statements and press F3.

   This shows the 22 filtered data items in the worksheet that are used in those statements.

4. Select the 22 data items and assign them the category A-Yes.

5. Use Display Filters to hide the data items with the A-Yes category.

   This shows the 31 filtered data items that are not of interest.

6. Select the 31 data items and assign them the category A-No.

7. Use Display Filters to redisplay the data items with the A-Yes category.

   This shows the 22 filtered data items of interest.

8.4 Reviewing Year Types

This section explains the year types that were automatically assigned to the data items, when you added them to the worksheet. If you understand year types and their implications on the data items, you can more easily decide whether a data item is already handled correctly or whether it needs fixing in some way.

How a data item should be expanded depends, among other things, on its logical date format (such as yymmdd or ccyy) and its physical format, such as numeric or alphanumeric, as defined in its PICTURE. Usually, a data item holds data of only one logical format and only one physical format. In this case, SmartFind Plus has automatically assigned year types based on rules for the data items' PICTUREs.

Year types are used to fix statements and to identify statements that involve data items with differing date formats. Year types also show in reports and audit trails.

In this section you review some of the year types assigned to data items in the worksheet.

1. Sort the data items in order of year type.

2. Examine the data items that have the year type, YY9. The 9 in the name of the year type indicates that each data item is numeric, as shown its PICTURE clause. Other year types end in X indicating that they represent non-numeric or group items. The YY in the name indicates that the data item represents a two-digit year.
   • Do all the YY9 data items hold two-digit years as numeric items?

   • Use View Source Code and Next Usage to check one or two of them. Examine the PICTURE clauses and examine how the data items are used. In fact, all hold two-digit years, except P-YEARS-TO-REPORT, which holds a duration.

   • Double click P-YEARS-TO-REPORT and change its year type to NOTDATE.

3. Examine the data items that have the year type _____9. The underscores in the name of the year type indicate that the date format is unknown, and in this case represents 5 or 6 digits. For example, the data definition of YR-START-DATE does not indicate whether the date format is yymmdd or ddmmyy.
   • What date format does YR-START-DATE hold?

   • Examine the code in the Source View. The data item YR-START-DATE is moved into RD-START-DATE-N, whose data definition shows the date format to be yymmdd. From this, you know that the year type of YR-START-DATE should be YYMMDD9.

     The reason that YR-START-DATE was not automatically assigned this year type is that RD-START-DATE-N is not in the worksheet, because it is not a problem in its own right.

   • Double click the data item and assign it the year type YYMMDD9.

4. Examine the data items WS*DATE, which also have the year type ______9. Notice that they are used in MOVE statements with a data item YYMMDD.

   You might wonder why the WS*DATE data items do not automatically have the correct year types. This is because the year types are assigned as each data item is added to the worksheet. Data items added early in the adding process might not receive full year types, which then become obvious when data items are added later in the process. In this case, the WS*DATE items were added early on and YYMMDD was added later on.

   Now that all the data items are added, you can resolve some of the remaining year type ambiguities by running a year type assignment, as follows:

   • Select all the data items with underscores in their year types.
   • Click Assign year type.

   Notice that the data items WS*DATE now have the year types YYMMDD9.

5. Examine the data items that have the year type, _____9, which show as 9(5) in the Pic column, such as MONTHS-ELAPSED-SINCE-JAN-1900.
   • What types of dates do these data items hold?
   • Review the relevant code in the Source View.
   • In fact, both data items hold a number of months and also record the century. There is no year type for this date format.
   • Change the year type for these data items to MANUAL.

6. Examine the data items with the year type CCYY9.
   • Do both data items really hold century information?
   • Review the relevant code in the Source View.
   • YYYY does hold the century and so is correctly assigned CCYY9.
   • TMP-END-YY does not hold the century. It holds the year without the century, even though it can hold four digits.
   • Change the year type for TMP-END-YY to YY9.

All the data items of interest are now assigned year types.

8.5 Checking for Mismatched Year Types

If a statement uses data items that have different year types, there might be a problem. If you are verifying, the fix in the statement might be incorrect. If you are intending to fix the statement, the fix that is automatically for the statement might be incorrect.

In either case, it is possible that one or other of the data items is assigned the wrong year type. You need to check for these mismatched year types and either confirm the mismatches as OK or correct the offending year types.

In this section, you run a Consistency check to find the statements containing data items with different year types.

1. Click Reports and click Consistency checks.

2. In the drop-down list at the top, select Check the year types of impacted data items in the worksheet.

3. Set the options to run the consistency check only on the statements and data items of interest, as follows:
   • Click Options.
   • Click Default values.
   • In Category towards the bottom, click .
   • Click Clear.
   • Check A-No.
   • Click OK twice.

4. Click Execute Consistency Check.

   This produces a report of the inconsistencies and a set of the statements that contain data items with mismatched year types. The report is shown in Figure 8-1.

5. The Set View shows that there are 11 inconsistent statements. Minimize the set for now.

Figure 8-1: Consistency Check

6. If necessary, enlarge the consistency checks window, so that you can see the report more clearly. The status line at the bottom of the report window shows that there are 14 inconsistencies.

7. The first inconsistency shows that the data item YR-START-DATE, which has the year type YYMMDD9, is used in three ADD statements with the data item TMP-YYYY, which has the year type MANUAL. Since the two data items have different year types these statements are considered to conflict.

   Why is the MANUAL year type used for TMP-YYYY?

   If you cannot remember why, look up the notes for TMP-YYYY. To do this, right click the data item in the first line of the conflict and click Go to details, and look in the Notes column. Since this data item needs handling differently in different statements, the MANUAL year type is correct and the mismatch is valid.

   A later section examines the inconsistencies in more detail.

8. Notice that you can use Save to File to save the report to a text file, which you can redisplay later or print or pass to your colleagues.

8.6 Categorizing the Mismatched Year Types

You need to examine the mismatches, and you can do this by systematically going through the report and recording your decisions and changes in the worksheet. However, it is easier to do this in the worksheet, in which case you first need to record these inconsistencies in the worksheet.

This section updates the categories of the mismatching statements, using the Worksheet channel.

1. If the Analysis Tools window is not displayed, click Analysis Tools on the Year 2000 menu.

2. Display the Set View "Mismatched date types", by clicking it on the Window menu. Drag this window so that it is next the Analysis Tools window.

3. Drag the set on to Worksheet channel as follows:
   • Click anywhere in the set "Mismatched date types".
   • Click , which is the handle for the set, to select the whole set.
   • Drag the set by its handle and point to Worksheet channel in the Analysis Tools window.
   • When the cursor changes to to indicate a set operation, drop the set onto Worksheet channel.

4. On the Operation tab:
   • Click Add set.
   • Make sure that Update existing items is checked.

5. On the Options tab:
   • Make sure that Note is unchecked.
   • Check Category.
   • Click to specify the category.
   • Click Clear to uncheck the existing categories.
   • Check the category S-CheckMismatch.

6. Click OK twice to run the worksheet channel and categorize the statements in the set.

7. Use Display Filters on the Statements tab to filter the worksheet to show the statements with the new category, S-CheckMismatch.

There are now 11 statements with mismatching year types.

8.7 Resolving Mismatched Year Types

Now that the statements with mismatched year types are categorized in the worksheet, you need to examine them one by one and resolve the mismatches.

1. Sort the statements into order.

2. Check the inconsistency in the first three ADD statements.
   • What are the inconsistent year types? Use F3 to find out.
   • Why does TMP-YYYY have the MANUAL year type? Look in the Notes column.

   MANUAL means that the year type for the data item needs to be manually assigned for each statement, since the data item holds different date formats in different statements. This means that this inconsistency is valid.

   Use Select Category for these statements to:

   • Add the category S-DateMsmatchOK.
   • Remove the category S-CheckMismatch.

     Note that you must remove the old category and after the new one, because removing causes the statement to be hidden from the active filter view.

   • There are now 8 filtered statements with the S-CheckMismatch category.

3. Check the inconsistency in the statement starting COMPUTE SUB2.
   • What year type does SUB2 have? Use F3 to find out.
   • Does the statement store a year value in SUB2? Examine the statement to find out.

   The data item SUB2 has the year type NOTDATE, but in this statement it holds a two-digit year. This means that the consistency check has uncovered a genuine problem. The year type of SUB2 needs to allow for non-dates and for years.

   • At the Data items tab, double click SUB2 and change its year type to MANUAL.

   • At the Statements tab, change the category of the statement by adding S-DateMsmatchOK and then removing S-CheckMismatch.

     Note that you must add the new category before removing the old one, because removing causes the statement to be hidden from the active filter view.

     There are now 7 filtered statements.

4. Normally, you would examine each mismatching statement one by one, examining the relevant source code to determine whether the mismatch is actually a problem or not. In this demonstration, all the other inconsistencies are not a problem, and so you can change their categories, as follows:
   • Change the category of the statements by adding S-DateMsmatchOK and then removing S-CheckMismatch.

     There are now no filtered statements.

5. Run the consistency check again, and on Options make sure that the categories to ignore are set to S-DateMsmatchOK and A-No.

   This shows that all the inconsistencies are resolved.

6. Tidy up by closing the consistency report and the set.

8.8 Before Continuing

The worksheet should now contain 53 data items and 119 statements. 17 of the statements have the category A-Yes and 11 have the category S-DateMsMatchOK.

The worksheet is persistent and is continually saved. However, it is good practice to save a backup of the worksheet from time to time.

1. At the worksheet, click Save to File.

2. Specify the filename, EndofChap8, for the backup worksheet for this chapter.

   If you already have a backup with this name, give a new name such as EndofChap8-1.

3. When asked whether to make the saved worksheet your current worksheet, reply No, because the saved worksheet is just a backup and you need to stay working in the project worksheet.

You can close SmartFind Plus, if you want to stop for now. You can then continue with the next tutorial some other time.

Analyzing Statements and Categorizing Them

Reporting on the Date Issues