Portability Guidelines

National Language Support (NLS) Demonstration

Chapter 7: Internationalization Support

This chapter describes the internationalization support provided by Server Express.

7.1 Introduction

Server Express offers several levels of internationalization support:

• National Language Support (NLS) for single-byte character sets.
  
  
• Double-Byte Character Set (DBCS) support
  
  
• Support for translation between ASCII and EBCDIC

7.2 National Language Support (NLS)

NLS provides a means of adapting your program to the character code set, collating sequence, and editing symbols associated with a particular country, without your program knowing these in advance. This facility is also useful in English-speaking countries for handling codepages, collating upper-case and lower-case letters correctly and selecting the appropriate currency symbol.

Examples of language and character sets that NLS accommodates are:

When you run a program compiled with the NLS directive, it behaves according to the language, country and character set specified in the language environment variable.

7.3 Double-Byte Character Set (DBCS) Support

DBCS support enables you to create native language applications that use double-byte character strings, for countries such as Japan, China and Korea.

Server Express includes some support for handling double-byte characters. A fully DBCS-enabled and localized version of Server Express is available in Japan.

You can create DBCS applications using existing tools and syntax, such as ACCEPT/DISPLAY with PIC X. However, Server Express also supports additional COBOL syntax used in DBCS environments. This syntax is described in your Language Reference - Additional Topics. To enable this support, use the DBCS, NCHAR or JAPANESE Compiler directive.

The library routine CBL_GET_OS_INFO enables your application to detect the character encoding for Far Eastern countries. See the chapter Library Routines.

7.3.1 DBCS Transparency Support

DBCS transparency support is available in Object COBOL on many platforms.

DBCS transparency support enables users of your applications to enter, store, manipulate, display and print single-byte and double-byte character strings in their native language. In addition, you can use tools such as Animator, Editor and Screens to create and maintain application programs that support your native language.

Editing operations such as backspace, delete and insert, as well as cursor movement, are by byte and not by character. For example, a delete operation needs two keystrokes to delete both bytes of a DBCS character. After the deletion of the first byte (half a DBCS character) the remaining characters may appear corrupted, but if you press the delete key a second time the other byte of the character is deleted with no corruption at all.

However, with some tools and on some terminal environments, editing operations and cursor movement are by DBCS character and not by single bytes.

DBCS transparency support is restricted in the following tools, which are described in your Utilities Handbook:

• Data File Editor - This component is not DBCS transparent. When editing a file you must enter DBCS characters in hexadecimal mode.
  
  
• Editor - You cannot split DBCS characters across margin boundaries. Also wrapping of paragraphs with DBCS is undefined.
  
  

7.3.2 Double-Byte Line Draw and Graphics Characters

The routines CBL_GET_SCR_LINE_DRAW and CBL_GET_SCR_GRAPHICS support the line draw and graphics characters used in all the environments supported by Object COBOL. See the chapter Generic Line Drawing in your Programmer's Guide to User Interfaces.

Some demonstration programs supplied with this system contain IBM PC encoded line draw characters. Under some environments, you need to amend these characters in the COBOL source before using these programs.

7.3.2.1 Additional Syntax to Support DBCS

Object COBOL supports the following items, which you can use in DBCS applications in addition to the standard PIC X support:

• PIC G and PIC N double-byte fields - For full details see the chapter Double-Byte Character Set Support in your Language Reference - Additional Topics.
  
  
• DBCS directives - The Compiler directives DBCS, DBCSSOSI, DBCHECK and DBSPACE control the behavior of the PIC G and PIC N functionality. For further details refer to the chapter Directives for Compiler in your Server Express User Guide.
  
  
• Japanese support - Additional syntax is available for Japanese environments and is described in the chapter Micro Focus Extensions for Double-Byte Character Support in your Language Reference - Additional Topics.

Note: When performing a Format 4 or Format 5 ACCEPT statement into an identifier of class DBCS, you must enter double-byte data only. See the chapter Double-Byte Character Set Support in your Language Reference - Additional Topics.

7.3.3 Support for Translation between ASCII and EBCDIC

Object COBOL provides the CHARSET Compiler directive and the _CODESET program for converting between ASCII and EBCDIC. It also provides some preconfigured _CODESET programs supporting ASCII and EBCDIC codepages for several Far Eastern languages and German.

7.4 Enabling Internationalization

To use NLS or DBCS transparency support:

• Your operating system must support the required character sets.
  
  
• Your terminals must support the required character sets.
  
  
• You need to set up the environment variable for the required language, country and character set.
  
  
• If your program uses full screen ACCEPT operations, you might need to set the range of data keys to be accepted in the adisctrl database. See the chapter Adis Configuration Utility (Adiscf) in your Programmer's Guide to Creating User Interfaces for details.

Server Express relies on UNIX to provide the appropriate national language tables for the language, country and character set that you specified. These are known as locales.

7.5 Setting Up the Environment for NLS and DBCS

Before you run a program that uses NLS or that uses DBCS support on UNIX, you must define your locale in the language environment variable. The locale is the combination of language, territory and codepage.

LANG=language[_territory[.codepage]] 
export LANG

where the parameters are:

Refer to your operating system documentation for valid territory and codepage values.

LANG=fr_FR
export LANG

7.6 Creating Programs That Use NLS

To use the NLS facility in your program, you must specify the NLS Compiler directive when you compile your program. By default, the NLS directive is not set. Throughout the rest of this chapter, a program compiled using the NLS directive is referred to as NLS-enabled.

You must not use the following syntax in an NLS-enabled program:

• In the Object-Computer paragraph, the phrase:

  PROGRAM COLLATING SEQUENCE IS alphabet-name
  

• In the Special-Names paragraph, the phrases:

  alphabet-name IS
  STANDARD-1
  NATIONAL
  literal-1 THRU literal-2
  CURRENCY SIGN IS literal
  DECIMAL-POINT IS COMMA
  

• In the Procedure Division, in MERGE or SORT statements, the phrase:

  COLLATING SEQUENCE IS alphabet-name.

You can NLS-enable any program in your application. Thus some programs within an application might use NLS facilities, while others might not. See the section Mixing Programs with and without NLS later in this chapter for details.

7.7 Running Your NLS-Enabled Program

You run NLS-enabled programs in the same way as you run programs without it.

The run-time system initializes the NLS facility only once during an application's run; that is, when it encounters the first NLS-enabled program. It uses the LANG environment variable to determine the language environment to set up for this program. The run-time system uses the same language environment for any subsequent programs that NLS-enabled. See the section Mixing Programs With and Without NLS later in this chapter for details.

If an error occurs during initialization, for example the language specified in the LANG environment variable is not supported, the run-time system issues an error message and terminates its run. Full details on NLS error messages are in your Error Messages.

You must not use indexed files with variable length records when the NLS directive is set.

You cannot use indexed files created with one collating sequence in a file language environment that uses a different collating sequence. If you do and you specified file status bytes in your program, the COBOL system returns the file status 9/45. If you did not specify file status bytes in your program, the run-time system issues an error message.

When you run an NLS-enabled program, the language specified in the LANG environment variable defines the behavior of:

• String comparisons, including comparisons of alphanumeric or group items
  
  
• Class condition tests (ALPHABETIC, ALPHABETIC-UPPER, ALPHABETIC-LOWER, NUMERIC and user-defined CLASS conditions), appropriate to national and multinational character sets
  
  
• Key comparisons in indexed sequential files
  
  
• Comparisons performed as part of a SORT or MERGE statement
  
  
• Case conversion
  
  
• Collating sequences operations, for example, for the correct handling of accented characters in West European languages
  
  
• Editing and de-editing moves
  
  
• The intrinsic functions Numval and Numval-c

Other language-dependent features, such as the symbols used to denote the decimal points and the currency character , also appear in the format of the specified language. However, in languages that define the currency sign as trailing, this has no effect and the usual COBOL rules are observed.

Certain NLS definitions have characters other than the ASCII characters 0-9 defined as numerics. You cannot use the ACCEPT syntax to enter such characters into numeric picture strings, nor can they be used in numeric operations. In all NLS operations in the COBOL environment, a numeric item must be formed only from the ASCII digits 0-9, with or without the ASCII operational signs "+" or "-". There is no means of automatically converting the NLS representation to the ASCII equivalent.

It is possible to enter European modifying characters into numeric ACCEPT fields. These are accepted as zero.

Note that the values assigned to figurative constants, for example LOW-VALUES, are not changed by using NLS features.

Unpredictable results occur in the following circumstances when you use the NLS facility:

• An item in a conditional expression contains the value LOW-VALUE.
  
  
• An indexed file key contains the value LOW-VALUE.
  
  
• An elementary item of a group that is part of a conditional expression is not USAGE DISPLAY.
  
  
• An elementary item of an indexed file key is not USAGE DISPLAY.

You can also use the Adis Flip Case Control key when using NLS characters. However, if you attempt to convert a European character to upper case using this key, the character will be replaced by spaces if it has no upper case equivalent.

See your Server Express User Guide for full details on running programs.

7.7.1 String Comparisons

For NLS-enabled programs, the run-time system invokes the CBL_NLS_COMPARE routine to compare strings. This routine is also called for alphanumeric comparisons if the program uses the intrinsic functions Max, Min, Ord-max or Ord-min on alphanumeric operands.

During a MOVE operation of one alphanumeric item to another which is longer, padding with spaces occurs. Similar padding is also implied before the comparison of two such items. In both cases, an ASCII space is assumed.

7.7.2 Class Condition Tests

For NLS-enabled programs, the run-time system invokes locale-specific tests when it needs to carry out class condition tests. These tests determine if a string of information is in ALPHABETIC, ALPHABETIC-UPPER, ALPHABETIC-LOWER or NUMERIC format. The numeric test always tests that all characters are in the range of ASCII 0-9.

7.7.3 Key Comparisons in Indexed Sequential Files

The run-time system uses the CBL_NLS_COMPARE routine for key comparisons associated with NLS files.

If the logical filename of an indexed sequential file is preceded by the five characters "%NLS%", the file is treated as being keyed according to the collating sequence specified in the LANG environment variable. This applies whether "%NLS%" precedes the logical filename before the filename is resolved using environment variables or after it is resolved. This also applies only if the program that OPENs the file was compiled with the NLS directive set.

If the logical filename is preceded by "%NLS%" after it is resolved using environment variables, but the program that OPENs the file was compiled without the NLS directive set, the OPEN fails and returns a run-time system error.

If your file contains variable length records, the NLS collating sequence is not used.

For details on assigning logical filenames to be resolved using environment variables, see the Filenames chapter in your File Handling.

7.7.4 SORT and MERGE Comparisons

If an NLS-enabled program performs a SORT or MERGE operation, it automatically uses NLS key comparisons.

The logical filename of the work file must be preceded by the characters "%NLS%" in the ASSIGN statement, and the program that opens it must be NLS-enabled. This is as described in the previous section for indexed sequential files.

The run-time system invokes the CBL_NLS_COMPARE routine for all key comparisons used in SORT or MERGE operations.

7.7.5 Case Conversion

If an NLS-enabled program performs a case conversion, either by using the intrinsic functions Upper-case and Lower-case, or by calling the library routines CBL_TOUPPER and CBL_TOLOWER, the run-time system invokes a routine to change the case of national characters correctly.

7.7.6 Collating Sequence Operations

For NLS-enabled programs, a collating sequence appropriate to the locale is used. If the program performs a collating sequence operation by using the intrinsic functions Char and Ord, then this special collating sequence is used.

7.7.7 Editing and De-editing Moves

If an NLS-enabled program performs an editing or de-editing move, then the decimal point and thousands separator appropriate to the national language locale are used. Also, the currency symbol used is the first character of the currency symbol of the locale territory.

7.7.8 Intrinsic Functions Numval and Numval-c

If an NLS-enabled program attempts to convert a number or a monetary value in a display item to a numeric value by using the intrinsic functions Numval or Numval-c, then the run-time system uses the decimal point and thousands separator appropriate to the locale. Also, if no second argument is supplied for Numval-c, then the currency symbol used is the currency symbol of the locale territory.

7.7.9 User Interfaces

If you require information concerning the language environment you are using, you can access the NLS routines supported by your COBOL system. Full details on all of these routines can be found later in this chapter.

You also can access the UNIX operating system routines, such as nl_langinfo(), though these routines are not portable to other environments. Full details on these routines can be found in your operating system documentation.

7.8 Mixing NLS-Enabled and Non-NLS-Enabled Programs

If an application comprises more than one program, you only need to set the NLS Compiler directive for those programs that you want to be NLS-enabled. Both NLS-enabled programs and non-NLS-enabled can call programs NLS-enabled and non-NLS-enabled programs.

An NLS-enabled program has no particular language or locale associated with it. It uses the application locale, which is initialized along with the NLS facility when the first NLS-enabled program in the application is called.

Once the application locale is set, it is used for all subsequently called NLS-enabled programs. You cannot change the language environment after this, even if you change the setting of the LANG environment variable. You must therefore ensure that all NLS-enabled programs use the same language.

The library routines for NLS can be called from any NLS-enabled program. Programs in the application that are not NLS-enabled can also call these routines, but only after the National Language Support module has been loaded and initialized by calling an NLS-enabled program.

You can pass parameters from an NLS-enabled program a non-NLS-enabled program. However, parameters that depend on the language environment in which they are created retain their format regardless of the language environment in which they are used. If you attempt to use a parameter created in an NLS-enabled program in a non-NLS-enabled program, the result might not be as you expect. We recommend that you do not attempt to pass such parameters to programs other than those that have the same language environment.

7.9 Converting Between ASCII and EBCDIC

Normally you use data that is in ASCII format, although with Server Express you can also use data that is in EBCDIC format. Storing data in EBCDIC format enhances the testing of programs targeted to run on the mainframe and eases data exchange between the mainframe and your system.

If you want to use EBCDIC data on your system, make sure your programs are in ASCII format, and then compile them with the CHARSET(EBCDIC) directive. You can then use the resulting compiled programs with unconverted data files.

By default, Object COBOL supports ASCII/EBCDIC conversions of a standard US codepage based on PC codepage 437. Additional preconfigured modules are available for converting various Far Eastern languages and German. To convert EBCDIC correctly in these environments, you need to replace the default module with the preconfigured module for the relevant character set.

7.9.1 Converting ASCII/EBCDIC Data Using the CHARSET Directive

You can convert your data between the ASCII and EBCDIC character sets by using the CHARSET directive. Compile your program with the CHARSET directive as follows:

CHARSET "character-set"

Where character-set is ASCII or EBCDIC.

The CHARSET directive also sets the related directives NATIVE and SIGN. If you have CHARSET(EBCDIC) set, you also have the behavior of SIGN(EBCDIC) and NATIVE(EBCDIC).

Both the called and the calling programs must be compiled with the same setting for the CHARSET directive.

Compiling with the EBCDIC option results in a Data Division that looks exactly the same as it would when compiling with a mainframe compiler running in an EBCDIC environment. This means that DISPLAY type values are in EBCDIC, and COMP values are binary fields, as normal. Similarly, any alphanumeric literal in the Procedure Division (for example, MOVE "HELLO" TO FRED or MOVE ALL "X" TO FRED) is in EBCDIC. The procedural code expects all DISPLAY type data, including numeric data, to be in EBCDIC.

When reading a file, if portions of the record represent characters while other portions are binary, it is necessary to have a suitable record definition in the COBOL program, just as it is on the mainframe. The intention is that data accessible to the user (such as file records and Working-Storage) should always appear exactly as on the mainframe.

The only concessions to an ASCII environment when using the EBCDIC support are as follows:

• Data to be displayed is moved to a work area and converted into ASCII before the DISPLAY happens.
  
  
• Data to be accepted is converted to EBCDIC after the ACCEPT occurs.
  
  
• Records written to text files (files with ORGANIZATION LINE SEQUENTIAL or with the LINE ADVANCING option in the ASSIGN clause) are moved to a work area and converted to ASCII before being written out.
  
  
• Records read from text files are converted to EBCDIC.
  
  
• Records written to files using the PRINTER option (of the External File Mapper) are converted to ASCII before being written out.
  
  
• Other translations occur internally to ensure smooth running, such as the contents of the data-name in:

  call data-name

  or:

  assign to data-name (a Micro Focus extension)

7.9.2 Current Limitations on ASCII and EBCDIC

ASCII and EBCDIC are limited in the following ways:

• Only ANSI standard ACCEPT and DISPLAY statements are supported when the data is stored in EBCDIC format. The Micro Focus extensions to the ACCEPT and DISPLAY statements are not supported.
  
  
• If a called program expects parameters to be passed in ASCII format, you must ensure that the data is in the correct form.
  

7.9.3 Collating Sequences for ASCII and EBCDIC

The collating sequences for ASCII and EBCDIC are different. If you convert from EBCDIC to ASCII, comparisons might no longer give the same results. ASCII numerics and upper and lower case orderings are different from EBCDIC. For example, the ASCII numbers (hexadecimal 30 to 39) come before uppercase letters (hexadecimal 41 to 5A), followed by lower-case letters (hexadecimal 61 to 7A).

EBCDIC lower-case letters sort first (hexadecimal 81 to 89, 91 to 99, A2 to A9), followed by uppercase letters (hexadecimal C1 to C9, D1 to D9, E2 to E9), followed by numbers (hexadecimal F0 to F9). EBCDIC has special characters throughout the range.

For example, in the following code, the results are different, depending on whether you are executing in an ASCII or EBCDIC environment.

 data division
 working-storage section.

 01 item-1    pic x value"A".
 01 item-2    pic x value"a".
      ...
 procedure division
      ...
     if item-1 < item-2
         display "ASCII"
     else
         display "EBCDIC"
     end-if

When a program collating sequence is in effect, alphanumeric comparisons (including group comparisons) that contain numeric data are likely to produce meaningless results.

For example, assume:

• field-a is PIC X and contains x"4F" (the ASCII character value for the letter "O")
  
  
• field-b is PIC X and contains x"5F" (the underscore character)

Without an EBCDIC collating sequence, field-a is less than field-b. With an EBCDIC collating sequence, however, the x"4F" is mapped onto the EBCDIC "O", which is x"D6". The underscore character, x"5F", is mapped onto the EBCDIC character x"6D". With theNATIVE(EBCDIC) Compiler directive, field-a is greater than field-b.

Some of the areas affected by using different collating sequences are IF statements, SORT statements, writing data to indexed files, and any areas where you assume data to contain the EBCDIC hexadecimal values (for example, assuming space = X"40").

7.9.4 Converting ASCII/EBCDIC Data Using the CODESET Program

For normal applications, you use the CHARSET directive to convert ASCII and EBCDIC. However, under some circumstances, you might want to call the _CODESET program yourself to convert them.

To invoke the _CODESET program, use the following command line:

Syntax:

call "_CODESET" using function-code, text-length, text-string

Parameters:

On Entry:

On Exit:

Comments:

Note that _CODESET program cannot work from a program that is compiled with the reserved word directives for one of the mainframe COBOL compilers (for example, OSVS for OS/VS COBOL and VSC2 for VS COBOL II) because it uses COMP-X data types, which are not supported on the mainframe.

codecomp -p filename-1 [filename-2]

codecomp name-1 [name-2]

MFCODESET=nnnn

MFCODESET=2001

     $ identifying-string 
     ; date/time 
     ;             00        corresponding-value
                   01              .... 
                   02              .... 
                   .               .... 
                   .               .... 

7.9.6 Configuring the _CODESET Program Manually

You can manually reconfigure the _CODESET program for single-byte characters, by amending and recompiling the source file and building the new module into the COBOL system.

The source and copyfile are provided in the src subdirectory within your COBOL system directory. You change the 256-byte conversion tables in the source. When amending these conversion tables, you must use the binary value of the source character+1 to index into the table. The character at this index point is the target character.

When you have modified the _CODESET source, you need to recompile it, create an executable version in some form, and package the executable version appropriately. Do this in the same way as if you had used the Codecomp utility to modify _CODESET.

Portability Guidelines

National Language Support (NLS) Demonstration