Forms

Building Internet Programs

Chapter 3: Server-side Programming

This chapter describes how you write server-side programs for Internet applications. The techniques in this chapter apply equally to NSAPI and CGI programming.

3.1 Overview

Server Express provides extensions to COBOL which make it much easier to write Internet server-side programs than with other languages. You can receive data from a form using the ACCEPT verb, and send a result back to a Web browser using either the DISPLAY verb or Embedded HTML (EHTML). EHTML enables you to construct HTML pages from within a COBOL program. This chapter covers four main areas:

Resource contentions
Input to a server-side program
Output from a server-side program
Maintaining application state

3.2 Resource Contentions

A server-side program can be executing simultaneously several times for different users. If it is a CGI program, each time the program is executed it runs as a separate process; if it is a NSAPI program, each time it is executed it runs in a separate thread. To obtain the full advantages of a NSAPI program, it should be written as a multi-threading program. See your Multi-threaded Programming book for details on multi-threaded programming. If your server-side program needs to access shared resources (for example files and databases), you must include logic to handle resource contentions.

If you are using COBOL files, COBOL includes a rich set of functions for sharing files. For more information, see your Fileshare book.

3.3 Programming Restrictions

There are various restrictions on the use of COBOL syntax in Internet programs. You also cannot use Dialog System from CGI and NSAPI programs. The restrictions on syntax are described below.

CGI and NSAPI Progr ams

You cannot use the following syntax in CGI and NSAPI programs:

Screen Section
Console input and output using ACCEPT and DISPLAY
Dialog System

NSAPI Programs

In addition to the restrictions described in the previous section, you should not:

Use syntax that enables the execution of UNIX commands, such as ls or chmod
Change the state of process using commands like PATH, COBDIR, cd and so on

These restrictions apply because all NSAPI applications running on the Web server share the same environment. If you change the environment then all of the threads running in the Web server's process will be affected. If one thread changes the environment, then the result on other threads is undefined.

3.4 Input to a Server-Side Program

Each control on a form has a name and a value. As explained in the chapter Forms, when the end-user submits the form, the information on the form is sent to the server-side program as a set of name/value pairs. Extensions to COBOL syntax enable you to associate the names of the controls on the form directly with COBOL data items in your server-side program. When the data in the form is posted to the server-side program, the data items are set to the values of the controls on the form.

The form below has a field that has been named "name". The end-user types in the value "Bob".

The server-side program ties into each of the named controls by having a declaration like this in Working-Storage:

01 inputdata is external input-form.
   03 name-field      pic x(30) identified by "name".
   03 email           pic x(15) identified by "emailid".
   03 phone-no        pic x(30) identified by "phone".

When the server-side program is run by the end-user clicking the Send Form button (which is the Submit control in this example), the value Bob is transferred into data item NAME-FIELD when the following statement is executed:

accept input-form

3.4.1 Syntax

Use the following syntax to map a COBOL data item to CGI input:

level-number data-name-1 IS EXTERNAL-FORM.

This declaration says that group item data-name-1 and its subordinate fields can have their values set by input from an HTML form. To map elementary data items to NAME attributes.

sub-level-number data-name-2 picture-string IDENTIFIED BY name.

where the parameters are:

`sub-level-number`	A COBOL data-item level number.
`data-name-2`	A COBOL data-name.
`picture-string`	A COBOL picture string. PIC X(n), PIC 9(n), and numeric edited fields are allowed. Strings longer than n characters are truncated, strings less than n characters are padded from the left with spaces.
`name`	The value of a Name or Groupname property on the corresponding control on the Form.

3.4.2 Example

This example shows a set of data-items mapped to the fields on a form.

01 input-form is external-form.
    03  name-field pic x(30) identified by "Name".
    03  phone-no   pic x(30) identified by "Phone".
    03  email      pic x(15) identified by "EmailID".

3.5 Output from a Server-side Program

Server Express provides two alternative methods for outputting HTML from a COBOL program to a Web browser. Both methods enable you to substitute the values of COBOL variables into the HTML output:

Embedded HTML (EHTML)
With EHTML you can include HTML in a COBOL program between EXEC HTML/END-EXEC delimeters. HTML statements can either be coded directly into your source program, or included as copyfiles.

The EXEC HTML mechanism provides complete program control over the output of HTML. The HTML source is actually linked into your server-side program, so if you change the HTML copyfiles, you need to recompile and relink the program.
DISPLAY
DISPLAY enables you to display an HTML page held in an external file.

3.5.1 Using EHTML

Embedded HTML (EHTML) enables you to output HTML directly from a COBOL program.

EHTML enables you to include complete HTML pages as copyfiles inside your program but with the advantage that you don't need any special data declarations. You can also use partial HTML pages as copyfiles, or output individual HTML statements, and build up a complete page under complete program control. This gives you tremendous flexibility.

You embed HTML into a COBOL program by using the EXEC HTML statement. This is translated by the EHTML preprocessor, htmlpp.

The EXEC HTML and END-EXEC keywords enable you to embed HTML directly into your COBOL source code. You must put keywords EXEC and HTML on the same line; otherwise there are no formatting restrictions.

This is the general format:

EXEC HTML
  [htmloutput] 
  [copy "file.htm".]
END-EXEC

where the parameters are:

`htmloutput`	HTML markup for output to a Web browser.
`file.htm`	A file containing HTML markup

The EHTML preprocessor does not validate or parse your HTML markup - it just outputs it directly to the Web server which started the CGI program.

With fixed format COBOL source code, the EHTML preprocessor treats column 8 as a virtual column 1 when outputting embedded HTML. This simplifies use of the HTML <PRE> tag. However, if you include a copyfile with an extension of .htm, the preprocessor treats the copyfile as free format source. You can override this behavior with the preprocessor directive, NOAUTOFORMAT.

3.5.1.1 Substitution Markers

You can substitute variable data from your COBOL program directly into the HTML output by using substitution markers. Substitution markers are COBOL data-names prefixed by a colon (:), as shown below:

:data-name

You should always use display items inside EHTML; if you use binary data items the data displayed on the form cannot be read by users.

Colons followed by a space or punctuation marker are always output as colons, and not treated as substitution markers. You can also force a colon to be output as a colon by preceding it by a backslash (\) character.

Note:

A :literal reference is not treated as a substitution marker if it is preceded by any of the following (regardless of case):

data
clsid
layout
layout
javascript
about
http
file
ftp
mailto
news
gopher

For example:

 working-storage section.
 01 acct-code    pic 9(8). 
 ...
 procedure division. 
 ...
*> First colon is followed by a space, so it is 
*> not treated as a substitution marker by 
*> EXEC HTML. Second colon is followed by a 
*> data-name, so it is treated as a 
*> substitution marker. The third colon is preceded 
*> by a backslash, so it is treated as a colon
 EXEC HTML
     Account Code: :acct-code <BR>
     \:acct-code
 END-EXEC

To qualify a substitution marker, use a period (.) between the group item data-name and the elementary item data-name. For example:

 working-storage section.
 01 customer.
  03 name         pic x(80).
  03 acct-code    pic 9(8). 
 ...
 procedure division. 
 ...
*> customer.acct-code equivalent inside EXEC HTML
*> block to acct-code of customer in COBOL source. 
 EXEC HTML
     Account Code: :customer.acct-code <BR>
 END-EXEC

You can also reference modify substitution markers:

 working-storage section.
 01 acct-code    pic 9(8). 
 ...
 procedure division. 
 ...
*> Reference modification outputs first four characters of 
*> acct-code.
 EXEC HTML
     Account Code: :acct-code(1:4) <BR>
 END-EXEC

3.5.1.2 EHTML Preprocessor Directives

To run the EHTML preprocessor, you need to set the following Compiler directive:

preprocess(htmlpp) [preprocessor-directives] end

Note: You can migrate to UNIX systems programs created on NetExpress using the tools Form Express or Form Designer; in this case, the PREPROCESS directive is already set in a $SET statement.

You can also set EHTML preprocessor directives by creating an ASCII file called htmlpp.dir, and putting it on the $COBDIR path for your Server Express system. This table lists all the directives for the EHTML preprocessor, htmlpp. Set these directives by placing them between keywords preprocess(htmlpp) and endp.

[NO]AUTOFORMAT or [NO]AF	Do not assume that .htm copy files are free format source. If COBOL is compiled fixed format then only columns 8 through 72 are read in .htm files.
DEBUG or D	Output extra debugging information to the specified output file.
NOWARN or NW	Suppress any warning messages that Htmlpp might produce.
OUT(name) or O(name)	Create a fully preprocessed output source file with the name name
PREPROCESS(name) or P(name)	Specify a nested preprocessor to read source files and process COPY statements.

3.5.2 Using DISPLAY

You can send an HTML page to a Web browser using the DISPLAY verb. The HTML page can include substitution markers for variable data to be supplied by the program. The substitution markers are not standard HTML; however, the COBOL system replaces them with the variable data before sending the page to the Web browser. Each substitution marker is in one of following formats:

%%name%%
%%!s name%%
%%!e name%%

where the parameters are:

`%%`	Delimits the substitution marker.
`name`	Name mapped to a COBOL data item by the IDENTIFIED BY clause on a data declaration. The contents of name are output as is, unless `!s` or `!e` are specified.
`!s`	Strip trailing spaces from the data item. When your program outputs plain text, the Web browser usually removes trailing spaces for you, but if you are using DISPLAY to put values into form elements, you need to strip trailing blanks using this feature.
`!e`	Convert double quote ("), ampersand (&), greater than (>) and less than (<) symbols into their equivalent entities.

This is the syntax to map a COBOL group data item to an output page:

level-number data-name-1 IS EXTERNAL-FORM identified by "pagefile.htm".

where the parameters are:

`level-number`	A COBOL data-item level number for a group item.
`data-name-1`	A COBOL data-name
`pagefile.htm`	The filename of an HTML page for output.

This declaration says that HTML form pagefile.htm can have data set from group item data-name-1 and its subordinate fields. To map elementary data items to form names:

sub-level-number data-name-2 picture-string IDENTIFIED BY name.

where the parameters are:

`sub-level-number`	A COBOL data-item level number.
`data-name-2`	A COBOL data-name
`picture-string`	A COBOL picture string. PIC X(n), PIC 9(n), and numeric edited fields are allowed. Strings longer than n characters are truncated, strings less than n characters are padded from the left with spaces.
`name`	The name of a substitution marker in the HTML page to be output.

Example:

This example shows a set of data-items mapped to the fields on a form.

01 output-form is external-form identified by "outpage1.htm".
    03  name-field pic x(30) identified by "Name".
    03  phone-no   pic x(30) identified by "Phone".
    03  email      pic x(15) identified by "EmailID".

This is an example HTML page with substitution markers set up for the names above:

<HTML><BODY>
<H1>This is the output-form </H1>
<P>Dear %%Name%%,</P>
<P>You entered the following information</P>
<UL>
<LI>'phone number - <B>%%Phone%%</B></LI>
<LI>Email id - <B>%%EmailID%%</B></LI>
</UL>
</BODY></HTML>

3.6 CGI Environment Variables

Every CGI program receives information about both the browser and the server through environment variables. This happens whether you pass form data to the program or not.

The CGI environment variables provide such information as:

Where on the network the browser is located
The type of browser and what type of documents it understands
The name and version of the server that called the CGI program
Instructions on how to receive and interpret data sent by the browser

Certain environment variables are always set by those servers that adhere to the CGI protocol. Other environment variables also exist that, while not adhering to the CGI protocol, are passed to CGI programs.

The CGI environment variables are listed at: http://hoohoo.ncsa.uiuc.edu/cgi/env.html.

To read and change the environment variables, you use the syntax:

DISPLAY ... UPON ENVIRONMENT-NAME to define which environment variable to change
ACCEPT ... ENVIRONMENT-NAME to read the value of an environment variable

For example, to get the value of the CGI environment variable SERVER_SOFTWARE, which contains the name and version of the server software:

procedure division.
display "server_software" upon environment-name
accept env from from environment-value
display "Server software is: " env
...

Example:

This example reads various CGI environment variables and writes them to a form:

$set mf preprocess(htmlpp) endp
 *References:
 * http://www.w3.org/CGI/
 *
 program-id. "cgivars".
 working-storage section.
 01 cgi-var-content-type         pic x(128).
 01 cgi-var-server-name          pic x(128).
 01 cgi-var-server-port          pic x(20).
 01 cgi-var-server-software      pic x(128).
 01 cgi-var-gateway-interface    pic x(40).
 01 cgi-var-remote-host          pic x(128).
 01 cgi-var-remote-addr          pic x(128).
 01 cgi-var-http-user-agent      pic x(128).
 01 cgi-var-http-referer         pic x(128).
 
 procedure division.
 main section.

 * Query CGI environment vars

 display "CONTENT_TYPE" upon environment-name
 accept cgi-var-content-type from environment-value

 display "SERVER_NAME" upon environment-name
 accept cgi-var-server-name from environment-value

 display "SERVER_PORT" upon environment-name
 accept cgi-var-server-port from environment-value

 display "SERVER_SOFTWARE" upon environment-name
 accept cgi-var-server-software from environment-value

 display "GATEWAY_INTERFACE" upon environment-name
 accept cgi-var-gateway-interface from environment-value

 display "REMOTE_HOST" upon environment-name
 accept cgi-var-remote-host from environment-value

 display "REMOTE_ADDR" upon environment-name
 accept cgi-var-remote-addr from environment-value

 display "HTTP_USER_AGENT" upon environment-name
 accept cgi-var-http-user-agent from environment-value

 display "HTTP_REFERER" upon environment-name
 accept cgi-var-http-referer from environment-value


 * Output HTML header back to the agent
 exec html
     <HTML>
      <HEAD>
       <TITLE>CGI Environment information</TITLE>
      </HEAD>
      <BODY>
       <H1>CGI Environment information</H1>
 end-exec
 
 exec html
     <B>Server Information</B><BR>
     This server is using :cgi-var-server-software and is
     called :cgi-var-server-name on port :cgi-var-server-port<BR>

     :cgi-var-server-name is using :cgi-var-gateway-interface<BR>
     <BR>
     <B>Form information</B><BR>
     The content-type is :cgi-var-content-type<BR><BR>
 end-exec

 * Find out the fully qualified domain name of the agent
 * Some servers may not return this information.  In CGI/1.1 it
 * states REMOTE_HOST should be supplied.
 
 if cgi-var-remote-host not equal spaces
 
     exec html
       <B>Client Information</B><BR>
       The remote hosts is called :cgi-var-remote-host, and
       the IP address of the agent is :cgi-var-remote-addr<BR>
     end-exec
 else
 
     exec html
       <B>Client Information</B><BR>
       The IP address of the agent is :cgi-var-remote-addr<BR>
     end-exec
 end-if

 *
 * HTTP_* variables are specific to the HTTP header sent 
 * via the agent
 *
 if cgi-var-http-user-agent not equal space
     exec html
       The agent is identified as :cgi-var-http-user-agent <BR>
     end-exec
 end-if

 if cgi-var-http-referer not equal space
     exec html
       The cgi program was referered to us by:
       :cgi-var-http-referer <BR>
     end-exec
 end-if

 exec html
      </BODY>
     </HTML>
 end-exec

 exit program

 end program "cgivars".

3.7 Maintaining Application State

One of the problems with Web-based applications is maintaining application state. Each time a CGI program is run, it has no memory of what went before, or which client is using it. There are two mechanisms you can use to maintain the state of an application for each client using it:

Hidden fields
State information is kept on the form sent to the end-user's Web browser, in fields which are not displayed in the browser. Hidden fields work with any browser that supports forms. However, state cannot be maintained between sessions, because as soon as the form is lost from the end-user's machine, all the state information is lost with it.
Cookies
State information is maintained even if the end-user sends their browser to other pages or sites before returning to your application. Cookies can maintain state between sessions. However, they must be using a browser which supports cookies.

There are two drawbacks to using either of these mechanisms on their own:

Sending large amounts of state information over a network can slow down the application; particularly for an Internet client connected over a dial-up line.
You might have security or other reasons for not wanting to send detailed state information over an intranet or Internet

Server Express provides an extra mechanism which enables you to store all the state information on your server, where it can be accessed rapidly by server-side programs. A set of call-by-name routines enables a server-side program to request a unique client-id for each client which connects, and store state data for the client in an indexed file.

The diagram below shows a server-side program sending a form to a client browser, together with a client-id baked inside a cookie. The next time the client browser makes a request to the Web server, it passes back the cookie. The server-side program (which might be the same as the first one, or a different one) uses the client ID to retrieve the record with the state data.

Figure 3-1: The server-side state file

Note: If you intend to store sensitive information in the state file (for example, credit card details), you should implement extra security measures such as only providing access to the application over secure network links, or encrypting data. A malicious user could potentially access the state records for someone else by forging a different client-id before resubmitting a form. Whether or not they see any of the actual state information depends on the design of your application, and what type of information it sends back to the Web browser.

The next three sections cover:

Using hidden fields
Using cookies
Using the Server Express server-state routines

3.7.1 Hidden Fields

You can use hidden fields on HTML forms. Hidden fields work like HTML Inputs (entry fields), except that they can't be seen by the end-user who loads your form.

When you create a CGI which uses the form as an input or output form, it generates COBOL data items corresponding to the hidden fields as it does for any entry field. You can store state information for your application in the hidden field when the form is output, and read it back in when the form is submitted to another CGI.

3.7.2 Cookies

A cookie is a name/value pair sent to a Web browser along with an HTML page. You can record application state information in the cookie, and retrieve it the next time that particular client accesses a server-side program. Netscape Navigator 2.0 or later and Microsoft Internet Explorer 3.0 or later both support cookies; not all other browsers do.

The full cookies specification can be found on the Netscape site - click here.

By default, cookies are retained by the Web browser until the end-user closes the browser down. You can set an expiry date, in which case the information is stored in a cache maintained by the browser until the expiry date.

There are two parts to using cookies:

Sending a cookie
Retrieving a cookie

The demonstration program howmany.cbl uses cookies to maintain an individual page count for each end-user who accesses the page, as well as a global count. Use Infomgr to locate the demonstraion program.

3.7.2.1 Sending a Cookie Using EHTML or DISPLAY

You can set a cookie by including the following in the <HEAD> section of the HTML page you send down to the Browser with your form:

<META HTTP-EQUIV="Set-Cookie"
CONTENT="name=value"[;expires=expiry]>

where the parameters are:

`name`	The name of your cookie
`value`	Its value
`expiry`	The expiry time/date and date. This information is optional - by default browsers erase all cookies when they are shut down.

The name and the value can be any string, up to a maximum of 4 Kbytes for the length of the name and value combined. If the form is output by a COBOL program, you can use substitution markers for name, value and expiry, and use EHTML or the DISPLAY verb to output the form

3.7.2.2 Receiving a Cookie

The Web browser identifies cookies according to the domain of the server which sent them. The domain is the part of the URL which identifies your Web site (for example merant.com is the domain for the MERANT Web site). If your server-side program has sent a cookie to a Web browser, the browser sends the cookie back each time it accesses a resource on the same server. You can read the information back inside your server-side program by using the ACCEPT verb.

To do this, declare data items for your input-form, identified by the name of the cookie. For example, to get the value of a cookie with the name mycookie:

 01 inputdata is external input-form.
    03 name-field      pic x(30) identified by "name".
    03 phone-no        pic x(30) identified by "phone".
    03 email           pic x(15) identified by "emailid".
    03 cookie-data     pic x(40) identified by "mycookie".
    ...
 procedure division. 
 ...
 accept inputdata
 ...

Note that if you have a control and a cookie with the same name on a form, the value returned by the cookie takes precedence.

3.7.3 Using the Server-side State Mechanism

The server-side state mechanism enables you to store information about the application state using a record format you define. One of the consequences of using the mechanism is that you always need a server-side program to output the first form in an application, to assign a client ID at the start of the session.

If you are using cookies to store the client ID, you can set their expiry information so that that state information is stored longer than the user's current session. This gives you the option of making an application persistent; the user can restart an application the following day, and be returned to where they were when last working on it. Programs written for a persistent application need to test the cookie with the client ID to determine whether this is the first time they have been run or not; if the client ID value is spaces, you need to allocate an id and start a new session, otherwise you need to restore the state information and display an appropriate form.

Notes:

The server-side state mechanism is implemented by a module called Sstate. The source code is provided for Sstate so that you can modify the subroutines it provides if you want to add extra functionality - encryption for example.

The source code, sstate.cbl, is in the $COBDIR/src/mfclient folder. If you want to make changes to it, copy sstate.cbl to your source directory. To get your applications to use your new sstate module you can build an sstate.so version and compile your main program with the Compiler directive INITCALL(SSTATE). For example:

cob -z sstate.cbl -o sstate.so
cob -xU -C 'initcall(sstate)' pizza.cbl

If you are porting or deploying an application from NetExpress, note that applications generated with the NetExpress Data Access Wizard all rely on sstate. You should not change the interface to sstate, or Data Access Wizard applications might fail unpredictably. The state maintenance library routines are described in the chapter Library Routines.

The state maintenance library routines are described in the chapter State Maintenance Library Routines.

The flowchart below shows the logic for using the server-side state mechanism. Click on the the boxes to see more information about how you carry out each step:

Figure 3-2: Using the server-side state mechanism

This is a summary of the information provided by the flowchart above:

Read the client ID from the cookie or hidden control returned by the browser.
Or:

Allocate a new client ID, by calling the MF_CLIENT_STATE_ALLOCATE routine.
Specify the name of your state file.
Call the MF_CLIENT_STATE_FILE routine.
Read the record for the client ID.
Call the MF_CLIENT_STATE_RESTORE routine.
Read and process the input form, using the state information as needed.
Save the client state, and return the next form to the browser.
Call the MF_CLIENT_STATE_SAVE routine.

These routines are documented in full in the chapter State Maintenance Routines. The following example is the skeleton of a program showing the use of the state maintenance routines.

 working-storage section.
 ...
 01 state-filename           pic x(255) value
                             "MF-STATE-SAVE.DAT".
 01 client-length            pic xxxx comp-x.
 01 client-state.
  03 accessCount             pic 9(9).
 01 state-status             pic x comp-x.
 01 browserInput is external-form.
 *> cookie
  03 client-id               pic x(30) identified by "clientid". 
 ...
 
 procedure division.
 *>  Open the client-state file.
 call "MF_CLIENT_STATE_FILE" using state-filename
 ...

 accept browserInput      *> read in the cookie with 
                          *> client ID
 move length of client-state to client-length
 if client-id = spaces
 *>  cookie is empty - first time in
     call "MF_CLIENT_STATE_ALLOCATE" using client-id 
                                           client-length 
                                           state-status
     ... *> any other first time in processing

 else
 *>  use client ID to restore state
     call "MF_CLIENT_STATE_RESTORE" using client-id
                                          client-state
                                          client-length
                                          state-status
 end-if
 ...
*>  Process data, and return form to client, including cookie
 ...
*>  Save client status
 call "MF_CLIENT_STATE_SAVE" using client-id 
                                   client-state
                                   client-length
                                   state-status

3.7.4 Removing Client State Records

Two routines are included for removing client state records when they are no longer required. One deletes a single record, the other purges all records more than a certain number of days old.

To delete a single record, call MF_CLIENT_STATE_DELETE. For example:

 working-storage section. 
 ... 
 01 state-status             pic x comp-x.
 01 client-id                pic x(30).
 01 state-filename           pic x(255) value
                             "MF-STATE-SAVE.DAT".
 
 procedure division. 
 ... 
*>  Open the client-state file.
 call "MF_CLIENT_STATE_FILE" using state-filename
 ...  
 call "MF_CLIENT_STATE_DELETE " using client-id
                                      server-status

To purge all records over a certain age, call MF_CLIENT_STATE_PURGE. For example:

 working-storage section. 
 ... 
 01 state-status             pic x comp-x.
 01 age-in-days              pic x(4) comp-x. 
 01 state-filename           pic x(255) value
                             "MF-STATE-SAVE.DAT".
 
 procedure division. 
 ... 
*>  Open the client-state file.
 call "MF_CLIENT_STATE_FILE" using state-filename
 ...  
 move 5 to age-in-days *> remove all records more than 5 days old
 call "MF_CLIENT_STATE_PURGE " using age-in-days
                                     server-status

These routines are documented in full in the chapter State Maintenance Routines

Forms

Building Internet Programs