Previous Page TOC Next Page See Page



— 6 —
The Win/CGI Interface


Anyone who has spent much time exploring the vast reaches of the World Wide Web has more than likely launched a CGI ("Common Gateway Interface") application. Chances are just as good that that person wasn't even aware of having done such a thing. CGI applications perform all sorts of tasks on the Web: access counting, information gathering, database searching, online ordering, connecting to information appliances (such as soda machines, weather instruments, cameras, and the like), and almost any other imaginable function. Unlike static Web pages, a CGI application enables a webmaster to display real-time, dynamic information to the client. CGI applications also allow Web clients to access information that is not in a format normally readable by a Web client. This is why the term gateway is used—the CGI application serves as a gateway between an external information source and the Web server.

The most common uses for CGI applications are information gathering and database searching. Figure 6.1 shows a typical registration form that could be found on any Web site. The form is coded using HTML (see Appendix A) and provides input controls into which the user will enter data. These HTML forms are analogous to data entry forms in a Visual Basic database application except that the HTML forms are used only for data input. HTML forms are not typically used for displaying data to the user.

Figure 6.1. A sample HTML form used with a CGI application.

The form in Figure 6.1 contains a button labeled "Submit Entry." This button instructs the Web browser application to send an HTTP request message to the server. Forms typically use the POST method in the request message. The resource specified in the request message for this form is a CGI application. The server launches the specified application. The application processes the data from the form and then produce some sort of output, which the server returns to the Web browser in the <Entity-Body> portion of the response message. The Web browser then displays this output to the user. After the user clicks the Submit button, the entire process that takes place is transparent to the user. Unless an error occurs, the user typically has no knowledge of what takes place "behind the scenes."



Chapter 2 covers the details of the HTTP protocol, including HTTP messages and methods.

CGI applications historically have been written for the UNIX platform. This is quickly changing as the number of available Windows-based (this includes Windows 3.x, Windows 95, and Windows NT) Web servers has begun to increase. CGI applications can be written in any language as long as that language can be executed by the HTTP server. The following are the typical languages used:

Applications written to run on Windows-based platforms use a special interface known as the Windows CGI (or Win/CGI) interface. The difference between Win/CGI and UNIX CGI applications lies essentially in how the data is passed from the HTTP server to the application and then back to the server.

This chapter discusses the Win/CGI interface in detail. Subsequent chapters will build on the material contained here to assist you in building Win/CGI applications that can be run on your own Windows-based Web server.



The Win/CGI interface specification is not a formal Internet specification. It is authored by Robert Denny, who wrote the Windows-based Web server application now sold by O'Reilly and Associates as WebSight. The specification is currently at version 1.3a and can be found at http://website.ora.com/wsdocs/32demo/windows-cgi.html. The WebSite demo version can be found on the CD-ROM accompanying this book.


How CGI Data Is Handled


When writing a CGI application, the developer has the same two concerns that developers of all applications have: data input and data output. This section discusses how data is input into a CGI application. The closing section of the chapter ("Returning the Results to the Client") will discuss how data is output by the application.

In the UNIX environment, CGI applications receive their data from command line arguments, environment variables, and the standard input (known as stdin to C programmers). By querying the value of one of a predefined set of these variables, the UNIX CGI application can determine the server context that it has been run under and the data that was entered at the client side. The application then returns its data to the server using the standard output (stdout).

In the various Windows environments (Windows 3.x, Windows 95, and Windows NT), the operating system makes the use of environment variables and the standard input/output difficult. For this reason, the Win/CGI interface uses a spooling paradigm for passing data between the server and the CGI application. Before executing the CGI application, the server creates a CGI data file. This file contains the same data fields that are used in UNIX CGI applications, along with fields specific to Win/CGI. The name of the data file is passed to the CGI application as a command line argument. Likewise, the CGI application places its output into a file whose location the server specifies in one of the fields of the CGI data file.

Decoding the HTML Form


As you learned in Chapter 2, the client is not limited to simply retrieving resources from HTTP servers. There are also two HTTP request methods that allow the client to interact in some way with a resource.

The first method uses a GET request message that includes search terms in the resource address:

GET //myserver.com/cgi-win/search.exe?last=smith+first=jim

This request message could have been generated by clicking the Submit button of a form or by some sort of user agent. Likewise, the user of a Web browser such as Netscape could enter the address portion of this string into the text box provided for Web addresses. Any of these three actions causes the HTTP server to launch search.exe and pass it the search text that appears after the "?".

The second method uses a POST request message. The difference between a POST message and the GET message illustrated previously is that the POST message uses the <Entity-Body> portion of the message to transfer the form data. This method is used by the Submit button or by a user agent. Because the form data entered on the HTML form is sent in the <Entity-Body>, it cannot be entered into the address box of a Web browser.

With the second method, the Win/CGI interface requires that the server parse the HTML form data from the POST message it received. This data is then stored in either the CGI data file previously discussed or in an external file. In the later case, the server places an entry in the CGI data file that specifies the filename and length of this external file. The client application uses the Content-Type entity header to specify how the data is encoded in the <Entity-Body>. There are two Content-Types used: application/x-www-urlencoded and multipart/form-data. The first is the Content-Type used in most cases. The second Content-Type provides for uploading files from the client by using a multipart MIME message. To date, multipart MIME messages are not widely used in HTTP messages. The Content-Type header is passed to the CGI application as one of the fields in the CGI data file.

Launching the Application


When a client application or user agent requests a CGI application resource, the server must launch the application. This section discusses how the application is launched and the environment that is available to it. Because this chapter discusses the Win/CGI interface, I won't be covering the intricacies involved in running UNIX applications. I will instead concentrate on the Windows applications.

Special Considerations When Using Win/CGI


You should keep in mind several considerations when writing Win/CGI applications. These considerations are extremely important to the health of your Web server machine as well as to the user's experience with your Web site. Failure to take into account some of these considerations can have disastrous consequences for your computer.

The first consideration involves keeping your server and its data secure. It is of utmost importance for you to remember that, by opening a CGI application to the Internet, anyone can run the application on your machine. Make sure you don't leave any "back doors" open in the application. You should also make sure the application is installed into a special directory set up to specifically house CGI applications. The documentation for your Web server application should tell you how to set up this directory. Also, make sure that you limit the application's use of system resources, such as printers, network resources, and directories. Some people on the Internet make a hobby of trying to infiltrate the systems of others and exploit weaknesses they find. The havoc that they can wreak in an unprotected system is far worse than what a virus can accomplish, so use caution.

The second thing you should keep in mind is that while your application is running, the user is wondering what's taking it so long to finish. No matter how quickly a developer feels the application executes, the user always wants it to move along faster. Even if your application produces a Web page with valuable information for the user, the user might never see it if he or she decides to click that "Stop" button on the Web browser. Although this action won't directly affect your application (in fact, because of the statelessness of HTTP, the application won't even be aware that the Stop has happened), you do lose the opportunity to present the output to the user. If your application is collecting information, you might want to limit the CGI application merely to storing the data in a table. If there is further processing to be performed on the data and that processing doesn't have to be completed before producing the output page the client will see, it is best to leave the processing to a batch process that runs at a later time.

The third consideration is that when the server launches a CGI application, it creates a separate process on the machine. For each concurrent request made, another concurrent process is created. This consumes both memory and processing time, and it can also affect the ability of the operating system to share resources that may be needed by the CGI application. Therefore, you should attempt to limit the maximum number of concurrent connections to your server. Most HTTP server applications provide an option for specifying the maximum number of concurrent connections. You should pick a number that is a compromise between the number of people you'd like to have viewing your static Web documents and the number you'll allow to run applications.

The Win/CGI Command Line


When the server launches the CGI application, it does so using the CreateProcess() API function. It then monitors this process to detect when the process has finished (that is, when the application has completed execution). Typically during the time the application is running, the TCP/IP connection between the client machine and the server machine is left open. The client application typically displays a busy indicator to the user. When the CGI application exits, the server then transfers its output to the client in the form of an HTTP response message.

The server may (and probably will) detect that the CGI application's process has ended before the application has been completely shut down. Do not rely on the program exiting for the closing of files and other resources—you should explicitly close them before the program ends. Visual Basic closes any open files when an application ends, but you should not rely on this "feature"; otherwise, your CGI application more than likely will not function correctly.

The command line used to execute the CGI application depends on the HTTP server. The current version of the CGI interface calls for the following command line:

<CGI-application-path> SP <cgi-data-file-path>

Both paths must be absolute paths. The server must not rely on the current directory (which has no meaning to the new process) or to the PATH environment variable.

Older versions of Windows-based HTTP servers used a command line with a few more parameters:

<CGI-application-path> SP <cgi-data-file-path> SP <Input-File> [ic:ccc]
SP <Output-File> SP <URL-parameters>

If your application will be executed only on servers that use the simpler command line, there is no need to look for the rest of the command line parameters. If, however, you plan to distribute your application publicly or aren't sure of the HTTP server's capabilities, you should plan for both possibilities.

The following section covers the CGI data file in depth.

Retrieving the CGI Data


Before the server launches a CGI application, it creates a CGI data file. This file contains information about the server and the client that submitted the request, as well as any data that is associated with the request (the HTML form data filled in by the user).

The file is formatted as a Windows initialization file. Therefore, the CGI application can use API calls such as GetPrivateProfileString() to retrieve the data. Listing 6.1 shows a portion of what this file might look like. In the next chapter we'll actually write the program that produced this Web page. If you've reviewed Chapter 2, a lot of the fields shown in Listing 6.1 will look familiar to you—they're the values of many of the HTTP request message header fields.

Listing 6.1 A sample CGI data file.

[CGI]
Request Protocol=HTTP/1.0
Request Method=POST
Query String=
Logical Path=
Request Keep-Alive=No
Executable Path=/cgi-win/project7.exe
Server Software=WebSite/1.1 (demo version)
Server Name=www.myserver.com
Server Port=80
Server Admin=webmaster@myserver.com
CGI Version=CGI/1.2 (Win)
Remote Address=127.0.0.1
Content Type=application/x-www-form-urlencoded
Content Length=25
[Accept]
image/gif=Yes
image/x-bitmap=Yes
image/jpeg=Yes
image/pjpeg=Yes
*/*=Yes
[System]
GMT Offset=-18000
Debug Mode=Yes
Output File=C:\WebSite\cgi-temp\3ews.out
Content File=C:\WEBSITE\CGI-TEMP\3EWS.INP
[Form Literal]
Name1=Craig

The remainder of this section discusses the different sections of the data file and the fields they contain. Some of the fields discussed in these sections aren't shown in Listing 6.1. This is because the server that generated this CGI data file is compliant with version 1.2 of the CGI interface. Also, if a field's value is empty, the interface is defined such that it should be omitted from the file. Obviously the server that generated this data file does not fully comply with the Win/CGI interface! This chapter discusses the current version (1.3a) and highlights the differences between the versions.

The [CGI] Section


The [CGI] section contains the fields that provide information about the client and server that are executing the application. Indeed, most of the fields in the data file are contained in this section.

Request Method

This field provides the value of the <method> portion of the HTTP request message. The standard methods are GET, POST, and HEAD. The HTTP protocol provides for additional methods as long as both the client and server applications both recognize them as request methods.

Query String

If the CGI application was launched in response to a GET or HEAD request containing search text in the URI, the portion of the URI appearing after the ? is contained in the value of the Query String field. The server does not attempt to decode the string in any way (recall from Chapter 2 that reserved characters are encoded using escape characters such as %20 for a space).

The value of this field also contains the HTML form data if the form's ACTION tag is defined as GET (see Appendix A for information on HTML forms). The format of the URI in this case is:

"//myserver.com/cgi-win/myapp.exe?key=value&key=value..."

Logical Path and Physical Path

If the request message includes additional path information, the server passes this to the CGI application using the Logical Path and Physical Path fields. The logical path is relative to the path space that is active for the server. The physical path is an absolute path that the application can always use. If the URI contained in the request uses an encoded path, the server must decode it before placing it in the CGI data file. The UNIX CGI specification calls these fields PATH_INFO and PATH_TRANSLATED respectively.

CGI Version

This field specifies the version of the CGI specification the server operates under. Listing 6.1 shows that the server is using version 1.2 of the Win/CGI specification. The format is "CGI/"<version>. The UNIX CGI specification calls this field GATEWAY_INTERFACE.

Request Protocol

This field specifies the protocol the request message used. This is the <HTTP-Version> element for an HTTP/1.0 request message. This field should be used when creating the output file to determine if you can send a complete response message. If the value is not HTTP/1.0, you should return only the entity portion (that is, the HTML) and no additional header fields. UNIX uses SERVER_PROTOCOL.

Referer

This field specifies the URI of the resource that contained the link to the CGI application. The value can be either an absolute or relative URI. This field can be used for back-linking to the resource that ran the application (back-linking refers to placing a link to the resource in the output document).

Server Software

This field holds the name and version number of the server software that received the request message and launched the application. The format is <name> "/" <version>. The example from Listing 6.1 specifies the demo version of WebSite 1.1.

Server Name and Server Port

The Server Name and Server Port fields specify the full host name (or IP address) and TCP port the server is listening on. These fields are used to create self-referencing URLs for the output file. They can also be used for logging purposes if the application accesses a database used by multiple servers.

Remote Host and Remote Address

These fields contain the host name and IP network address of the remote client that produced the request message. The remote host name may not always be available. These fields are used for logging purposes and can be used to compare CGI usage to the usage of other logged resources.

Content File, Content Type, and Content Length

If the request has data associated with it, these fields specify how the data can be retrieved. The Content File field specifies the full path to a file that holds the data. The Content Type field specifies the MIME type of the data. Its format is <type> "/" <subtype>. The Content Length specifies the length of the content (in bytes).

Document Root

This is the physical path to the server's logical root directory (referred to as /) as defined in the server setup. For example, the server at myserver.com may be set up to recognize c:\wwwroot as the root directory for Web services. A client that requests the document //myserver.com/index.htm is sent the file c:\wwwroot\index.htm. This field is new to version 1.3a of the specification.

Executable Path

Executable Path is the logical path that points to the CGI application. This is the path that the HTTP server recognizes but may not be the physical path. It is relative to the server's root HTTP directory as defined in the server setup.

User Agent

This field holds the value of the User Agent request message header field. The value specifies the client application that generated the request. Not all user agents generate this field.

From

The From field is the value of the From request message header field. The value specifies the e-mail address of the user that generated the request. This is in the HTTP specification but is typically not used due to privacy issues.

Server Admin

This field specifies the e-mail address of the server's administrator. It can be used in output files when an error occurs (use the mailto: link) or for sending mail to the administrator from within the CGI application.

Authentication Method and Authentication Realm

These fields are present if the request message was for a resource that the server requires to be authenticated. The Authentication Method field specifies the method being used to authenticate the user. The Authentication Realm specifies the server "realm" that describes the address space for the secure area of the server. Chapter 2 describes HTTP authentication to some extent, but the HTTP/1.0 Internet Draft document provides a complete description.

The server must provide these fields to the CGI application if they were included in the request message, even if the resource did not require authentication.

Authenticated Username and Authenticated Password

These are the username and password specified in the request message. They make up a valid and authenticated combination. If the user had failed the authentication process (by entering an invalid username/password combination), the server would not have launched the CGI application. It would have sent a response message with a status code of 401 (unauthorized).

The server must provide these fields to the CGI application if they were included in the request message, even if the resource did not require authentication.

The [Accept] Section


This section contains the client's acceptable MIME types. These are taken from the request message's Accept header field. Listing 6.1 illustrates five different values. The format of the Accept header field is

"Accept:" 1#(<type> "/" <subtype> [";" "q=" ("0" | "1" | FLOAT )] [";" "mxb=" 1*DIGIT])

The CGI data file enumerates each of the Accept header fields in the [Accept] section. The value is "Yes" unless the optional q parameter is present. The value of the q parameter becomes the value used in the CGI data file. The q parameter specifies the user's preference for the specified MIME type. For example:

"Accept: text/plain; q=0.5, text/html, text/x-div; q=0.4"

specifies that the client prefers HTML text but will accept plain text or x-div encoded text if HTML is not available. The client prefers plain text over x-div text if both are available.



To retrieve all the values in the [Accept] section, make a call to the GetPrivateProfileString() with a NULL value in the key name parameter. This places all the fields from the [Accept] section in the string buffer provided. Each substring ends with a null character (Chr$(0)), and the entire returned string has two nulls at the end.


The [System] Section


The fields in this section are fields used by the Windows implementations of CGI. They are not found in UNIX CGI systems. This section defines these fields.

GMT Offset

This is the number of seconds to be added to Greenwich Mean Time (GMT) to obtain the local time. For Eastern Standard Time, the value is -18000.

Debug Mode

This field defaults to "No" unless the server has enable CGI script tracing. The value is "Yes" if it is enabled. This can be used for conditional tracing in a CGI application. On some Web servers (such as WebSite), turning CGI tracing on also causes the server to leave all CGI data files intact after the CGI application ends. Usually these files would be deleted.

Output File

This field provides the full path to the file the application should use for its output. The server generates a unique file name to be used for this purpose. When the CGI application ends, the server sends this file to the client in (or in place of, depending on the contents of the file) the response message.

Content File

This field contains the file name of the file containing any content sent to the server in the request message.

The [Extra Headers] Section


This section contains any additional header fields sent with the request message. The keys and values must be decoded by the server if they included reserved characters.



To retrieve all the values in the [Extra Headers] section, make a call to the GetPrivateProfileString() with a NULL value in the key name parameter. This places all the fields from the [Extra Headers] section in the string buffer provided. Each substring will end with a null character (Chr$(0)) and the entire returned string will have two nulls at the end.


The [Form Literal] Section


If the application was launched in response to a POST from an HTML form, this section contains the decoded form data. Each form element has an entry in this section. Listing 6.1 shows several fields in the form of key=value. The key element is the NAME tag from the HTML form element that produced the data.

If the HTML form contains any SELECT MULTIPLE elements, the [Form Literal] section contains multiple occurrences of the key. The server generates a key=value field for the first occurrence. Subsequent appearances of the element in the form data cause a sequence number to be appended to the key (key1=value for example). The CGI application must be aware of this possibility in order to properly use these form elements.

Generally the CGI application is designed to expect certain key values from the request message. The application is typically written alongside the HTML form, similar to the way Visual Basic applications are designed.

The [Form External] Section


If the form data contained any fields longer than 254 characters, control characters, or double-quotes, the server places this data in a file. The name of the file is then specified along with the form element that produced the data. The format is

key=pathname length

The length is the length is bytes. The file should be opened as binary unless you're sure that the contents of the field is text only.

The [Form Huge] Section


If a form data element is longer than 65,535 bytes long, the server does not decode the element from the Content File. Instead, it produces an entry in the [Form Huge] section as follows:

key=offset length

where key is the form element name, offset is the byte offset from the beginning of the content file, and length is the length in bytes of the value. In the application, you would use the Seek statement on the open content file to move the file pointer to the start of the value string. You would then read length bytes to retrieve the entire value.

As with the [Form External] section, you should open the Content File as Binary unless you're positive it contains only text data.

The [Form File] Section


This section contains a list of files where uploaded file data has been placed. This is used only if the content was sent as multipart/form-data content. The format of the entries in this section is

key="["<pathname>"]" SP <length> SP <type> SP <xfer> SP "[" <filename> "]"

The <pathname> element specifies the path to the file stored on the server. The square brackets are used to delimit the file names in case they contain spaces. The <length> element is the length of the file in bytes. The <type> element indicates the MIME content type of the file, and <xfer> specifies the transfer encoding of the file. Finally, <filename> specifies the original file name of the uploaded file.

Returning the Results to the Client


The goal of a CGI application is to return some sort of response to the user. Even if the application is a guest book registration application, the user should still be informed of the fact that the entry was or wasn't added to the guest book. This section explains how data is returned to the client by the CGI application and the server.

The server is responsible for actually creating the HTTP response message sent to the client. The server is also typically responsible for adding the proper HTTP header fields to the response message. However, you can format the output such that the CGI application specifies these header fields. This is discussed below.

The body of the output file typically (but not necessarily) contains HTML formatted text. The body is displayed by the user's Web browser using the specified Content-Type header (discussed below). Several examples are given at the end of this section.

How Header Fields Are Handled


The header section consists of one or more lines of text. It is separated from the body of the message by a blank line. There are several header fields that may be placed in the application's output file.

If the application will be returning data to the client and you aren't using the direct return method discussed in following sections, the first line is used to specify the MIME type of the body. It should read as

Content-Type: <type>/<subtype>

If the application won't be returning data, but desires to send the user another file, or send them to another resource, the URI: header is used. It specifies a resource to be returned to the user in place of the body that would have been produced by the CGI application. The format is

"URI:" "<" <URI> ">"

Note that the URI is enclosed in angled brackets. If the URI points to a local file, the server sends this file within the response message. If the URI is a full URL, the server set the response message status to 401 (redirect), which informs the client that it should retrieve the URI directly.

The Location: header is used for the same reason as URI: but the address must not be enclosed in angled brackets.

If you wish to specify the response message's status, use the Status: header. The format is

"Status:" SP <Status-Code> SP <Reason-Text>

The <Status-Code> and <Reason-Text> elements should follow the definitions given for those elements in Chapter 2. However, the HTTP protocol does not require these elements to have any specific value. They should, though, follow the general classifications given in Chapter 2.

If the CGI application places any other header fields before the body, the server passes them unmodified to the client. Care must be taken to ensure that these headers do not conflict with any other response message headers the server may be using in the response message.

Creating the Complete HTTP Response Message


If you desire to create the entire response message, bypassing the server's creation of the message, you can do so. The output file would contain the complete HTTP response message (see Chapter 2 for details). This message must be a valid HTTP response or the user will likely not be able to interpret the results of the application.

To create the entire response message with the CGI application, begin the first line of the output file with HTTP/1.0. If the server sees this, it assumes the application is creating the response message. The server then returns the entire output file to the client without any modification.

Example Output Files


The following file would be returned by the server in the <Entity-Body> portion of the response message. It is an HTML document.

Content-Type: text/html
                                                   <== header/body separator (blank line)
<html><head><title>Entry Submitted</title></head>
<body><H1>Entry Submitted!</H1>
Your entry has been submitted to the Guest Book.<br>
Thank you for your support!<br><hr>
<a href="/home.htm">Return Home</a></body></html>

The next example uses the Location feature to redirect the client to another resource. In this case, the resource is located on an ftp server.

Location: ftp://ftp.myserver.com/pub/demo/demo.zip
                                                     <== blank line

Finally, this example produces the entire HTTP response message, including some additional response message header fields.

HTTP/1.0 200 OK                                    <== start of HTTP header
Date: Mon, 25 Mar 1996 05:23:01 GMT
Server: Microsoft-Internet-Information-Server/1.0
Content-Type: text/html
Content-Length: 3202
Expires: Mon, 25 Mar 1996 13:23:01 GMT
                                                   <== header/body separator (blank line)
<html><head><title>Query Results</title></head>
(etc.)

Summary


This chapter has presented the basics of using the Win/CGI interface to write server-side applications. You'll apply this information in the next chapter where you'll build a basic Win/CGI application. Subsequent chapters expand on this information in even greater detail.

Previous Page Page Top TOC Next Page See Page