PC Magazine - CGI content management system - part five - 2 pages (1,600 words) - Alex Cruickshank

In previous issues of PC Magazine we've built the basic structure of a content management system (CMS) using Perl. Once finished, this could be used to generate and serve HTML pages for a Web site, separating content from design so that little or no HTML knowledge is required in order to update the site.

So far the content-generation section of the CMS has been built, so that source text files can be created and edited. We've also incorporated a script that will handle the uploading of images from within a browser, eliminating the need for FTP transfer and therefore improving site security. Last month we looked at the part of the CMS that handles page delivery, taking the source text files and wrapping them in HTML. This month we'll investigate this in more detail, explaining how the content-serving sub-routines actually work.

The file concerned is content.cgi, which can be opened in any text editor. Having ensured that the request for a page is valid, as discussed last month, the next step is to open the chosen source text file and extract its contents, placing them into the relevant variables. This is done by the section of code shown in Figure 1.

[Figure 1 - reading in the contents of the source text file]

open (READIN, "$targetfile") || &trap_error("$!");
@rlines = <READIN>;
close(READIN);

# read in the variables
foreach $readline (@rlines) {

 if ($readline =~ m/\heading\=(.*)/i) {
  chomp($html_title=$1);
  $maintitle=uc($html_title);
 }
 elsif ($readline =~ m/^pagename\=(.*)/i) {chomp($filepagename=$1);}
 elsif ($readline =~ m/^description\=(.*)/i) {chomp($description=$1);}
 elsif ($readline =~ m/^keywords\=(.*)/i) {chomp($keywords=$1);}
 elsif ($readline =~ m/^publication_date\=(.*)/i) {chomp($publication_date=$1);}
 elsif ($readline =~ m/^body\=(.*)/i) {chomp($body=$1);}
 elsif ($readline =~ m/^note\=(.*)/i) {chomp($note=$1);}
 else {next;}
}

[Figure 1 - end]

Advanced programmers will note that some code overhead could be saved by using 'eval' statements here, but as with the rest of the CMS, clarity and ease of modification are more important for now. Script optimisation can be carried out at a later date if necessary.

At this point, the variables contain code that includes proprietary formatting commands, such as '(!paragraph)' and '(!fontsize=7)'. As discussed in previous issues, the reason for doing this is so that the CMS can be used to serve data to a variety of clients (WAP, XML, etc.) and not just in HTML. However, it means we need an extra processing stage here to convert these proprietary tags back to HTML. This is done by the unprocess_html_tags sub-routine, which is the exact opposite of the process_html_tags sub-routine in the editor.cgi script. Part of this sub-routine is shown in Figure 2 (the rest has been omitted for reasons of space).

[Figure 2 - converting proprietary tags back to HTML]

# handle the body image bit
$body =~ s/\(\!bodyimagestart\)(.*?),(.*?)\(\!bodyimageend\)/<IMG SRC=\"$1\"$2\>/gis;

# general formatting and tweaking
s/\(\!bulletlistline\)/\<LI\>/gis;
s/\(\!indentliststart\)(.*?)\(\!indentlistend\)/\<UL\>$1\<\/UL\>/gis;
s/\<UL\>\<LI\>/\<UL\>\n\<LI\>/gis;
s/\(\!underlinestart\)(.*?)\(\!underlineend\)/\<U\>$1\<\/U\>/gis;
s/\(\!italicstart\)(.*?)\(\!italicend\)/\<I\>$1\<\/I\>/gis;
s/\(\!boldstart\)(.*?)\(\!boldend\)/\<B\>$1\<\/B\>/gis;
s/\(\!fontcolour=(.*?)\)(.*?)\(\!fontcolourend\)/\<FONT COLOR=\"$1\"\>$2\<\/FONT\>/gis;
s/\(\!fontsize=(.*?)\)(.*?)\(\!fontsizeend\)/\<FONT SIZE=\"$1\"\>$2\<\/FONT\>/gis;

...

[Figure 2 - end]

Some more processing has to be done to ensure that there are no high ASCII characters in the text, since these shouldn't be present in content served to browsers. For most UK users there wouldn't necessarily be a problem, but you can't assume that a client browser is using the same character code page as you. Instead, any high ASCII characters (such as the '£' sign and any accented characters) must be converted to true HTML. For example, the '&' and '£' signs should be converted to '&amp;' and '&pound;' respectively. There are many other similar conversions that need to be carried out in order to ensure full compliance with HTML standards, and these are handled by the section of code shown in Figure 3 (note that only part of the sub-routine is shown here, for reasons of space).

[Figure 3 - converting high ASCII characters to valid HTML]

sub fix_special_chars {
 # replace 'illegal' characters with HTML code

 $_ = shift;

 s/\s\&\s/ \&amp; /g;
 s/¡/\&iexcl;/g;
 s/¦/\&brvbar;/g;
 s/¨/\&uml;/g;
 s/¯/\&macr;/g;
 s/´/\&acute;/g;
 s/¸/\&cedil;/g;
 s/¿/\&iquest;/g;
 s/±/\&plusmn;/g;
 s/«/\&laquo;/g;
 s/»/\&raquo;/g;

 ...

  return $_;
}

[Figure 3 - end]

This sub-routine will accept any string, convert all instances of the 'illegal' characters to valid HTML, and then return the updated string value. This makes it easy to use; it can be called simply by the statement;

$body = &fix_special_chars($body);

This will place the new, fixed contents of the original $body variable back into $body. In our script, this is also done for the $heading, $description and $note variables in exactly the same way. Now that we know our page content is relatively 'legal', we can send it to the browser. That's handled by the serve_page sub-routine, as shown in Figure 4.

[Figure 4 - sending the page to the browser]

sub serve_page {
 # handle the page generation

 open (TEMPLATE, $html_template);
 @template = <TEMPLATE>;
 close (TEMPLATE);

 foreach $template_line (@template) {
        
  if ($template_line =~ m/<\/TITLE>/i){ # insert the title and relevant meta tags
   $template_line =~ s/<\/TITLE>/ - $html_title<\/TITLE>/i;
   $template_line .= "<META NAME=\"keywords\" CONTENT=\"$keywords\">\n";
   $template_line .= "<META NAME=\"description\" CONTENT=\"$description\">\n";
  }
        
  elsif ($template_line =~ m/<!--body-->/i){ # insert body and associated components
   $template_line = "<H2>$maintitle</H2>\n";
   $template_line .= "<SMALL><I>$description ($publication_date)</I></SMALL><BR>\n";
   $template_line .= "$body\n";
   $template_line .= "<P><SMALL><I>$note</I></SMALL></P>\n";
  }
        
  print "$template_line";
 }
}

[Figure 4 - end]

First, the HTML template file is opened, as defined in the variables section at the top of the script. This document is then parsed on a line-by-line basis, with the script looking for particular trigger entries. If it finds a </TITLE> tag, it will append the $html_title variable to the existing page title. It will then add two extra meta-tag lines, for search engine keywords and general page descriptions. These are optional, but they do tend to help sites achieve better search engine rankings, as well as making their entries in those rankings easier to read. You can also add more meta-tags and other header code here if you wish.

The second trigger is the <!--body--> comment, which should be embedded in the HTML template at the point where you want the body text to appear. When this is found, the title heading is displayed along with the publication date (in brackets and in a smaller font). Then the actual body text is displayed, followed by a note if one exists. You can change the formatting, order and layout of these page components to suit your design preferences. You can also add extra fields if you need them (perhaps an author field or a product code field), but note that you will have to make changes to both the content.cgi script and the editor.cgi script at the same time for any new fields to be properly processed.

Now that the content-serving part of the CMS is working, it would be nice to add a 'preview' button to the editorial script, so that users can see what the page will look like as soon as they've finished editing it. This is a simple thing to do - we simply add a link to the output page that includes the current $pagename variable, as shown in Figure 5 (note that this particular listing applies to the editor.cgi file, not the content.cgi file).

[Figure 5 - adding a 'preview' link to the output screen in editor.cgi]

&header;
print "<P>Finished. The content has been written to $full_textfilename</P>\n";
print "<P>To preview the page, click <a target=\"_blank\" href=\"$content_script?page=/$pagename\">here</a></P>\n";
&footer;

[Figure 5 - end]

Clicking on the link will open the new page in a separate window, so that the user can see what it will look like when it goes live.

The core components of the CMS - editor.cgi and content.cgi - are now just about finished. You can make them more appealing by adding more aesthetically-pleasing headers and footers, but the core functionality is complete. We've included a basic HTML template on the cover CD, but you should use this merely as a guide to creating your own, more attractive design. For the most part you'll only need one template, although if you do need to run separate parts of your site with a different design, the easiest way is to set up duplicate installations of the two main scripts along with different HTML templates for each.

[example.tif - this site is based on the content management system described in this article]

There are still some things to bear in mind, though. As discussed in last month's issue, error-trapping code has been included at every stage of the page-creation routine. This is important with any user-facing script, because it's so easy for things to go wrong, either due to a mis-typed URL or through malice. Next month we'll take things a step further, putting in sub-routines that are specifically designed to minimise the risk of hacking. We'll also look at ways in which you can remap the existing URLs on your site to match the new ones, so that you don't lose traffic or generate a rash of '404 page not found' errors when you convert your site to the CMS model.

[sidebar - What you'll need]

All the scripts, data files and other components of the CMS are included on the cover CD; make sure you maintain the existing directory format when copying it to your server. For an explanation of the use and purpose of any files not mentioned in this month's issue, please see earlier features in this series.

In order to use this content management system you'll need a Perl interpreter running on your Web server. If your server's running on a Unix box (including Linux) then you'll find it's almost impossible to install Apache or Zeus without also installing Perl. The same is not true of Windows servers running IIS, but a free port of Perl for 32-bit Windows machines is available from www.activestate.com. This comes with all documentation and is simple to install. The only caveat is that file paths may be slightly different compared with a UNIX box. If in doubt, use absolute rather than relative paths (e.g. /site/www/data rather than ../data)

Perl scripts need to be set to 'executable' in order to run. Using your FTP client, you should be able to right-click on a script once you've uploaded it to your server and select 'File properties' or 'File attributes' or 'chmod'. Then you'll be asked to define the file's attributes. The shortcut for this is to type '755' into the 'chmod/manual' box if your FTP client allows it. If not, you need to set the permissions as follows;

owner: read, write and execute
group: read and execute
public: read and execute

Generally this is only necessary with Unix servers. With Windows servers you probably won't need to change any permissions (or indeed be able to). But you may have to tweak IIS to ensure that .pl and .cgi files in the cgi-bin or cgi-local folder are executable.

We will be using some additional Perl modules that may or may not be included as part of your standard configuration. We'll make a note of these as we progress through the CMS development, but you will definitely need the "Time::Local" and "Image::Size" modules. If these are not already included in your server, you can download them from www.cpan.org. See your Perl documentation for notes on installing new modules; usually it requires a single 'install' command.

[sidebar end]

[end]