Copyright: copyright 1996 KPGB Software


Contents

The TFParse Delphi component makes it easy to parse words from files, strings, or PChars.  It allows you to carefully define what makes a word and what words will be parsed.  TFParse makes it easy to perform tasks like indexing words for full text search and retrieval applications.

For instance, using the NormalCharacters property you can define what characters will be considered to be parts of words (non-whitespace).  And, using the SignificantCharacters property you can define which characters will be considered to be parts of words only if they are directly between two NormalCharacters.  Otherwise the character will be considered to be whitespace.  Other properties allow you to narrow down even further which words will be accepted as being parsable.

While any type of file, string, or PChar may be parsed, at this time, only ascii text and HTML files/strings/PChars (See DocumentType) can be reliably parsed.  When processing anything else, only ascii characters will be parsed, however you may end up retrieving many garbage words, for instance you may retrieve formatting information when parsing wordprocessing documents.  Future versions may include the ability to parse RTF, and WORD, and possibly other types of documents.

This component works under both Delphi 1 and Delphi 2.

REGISTRATION:    $12        (SWREG# 12486)

This component is SHAREWARE.  If you find this component to be useful, PLEASE register on Compuserve.  GO SWREG and register using the number 12486.  I have kept the price at a minimum as I am only trying to recoup some of the cost and justify the time I have put into developing this component.  While there is nothing that cripples or keeps you from running this software if you do not register, I would hope that if you find this component useful and use it in your software that you would not find $12 to be too much to register,  get your name onto the mailing list letting you know about any future upgrades and receive some support.

If you find any bugs, or would like to see any particular added features,  feel free to let me know. I'll fix any bugs and will consider any feasible upgrades.

My email address is:

Compuserve:   75221,1057

Internet:	         75221.1057@compuserve.com

See Also:

TFParse Component Installation

TFParse Properties

Properties to Set:

NormalCharacters

SignificantCharacters

InsignificantChars

FileToParse

StringToParse

PCharToParse

DocumentType

MAX_WORD_LENGTH

MIN_WORD_LENGTH

CommonWords

CmnWrdsActive

AutoDetect

Function Calls:

GetNext

PercentDone

Events:

OnHTMLTag


Installation

How to Install the Component

Put the Component files (FPARSE.PAS and FPARSE.DCR) into a directory where you will keep 3rd party components. If you are installing this component for Delphi 1 then copy the FPARSE16.DCR instead, and rename it to FPARSE.DCR.   Then in Delphi click the Options|Install Components menu selection.  Be sure that the directory with the component files is included in the 'Search Path' edit box on this screen.  If not, add it.  Next, click the Add button and then the Browse button and find the directory that you just copied the component's files to.  Find  and select the FPARSE.PAS file.  Now click OK.  Click OK again and the component will be compiled. You are now out of the Install Components dialog box.  You will find the TFParse component has been added to the Samples pallette.

How to Install the Component's Help

Move theTFPARSE.HLP file from your component directory to your DELPHI\BIN directory.  Move the TFPARSE.KWF file to your DELPHI\HELP directory.  Now run the HELPINST.EXE (either double click from File Manager in the HELP directory or from the icon in Program Manager.   Now, while in HELPINST, click the open file toolbar button and find and select the DELPHI.HDX file in the DELPHI\BIN directory.  Then click the '+' toolbar button and find and select the TFPARSE.KWF file in the DELPHI\HELP directory.  This will add the TFPARSE.KWF file to the listbox that you can see.  Now click the 'Save' toolbar button and the TFPARSE help file will be registered with Delphi.  You may now exit.

Installing help in this way will allow you to get context sensitive help on each property and event by pressing F1 while a particular property or event is selected in the Object Inspector.


NormalCharacters

Default Value:  0-9A-Za-z

Type: String

Published

The NormalCharacters property  defines those characters that will NOT be considered to be whitespace.  These characters, as they are being parsed, will be considered to be parts of words.  Contiguous NormalCharacters with whitespace on either side is what constitutes a word.

You may define ranges of characters by putting a '-' in between two characters.  For instance:

0-9 means all characters from zero through nine which would be the same as if you had entered 0123456789.

A-Z means all capital letters in the alphabet.

a-z would be all lowercase letters in the alphabet.

:-@ would be all symbols in the ascii chart from : through @ which would be the same as entering  :;<=>?@

If you wish to enter the '-' character as a NormalCharacter then, either it must be within a range that has been entered, or you can enter it as --- (that's three dashes in a row) which essentially defines a range of one character.

Enter the character values without spaces or commas or any other delimeters because those delimeters will be considered to be characters in the set of NormalCharacters.


SignificantCharacters

Default Value: ./

Type: String

Published

The SignificantCharacters property allows you to specify characters that, while normally considered to be whitespace, will be considered to be part of a word if it is embedded between two NormalCharacters. This allows you to capture things like dates and floating point numbers.  For instance, using the default values for NormalCharacters and SignificantCharacters:

If the '.' (period symbol) was removed from the set of SignificantCharacters then the number '145.97' would parse as 2 separate words. Those words would be '145', and '97'.  However, with '.' as a significant character, this parses as one complete number.  However, if the '.' character is not embeded between two NormalCharacters then it is ignored.  Therefore, the periods at the ends of sentences will be ignored since there will be spaces after them. That is, unless you do something silly like include the space character in the set of NormalCharacters.


InsignificantChars

Default Value: ,

Type: String

Published

The InsignificantChars property allows you to specify characters that will be removed from between two NormalCharacters as if nothing was in its place.  For instance, using the default value of ',' as an InsignificantChars, any numbers like '3,897' would simply be parsed as '3897'.  Normal commas with spaces after them will be ignored and have no effect.


MAX_WORD_LENGTH

Default Value: 20

Type: Integer

Published

This defines the longest word length that will be returned by the TFParse component.  Any word that is longer than this value will be truncated to this length.  Use the MIN_WORD_LENGTH property to define the shortest word that will be returned.


MIN_WORD_LENGTH

Default Value: 3

Type: Integer

Published

This defines the shortest word length that will be returned by the TFParse component.  Any word that is shorter than this value will ignored.  Use the MAX_WORD_LENGTH property to define thelongest word that will be returned.


FileToParse

Default Value: None

Type: String

Published

This property should be set to the path and filename of the file that should be parsed.  Remember to include the path.


CommonWords

Default Value: None -  though a list of words is supplied and can be found in CMNWRDS.TXT

Type: TStrings

Published

This allows you to define a list of words that you do not want the parser to return when they are found. Initially, this property is empty, but a list of common words has been provided and can be loaded from the file named CMNWRDS.TXT when you are in the String List Property Editor for this property, however you don't have to use this.  You may enter any words you wish or you may load a list from any text file.

You may make this property active or non-active using the CmnWrdsActive property.


CmnWrdsActive

Default Value: False

Type: Boolean

Published

Determines whether the CommonWords list should be checked for words to discard.  The CommonWords property will be checked when this property is True.


GetNext

Type: String

Function Call

Make a call to GetNext to retrieve the next word from the file that you are parsing.  When the last word has been parsed, this function returns a blank string.

Example:

theWord := Parser.GetNext;

While theWord <> '' do

   begin

      SaveTheWord; (* do something with the word *)

      Gauge.Percent := Parser.PercentDone;

      theWord := Parser.GetNext;

   end;


PercentDone

Type: LongInt

Function Call

Call PercentDone to determine what percent of the file has been parsed.

Example:

theWord := Parser.GetNext;

While theWord <> '' do

   begin

      SaveTheWord; (* do something with the word *)

      Gauge.Percent := Parser.PercentDone;

      theWord := Parser.GetNext;

   end;


StringToParse

Default Value: None

Type:String

Public

Set this property to a String that you wish to parse.  The String will be parsed just as if it were a file.


PCharToParse

Default Value: None

Type:PChar

Public

Set this property to a PChar that you wish to parse.  The null terminated string will be parsed just as if it were a file.


DocumentType

Default Value: dtAscii

Type: EDocType = (dtAscii, dtHTML)

Public

Set this property to the type of document that is to be parsed.  At this time, the valid options for this property are dtAscii and dtHTML.

dtAscii is what you should set this to if you are parsing a regular text file.

dtHTML is what you should set this property to if you wish to parse the words from an HTML file.  This will cause the parser to skip any HTML formatting commands and return any HTML tags through the OnHTMLTag event.

See Also:

AutoDetect


AutoDetect

Default Value: False

Type: Boolean

Public

When set to True, TFParse will try to determine the DocumentType.  At this time the DocumentType will be determined to be dtAscii if it is not determined to be dtHTML.

The document is determined to be dtHTML if:

1.  The file extention is .HTM

or

2. Both the character '<' AND the string 'html' (case insensitive) are found within the first 50 characters.

If either of these are true then the file is considered to be HTML.  If neither is true then it is parsed as an ascii document.

If AutoDetect is set to False, then the DocumentType property value is used.


OnHTMLTag

procedure OnHTMLTag(HTMLTag: String )

Type: Event

This event is triggered for every HTML Tag that is parsed.  The tag is passed in through the HTMLTag parameter.  Note that since this is a string, it may not be any longer than 255 characters when running in 16 bit mode.

This event will only be triggered when DocumentType is dtHTML.