ftp.wwiv.com

home *** CD-ROM | disk | FTP | other *** search

/ ftp.wwiv.com / ftp.wwiv.com.zip / ftp.wwiv.com / pub / MISC / TGARTS.ZIP / TGDEV309.ZIP / DEVEL309.DOC < prev next >

Wrap

Text File | 1997-11-06 | 68KB | 1,326 lines

Telegard 3.10 Development Kit (v3.09) November 6th, 1997 Copyright 1995,1997 by Tim Strike. All rights reserved. INTRODUCTION I _____________________________________________________________________ PREFACE This documentation was designed to help utility developers find information regarding Telegard data files and formats with relative ease and speed. | This kit is for the first 3.09 gamma (1), and may not represent | the final structures for Telegard 3.10. I have marked all changes | from the 3.01 development kit release with a vertical bar in the | left margin. For Telegard 3.0 references which have not changed | in Telegard 3.10, I have used 3.0+ as the version reference. Before developing utilities for Telegard, it would be appreciated if this document was thoroughly read. This documentation contains several discussion sections which outline methods, procedures and data file formats. The discussion section explains all of the data files that Telegard maintains, and how to access the data and what must be done to properly access the information therein. It is imperative that important items in this document are followed when developing utilities for use with Telegard. By following these specifications and helpful hints, you reduce the chance of producing a poorly written/working utility. | Telegard 3.0+ brought about a major revision in the way that many of the data files are used. You will find that many of the files with the same name no longer use the same data format; please ensure that even when UPDATING utilities that you look at the information which is provided for each file type (if they are not listed under major headings in the DEVELOPMENT DISCUSSION section, they will be listed under the heading OTHER FILES). After the discussion section follows a section on distribution. This section outlines the best ways to document your work, the best ways to ensure accurate distribution and even how to access the Telegard File Distribution Network, which will distribute your files across North America and overseas and give you the widest distribution for nothing more then a simple phone call to one of the development sites (provided that your distribution package follows the simple guidelines that are set forth in this section). | We urge the development of quality utilities for Telegard 3.10. We also urge you to look over this document in it's entirety so that you will be able to accomplish your goal of writing a utility which works properly and efficiently with Telegard. CONTACT INFORMATION The best way to contact the Telegard development team is through our support conference on Fidonet, TG_SUPPORT. In this conference we will answer all questions relating to Telegard and Telegard third-party utility development. This is also the preferred method of contact. The next best way is to contact us through Fidonet/Internet netmail. The author of this document can be reached via the following addresses: Fidonet 1:259/423 | Internet tim@telegard.net Please feel free to contact us with regard to Telegard utility development -- questions relating to data file format, access or any other questions are welcomed. INCLUDED FILES Several files have been included in this package. If any of these files are missing, please obtain a new development package from one of our development sites. DEVEL309.DOC Telegard 3.09 Development Documentation WHATSNEW.309 Telegard 3.09 Structure Change Log README.TXT 'Read-Me' information TELEGARD.H C/C++ structure file TELEGARD.INC Turbo/Borland Pascal structure file TEXT_ADF.TXT Text File Description - Archive Definitions TEXT_MDF.TXT Text File Description - Modem Definitions TEXT_PDF.TXT Text File Description - Protocol Definitions | TESTINC.PAS List structure sizes for Pascal | TESTH.C " " for C/C++ | TEST.OUT Expected output from TESTINC.PAS/TESTH.C | BITPLANE.PAS Sample routines for modifying bitplanes FILE_ID.DIZ Automated package description PACKING.LST Package file listing 97-11-05.PR Press Release: 05 Nov 97 DEVELOPMENT DISCUSSION II _____________________________________________________________________ | SOURCE CODE EXAMPLES | | I have not released any secondary packages contrary to what | previous development kits have discussed. If you wish to have | some source code examples, please provide me with a list of short | examples that you would like to see (C or Pascal). | TELEGARD 3.10 (3.09.g1) | Telegard 3.10 was developed in Borland Pascal 7.0 -- with portions compiled in C and Assembly. | All the Telegard 3.10 internal data structures use standard pascal methods and constructors, and this should be noted by utility developers who will use other languages. Structures which are | external to Telegard 3.10 (i.e. data files which other programs create and Telegard just uses, things like JAM, Squish, etc.) use either C or Pascal methods and constructors; ensure that you determine which is which and make the necessary adjustments in your programs. | C/C++ users, please remember that Turbo Pascal uses a different | string storage method. This method uses the first byte of the | string to indicate the length, and the remainder of the bytes to | store the string. The string is not necessarily null terminated. DATE FORMATS | Telegard 3.10 uses two different types of date formats; the first is a simple string format and the second is a Unix timestamp. STRING DATE FORMAT -- -- -- -- -- -- -- -- -- The string dates are always in the same format (MM/DD/YY) and are converted to other formats on the fly. This date format is stored | in a string of length 8, with "/" as the date separator. UNIX DATE FORMAT -- -- -- -- -- -- -- -- -- -- The second date format is a Unix timestamp. Effectively this is the number of seconds since midnight, January 1st, 1970. Several compilers and implementations use different algorithms to calculate the date format. Telegard uses the local system time (i.e. the time on the machine) and calculates the number of seconds since midnight, January 1st, 1970, including the extra days during all leap years (which some implementations ignore). No adjustments are made for timezone, and no conversion is done to Greenwich Mean Time (GMT). CONFIGURATION FILES Data file CONFIG.TG -- -- -- -- -- -- -- -- -- The main configuration is stored in CONFIG.TG in the main BBS directory. This file contains all paths and all necessary configuration items that pertain to operation and selection of the BBS' features. The CONFIG.TG file can be opened and modified; please note that modifications to the file will not immediately be loaded into memory in a multinode environment. If another task overwrites the file contents, another node could still overwrite and replace those contents at another time. | The hiddenPW, resvmsgarea and resvfilearea fields should *never* | be updated by an external utility, as the consequences of doing | this could be very dire. Data file SYSTEM.DAT -- -- -- -- -- -- -- -- -- The SYSTEM.DAT file contains operation variables which are very sensitive to changes, and thus, the following procedure must be followed (in single node or multinode environments): (1) Open file (2) Obtain a lock on the entire files contents - if lock can not be obtained, cycle for waiting period abort w/error if still unable (3) Read the current data record into memory (4) Update record (do *NOT* sit idle waiting for input or other processes) (5) Write the new information to the file (6) Unlock file contents, close file The information in this record must be current, and always must be accessed by only one source at a time. SEARCHING INDEXES Several of the data files used in Telegard 3.0+ are indexed to make searching for data quicker and easier to accomplish. The index files, marked with TG_INDEX in the structures, contain different information but are built and maintained the same way. The purpose of the index files (*.IDX) is to ensure quick access to the information. The index file design used in Telegard facilitates not only the searching of existing records, but also the addition of new records. The records in the index files all contain some data [key] and a number location [offset]. The [key] is used to find the correct record, and the [offset] can be used with another file to directly load the necessary information. The index files (*.IDX) have two distinct sections; the first is a sorted database, and the second part is an unsorted database (used primarily for additions to the index). A binary search is used for the first section, and a sequential search for the second section. The binary search uses the sorted characteristic to chop the search area in half each time; this results in a very quick search of the sorted section of the database. If the [key] can not be found there, the sequential search will check through the small unsorted portion. If the [key] is not found there, the [key] does not exist. The important thing to note about this indexing method is that the sorted database will be significantly large compared to the unsorted portion. Frequent updates of the database using the indexing utilities provided with Telegard will ensure that the fastest searching time is always maintained. When the index files are first built, all valid records will be added to the file and sorted. When new records are created, the records will be added to the end of the database and remain there until index maintenance is completed. The first record in the index files contains the record offset of the first unsorted record. This is important to note, since the binary search should ONLY be run on the database starting at record #1 and ending at the last sorted record. The sequential search should start at the first unsorted record and end at the last record in the index file. Record 0 Unsorted index record [offset=NS] Record 1 First sorted record, if NS <> 1 Record NS-1 Last sorted record Record NS First unsorted record -- could be <EOF> to <EOF> Last unsorted record To search for a [key] the following procedure should be followed: 1. Read record #0, store [offset]=NS 2. Binary search on record #1 through record NS-1. If [key] found, skip to FOUND. 3. Sequential search of record NS through <EOF>. If [key] found, skip to FOUND. 4. [key] does not exist, skip to END. FOUND. If [offset] = -1, [key] does not exist, skip to END. Otherwise, [key] exists, use [offset] to find data. END. The indexing methods used in Telegard 3.09 provide fast and efficient loading and manipulation of the data. There are other ways to search for information, but if the index files exist for the information you are looking for, they are the best way to find the information. ADDING INDEX RECORDS -- -- -- -- -- -- -- -- -- Search the index to see if a record already exists for that [key]. If it does exist, there are two possibilities (a) this is an adjusting record, in which case adjust the [offset] field and rewrite the record or (b) this [key] already exists and does not need to be added, or can not be added. If is does not exist, simply create the necessary record and then write it to the end of the index file (simple, eh?). REMOVING INDEX RECORDS -- -- -- -- -- -- -- -- Search the index for the necessary [key] - if a record already exists, set the [offset] field to -1 and rewrite the record. If the record doesn't exist, it can't be deleted now can it (smile). USERS | Telegard 3.0+ no longer uses USER NUMBERS to find records in the data files. Record #2, therefore, is not User #2 like older versions of Telegard had implemented. USER-ID -- -- -- -- -- -- -- -- -- -- -- Since user numbers were no longer tied to user records, a user-id was created to give each user a 'id-tag' that was not associated with user/real names (since those change frequently). The user-id number is generated when the user first logs on and is ordered sequentially from 1 (i.e. the first available user-id after 0 is assigned to the new user). This user-id field is only used to reference the user' records in other files. It is *not* associated with the users data file position; i.e. user-id #2 is not necessarily found in record position #2. PASSWORD TEXT/CRC FIELDS -- -- -- -- -- -- -- -- The pwtext field *may* be empty. Please observe the following when updating passwords: 1. All password input is UPPERCASE 2. If ConfigRec.HiddenPW = TRUE, then only the CRC32 of the password is stored in the user record (userrec.crcpw). 3. If ConfigRec.HiddenPW = FALSE, both the CRC32 and the actual password are stored in the user record. *4. When checking input passwords against the file, only the CRC32 value should be tested (the pwtext may or may not be correct). If ConfigRec.HiddenPW = FALSE, and the pwtext field is EMPTY, then after a correct password the field will be automatically updated. If ConfigRec.HiddenPW = TRUE, the pwtext will *not* be updated (this is obvious). *5. Passwords should *only* be displayed (pwtext) if the CRC32 of the pwtext field *matches* the pwcrc field. Otherwise, the pwtext field is likely incorrect. When changing passwords, the user should be prompted only, not shown their previous password. *6. The CRC32 calculation used by Telegard is a standard CRC32 calculation. The initial value is 0xFFFFFFFF (FFFFFFFFh) and only the contents of the password are used (i.e. start at position 1, not 0 (the length byte). Remember, the passwords are in uppercase. Data file USERS.DAT -- -- -- -- -- -- -- -- -- The USERS.DAT data file has no specific order, except that the SysOp Record must be record #0. The data file may be sorted in any manner that is deemed important, except that record #0 should never be moved; that is the only restriction to the order of the file. Data file USERS.IDX [TG_INDEX] -- -- -- -- -- -- -- To speed up data searches a USERS.IDX file has been created (replacing NAMES.LST from older versions of Telegard). The index file contains both user aliases and real names (if different) so that searches can be completed using either one. The offset provided in the index record is the record position offset in USERS.DAT. Data file USERID.IDX [TG_INDEX] -- -- -- -- -- -- To speed up associative data searches, a USERID.IDX file has been created. The index file contains the userid's of all active users so that searches can be completed by finding the matching id. The offset provided in the index record is the record position offset in USERS.DAT. SEARCHING FOR USERS -- -- -- -- -- -- -- -- -- Use the index files to quickly search for users (user name/alias or id) or use a sequential search on USERS.DAT to search for other fields. DELETING USERS -- -- -- -- -- -- -- -- -- -- Deleting users from the database is a relatively simple task, but has many associated items which must be properly executed for the data files to be tidy and efficient. (1) Remove user records from *.IDX files (see section on TELEGARD INDEXES - REMOVING INDEX RECORDS ) (2) Remove all email (base 0) to user (3) Remove all shortmsgs (SHORTMSG.DAT) to user (4) Adjust voting records for user (VOTING.DAT - adjust totals) (5) Reset all lastread pointers to ZERO (6) Remove macro record of user (null MACRO.DAT.UserId, set macro pointer to -1) (7) Set USERID field to ZERO (8) Set udeleted in user flags Following those instructions will lead to the most compact data files and ensure that data records are not accidentally cross-linked. PACKING USERS -- -- -- -- -- -- -- -- -- -- All deleted users can be removed from the data file by simply removing their data record (provided that the users were properly deleted as above). All this entails is that the users record is overwritten with an active record, or a new data file is created with only the active user records). SORTING USERS -- -- -- -- -- -- -- -- -- -- Since the dependency on user numbers no longer exists, user records can be sorted in any manner with any method. The only restriction (as noted above) is that Record #0 must remain as Record #0 at all times (it should *not* be sorted). | SCAN RECORDS | | Telegard 3.10 uses a new method for storing user scan records; | this method is considerably faster, and uses less storage than | it's predecessor found in Telegard 3.02-. Unfortunately with | that speed increase also comes some complexity. | | 1) File and message areas can not be arbitrarily inserted | and deleted without adjusting the scan records | 2) File and message areas can not be updated while a user | is online (since their records would not get updated) | | The speed increase is accomplished by loading all the scan records | from a file, and storing them in memory. All comparisons are then | done from memory instead of from disk. This will make day-to-day | operation of the BBS much faster. | | In order to maintain the same level of message and file area | support (up to 32,000+ areas each), without sacrificing huge | amounts of storage space or memory, these tables of boolean values | (true or false, 1 or 0) are compressed into bitplanes. So each | byte can store 8 areas instead of only 1. | | The sample routines BITPLANE.PAS (No C/C++ version yet, sorry | folks; ask and ye shall receive however) have been included for | manipulating these bitplanes. These are the same routines used | within Telegard. | | The MSCAN.DAT holds Message Scan records, and FSCAN.DAT holds File | Scan records. The offset to any given user is | (userid-1)*(systat.resv...areas). The bitplane is then stored in | systat.resv...areas sequential bytes. So, to find the msg scan | records for userid N, the following is done: | | memalloc bitplane,systat.resvmsgareas | open MSCAN.DAT; | if (N-1)*(systat.resvmsgareas) < filesize | seek (N-1)*(systat.resvmsgareas); | read bitplane,systat.resvmsgareas; | else | // record does not exist | endif | close MSCAN.DAT | { . . . use as desired . . . } | dealloc bitplane | | NOTE: The user may not have any scan records stored if they are a | new user and have not successfully completed a full login. MESSAGES | Telegard 3.0+ no longer uses proprietary message formats, but instead uses the standard formats SQUISH and JAM. These two formats provide standard access to the message bases, are considerably more adaptable and provide similar performance to the old Telegard formats without losing compatibility. Data file MAREAS.DAT -- -- -- -- -- -- -- -- -- The MAREAS.DAT data file stores all the datafile definitions for Telegard message bases. There are only a few areas of note in the record format which will be outlined shortly. The message bases are stored in numerical order (i.e. base #1 is record #1) -- this number will correspond to the message base number within Telegard unless the base compression tables are on (this will have to be handled differently by other utilities). * If the FileName is blank, the message base is not active and should not be processed. | * JAM/Squish files will be stored in the MsgPath field -- if | they are not there, create them there. * The QwkIndex field should be a permanent number which should not be changed except for extreme repair procedures or for controlled maintenance purposes; the QwkIndex field is best changed by the indexing utility provided with Telegard. * When changing a FileName the MAREAS.IDX file should be updated; the old record should be deleted and the new record should be created (see section on TELEGARD INDEXES - ADDING AND REMOVING INDEX RECORDS). * Similarly, if and when changing a QwkIndex number, the QWK.IDX file must be updated in the same fashion. Data file MAREAS.IDX [TG_INDEX] -- -- -- -- -- -- The MAREAS.IDX file contains the base filenames and file offsets into the file MAREAS.DAT for those bases. This file can be used to quickly locate a specific base by filename. Data file QWK.IDX [TG_INDEX] -- -- -- -- -- -- -- The QWK.IDX file contains the QWK base numbers (QwkIndex) and the file offsets into the file MAREAS.DAT for those bases. This file can be used to quickly locate a specific base by QwkIndex number. JAM FORMAT -- -- -- -- -- -- -- -- -- -- -- The JAM format uses an expandable record format which essentially allows it to grow as needed for application purposes. The purpose of this section is not to outline how JAM works (the JAM API released by the authors of JAM provides code examples and a text development document) but rather to just quickly outline the format and some assumptions used with Telegard 3.0+. The *.JHR file contains most of the information that is required for messages. The first record contains information about the base that is being looked at. The records afterward are variable in length because of the numerous subfields that may or may not exist for each message: jaminforec jamhdrrec[1] subfield[1] subfield character buffer : subfield[n] subfield character buffer jamhdrrec[2] : jamhdrrec[n] | The subfield codes used by Telegard 3.0+ match all the standard JAM subfield codes. ** NOTE: For bases where anonymous, anyname, etc postings are used, the sender/receiver names will be "anonymous", "anyname", etc. The FTS-kludge "^aREALNAME: <name>" will be used to store the posting users name (which will depend on the realname/alias toggled on the base). This kludge uses the standard subfield-id of 2000. JAM stores its own lastread pointers in the *.JLR files. To find the correct lastread record for any user, a sequential search for the matching user-id should be done. SQUISH FORMAT -- -- -- -- -- -- -- -- -- -- The Squish format is another dynamic message base, which allocates itself in data chunks (frames) and links itself on the fly. The Squish bases automatically attempt to pack themselves for all newly posted or incoming messages, which alleviates the job of running a packer each night for the Squish message format (it is however a good idea to run the packer on a frequent basis to remove the unused frames and reduce disk usage (even at the lowest level)). The Squish format is outlined well by it's author in the SQUISH API package, which includes a comprehensive discussion of the Squish format and methods for reading, writing and indexing the message areas. The Squish format uses it's own lastread pointers, stored in the *.SQL files. To find the lastread record for any user, the offset in the *.SQL file is userrec.userid-1 (so if the userid = 2, the offset = 1 and therefore the record position = 1). QWK OFFLINE MAIL FORMAT -- -- -- -- -- -- -- -- Telegard 3.0+ implements the QWK mail format as per the QWK specifications (covered well in the document package dealing with the QWK layout, normally distributed with filenames similar to QWKLAYnn.ZIP, where nn is the revision number (the latest at the time of this release was nn=17). Data file *.QPT -- -- -- -- -- -- -- -- -- -- | Telegard 3.0+ also adds it's own pointer file, *.QPT which stores the lastread pointers for the requested message bases. This file only contains the QWK base number and the lastread number, and is sent with the QWK offline mail package to the user. FILES | Telegard 3.10 uses a proprietary file base format, since there are few standard filebase formats which are suitable and efficient. Data file FAREAS.DAT -- -- -- -- -- -- -- -- -- The FAREAS.DAT data file stores all the datafile definitions for Telegard file bases. There are only a few areas of note in the record format which will be outlined shortly. The file bases are stored in numerical order (i.e. base #1 is record #1) -- this number will correspond to the file base number within Telegard unless the base compression tables are on (this will have to be handled differently by other utilities). * If the FileName is blank, the file base is not active and should not be processed. | * *.FA and *.FAD files will be stored in the FilePath field | -- if they are not there, create them there. * Files are actually stored in the path pointed to be FAreaRec.Path. Offline files are obviously not here. Check CD-ROM status to check access for writing to this path (read-only toggle means no writing!). * Telegard keeps track of CD-ROM titles that are in the drive at the current time - if the FAreaRec.CdLabel field contains a value and the area is toggled as CD-ROM Telegard will check the drive for the correct CD-ROM before trying to download/access. * When changing a FileName the QFAREAS.IDX file should be updated; the old record should be deleted and the new record should be created (see section on TELEGARD INDEXES - ADDING AND REMOVING INDEX RECORDS). Data file FAREAS.IDX [TG_INDEX] -- -- -- -- -- -- The FAREAS.IDX file contains the base filenames and file offsets into the file FAREAS.DAT for those bases. This file can be used to quickly locate a specific base by filename. | Data file *.FA -- -- -- -- -- -- -- -- -- -- | The *.FA data files contain the actual file records for the files that are available and currently in that file base. File records are stored with a variable length description (which will be touched on later). | Unlike previous versions of Telegard (2.7), record #0 is now used for file records. Deleted records are completely removed from the file instead of sitting at the end of the file (like previous versions as well). The fields in the file records are not updated on the fly; if a file is moved offline external to Telegard, then Telegard will not know this unless the file has its status changed. | Data file *.FAD -- -- -- -- -- -- -- -- -- -- | | The *.FAD data files contain the extended file descriptions | (stored in variable length text blocks). | Variable Length Descriptions -- -- -- -- -- -- -- | | As mentioned before, Telegard stores the file records with | variable length descriptions, and uses a description ID key to | ensure that the correct description is being used. | | The descriptions are organized as follows: | | File (FBREC) | DescOffset { offset into .FAD for desc } | DescLength { length of description, not } | { including description key } | File (*.FAD) | Key [13 bytes] { match to FBREC.Filename } | Bounded Text Buffer { description of DescLength } | (0..DescLength..MAXFDESCLENGTH) { length } | | Telegard expects the text in the description to be written with | CR where appropriate. In addition, Telegard currently expects the | descriptions to be limited to 45 characters per line, although | Telegard will reformat (as configured) as necessary. | Data file QFILES.IDX [TG_INDEX] -- -- -- -- -- -- | The data file QFILES.IDX contains a listing of all the filenames | that are currently available on the system. It can be used to check for duplicate files, to find files quickly (i.e. Request Processors) or many other useful tasks. The file *must* be | maintained properly. A file is unique to the filename and area | that it is stored in; two files can not have the same filename and | be stored in the same area, although they can have the same | filename and be stored in different areas. ADDING FILES -- -- -- -- -- -- -- -- -- -- When adding a file to a directory, the index should first be | checked for a matching filename & area number. If the file | already exists (and is not listed in the index as fnDeleted), the | utility should either (a) replace the old file with the new file, | removing the old file if necessary from other areas or (b) abort | adding the new file to the database. | To add a file, simply write it's record into the correct *.FA | file, the description to the .FAD file. Then add the file into | the QFILES.IDX (see information on Telegard Indexes - Adding Index | Records). DO NOT WORRY ABOUT UPDATING/REPLACING .IDX RECORDS | WHICH HAVE FNDELETED TOGGLED TRUE--THIS IS NOT NECESSARY. DELETING FILES -- -- -- -- -- -- -- -- -- -- A little more care should be taken when deleting files. The file | should be removed from the *.FA file (rewrite all records after that record to their current position-1, truncate the last record | from the file -- this ensures that the order of the *.FA file is | maintained). The .FAD file does not need to be touched. Finally, | the record must be removed from the QFILES.IDX file -- this is | done by toggling the fnDeleted flag on the appropriate QFILES | record. DO NOT DELETE THE .IDX RECORD OR SET THE FILEAREA TO -1 | AS WAS PREVIOUSLY EXPECTED. The file should be removed from the | filepath if (a) necessary and (b) if possible. Data File PROTOCOL.DAT and Transfer Protocols -- -- -- -- Telegard maintains the transfer protocols in a file called PROTOCOL.DAT. Internally to Telegard, the protocols are referred to as A-Z (i.e. 26 protocols can be stored in Telegard - one record per protocol type). Unlike previous versions of Telegard, single, batch and resume protocols no longer require separate records. Record #0 refers to Protocol A, and record #25 refers to Protocol Z. All the protocol records are created in the file at once, even if they are not used. If the ProtRec.Batch field is toggled TRUE, then the files will be compiled into a filelist and this filelist will be sent as the protocol filename (~PF). Most protocols require a special flag to list this file as a 'list' of files - this should be stored in the ProtRec.flist field. If the ProtRec.Bidirec field is toggled TRUE, Telegard will search the temporary incoming directory for files after the download. If ProtRec.Desc field is null, the protocol has not been defined. | Data file *.QQQ, *.QQD - Download Queues -- -- -- -- -- | The download queues are stored in the temporary directory as the | files *.QQQ and *.QQD. The download queues store information | about flagged files that are pending download by the user. | Information may be added/removed from these queues during a shell | since the queues will be re-processed after the shell. | The maximum number of queued files is maxqueue, any entries after | the maxqueue number will not be read by Telegard. | The description queues (*.QQD) use the same format as the file | area description files (*.FAD). Data file TESTINFO.DAT -- -- -- -- -- -- -- -- TESTINFO.DAT is a file that is created by virus scanners and upload processors. It includes information on file conversions, descriptions, testing information, etc. Telegard will read this file (if toggled on) to process conversions and get information regarding the file testing that has just occurred (after the virus scanner has been run on the file). Records which are not scanned for a period of time can be removed from the TESTINFO.DAT file - this indicates that errors have occurred with the importation of these file records. THDPRO V11 and subsequent versions will produce this file. At the time of this writing, this is the only known product to use the format. Telegard will only work with THDPRO V11, since prior | versions do not include the TG3.0+ product code. OTHER FILES If the data file is not listed below, then no extra information is required to access the data file; the record format is straight forward, and the data is not organized in any varying manner (usually there is only one record). Data file *.MNU -- -- -- -- -- -- -- -- -- -- The menu files are stored in a number of different paths; the first being the default *.MNU path, stored in CONFIG.TG. The other paths are language specific paths (for more information see section LANGUAGE CONSIDERATIONS). The menu files have a base header (MenuRec) and are followed by up to MaxMenuCommands of the menu command records (CommandRec). The *.MNU files must be opened in non-record specific file formats (i.e. for those Turbo Pascal people, define the type as FILE and use BLOCKREAD/BLOCKWRITE to access the records). Data file *.BBS -- -- -- -- -- -- -- -- -- -- The BBSlisting files are stored in the ConfigRec.DataPath and contain the BBS listing entries. To add an entry, add to the end of the file. To remove an entry, physically remove the record. If updating the entry, be sure to touch the BBSListRec.LastEdit field with the current date. MCI codes should be removed from all BBSlist entries. Data file ARCHIVE.DAT -- -- -- -- -- -- -- -- -- Telegard maintains the archivers in a file called ARCHIVE.DAT. Internally to Telegard, the archivers are referred to as 1-MaxArcs (i.e. MaxArcs archivers be stored in Telegard - one record per archive type). Record #0 refers to Archiver #1, Record #1 to Archiver #2, etc. All the archive records are created in the file at once, even if they are not used. If ArchiveRec.Extension field is null, the archiver has not been defined. Data file EVENTS.DAT -- -- -- -- -- -- -- -- -- The events that Telegard maintains are stored in the file EVENTS.DAT. This file contains an array of all the events: Event(#1) - Event(MaxEvents) and includes a reserved record Event(#0) which is used for processing external events. If the event is forced, users will be kicked off when the event time occurs. If the event is not forced, users will be allowed to stay on past the duration of the event and the event will be run afterwards. If Telegard is loaded from an external program (Mailer, external call answer door, etc.) the Telegard events will still be adhered to. If the External Event command line is used (-X[n]) then Telegard will also include this event time in its event calculations. Data file GROUPS.DAT -- -- -- -- -- -- -- -- -- The groups that Telegard maintains are stored in the file GROUPS.DAT. This file contains an array of all the groups: Group @ is record #0, Group A is record #1, etc. If GroupsRec.Name field is null, the group has not been defined. | The GROUPS.DAT file contains two records, the primary and | secondary group sets (main & file). Data file HISTORY.DAT -- -- -- -- -- -- -- -- -- Telegard stores the call information in a file called HISTORY.DAT. Unlike previous versions of Telegard in which this file was stored | in reverse order, Telegard 3.0+ stores this file in ascending order - i.e. the first record is the oldest record, the last record is the newest record. To find the current days entry, check the last record in the file. If the dates do not match up, the daily turnover event has not been run; a new record can be created for the new day (this will however result in Telegard not processing it's turnover events, which include packing away the SysOp logs and other minor maintenance processes) or the old record can be added to. Data file IEMSI.DAT -- -- -- -- -- -- -- -- -- The IEMSI.DAT data file is created in the \temp directory only if a user connects with IEMSI. The file contains the IEMSI record as Telegard has processed the incoming IEMSI information. At the end of this record will be the actual buffer dump of the IEMSI handshaking package which was received by the system - just in case for some reason it is required or needed. The IEMSI buffer/protocol contains several fields which Telegard can not process (either lack of functionality, or no real purpose to process). See the record structure for specific entries which Telegard will not do anything with. Data file LANGUAGE.DAT -- -- -- -- -- -- -- -- The LANGUAGE.DAT file contains entries for all active languages that have been defined for the system. There are a couple of fields of note which should be looked at when searching to load menus or to display text files -- Telegard searches the language specific entries first, and then if that fails, looks at the default entries (if the language is setup to do that): LANGUAGEREC RECORD CONFIGREC RECORD YES START -> Is path field empty/null? -----> Search for file in | | default path (MenuPath/ NO | | TextPath). | | File found? -----> Use Search for file in defined | | YES YES path (MenuPath/TextPath). | NO | Use <----- File found? | | | | File does not exist. | NO | | | Check defined boolean for | further searching. | (CheckMenus/CheckText). | Boolean = TRUE --------------' | NO | | File does not exist. The actual language text is stored in the *.TGL file with the filename LanguageRec.Filename. There are more considerations for | Telegard 3.0+ with it's new multilingual capabilities; see the section titled LANGUAGE CONSIDERATIONS. Data file LASTON.DAT -- -- -- -- -- -- -- -- -- The LASTON.DAT file is kept in sequential order - the last record is therefore the newest record, and the first record is the oldest. The LCallers record is written when the user first logs on. The LCallers.Caller field is used to update the record when the user logs off to include the logoff time and download totals. If writing a utility which handles this information, please observe the HIDDEN flag for any records that are going to be displayed, and only display to the users with Co-SysOp ACS or above. Data file LEVELS.DAT -- -- -- -- -- -- -- -- -- Telegard maintains the validation levels in a file called LEVELS.DAT. Internally to Telegard, the levels are referred to as A-Z (i.e. 26 levels can be stored in Telegard). Record #0 refers to Level A, and record #25 refers to Level Z. All the level records are created in the file at once, even if they are not used. If LevelsRec.Desc field is null, the level has not been defined. Data file MACRO.DAT -- -- -- -- -- -- -- -- -- Telegard stores the macros in this file. When searching for a users macro record, check UserRec.MacroPtr and check there for the macro offset. If this record does not match the correct User-ID, then set the UserRec.MacroPtr field to -1. The order of the macros in the array is alphabetical; ^D,^E,^F and ^R. Data file MODEM.DAT/MODEMnnn.DAT -- -- -- -- -- -- Modem information for single node boards is stored in MODEM.DAT. For multinode boards, the default modem information is stored in MODEM.DAT (will be used for all new node entries). Each node uses the file MODEMnnn.DAT where nnn is the node number of that node. | Telegard 3.0+ uses VERBOSE result codes instead of NUMERIC result codes. This should make the configuration of modems considerably easier. If Telegard detects the ModemRec.Code_ARQ string in the connect string then the connection is considered Error Correcting. The modems are very sensitive to the ModemRec.Delay* fields; adjusting these fields even slightly will affect most modems considerably. | Telegard 3.0+ stores the IRQ/Base I/O address for the port. These are purely informational for Telegard - since TG uses the fossil drivers, non-standard ports are addressed through the fossil driver. However, GSZ/DSZ and several other protocols access the ports directly and may need this information on a per-node basis. For more multinode considerations, see the section titled MULTINODE CONSIDERATIONS. Data file NODES.DAT -- -- -- -- -- -- -- -- -- Stores information on the node configuration (acs, logon times, phone numbers) and the whos online information for each user. The status text can be freely modified (it's displayed with an MCI code in the language file) -- as long as the statusid is correctly maintained for each node. Data file SHORTMSG.DAT/INODEnnn.MSG -- -- -- -- -- -- When searching for short messages to a user, search the file for matching user-id fields -- the messages are in no particular order, so the entire file will need to be sequentially searched. The status ID should always be 0 for all external Telegard utilities. Telegard itself uses this field for internal transfer stubs, and using a value other than 0 for external utilities can potentially cause many problems. To send a message to an online user for them to read, write the message to the appropriate INODEnnn.MSG file. If the user logs off before the message is received, it will be deleted. To delete the short message (when deleting a user or after the message has been sent to the user), set the ShortMsgRec.UserId field to 0 (instead of the normal -1 as most other deleted records do). Do not delete short messages from the INODEnnn.MSG file. Telegard will clean these up as neccessary (on the next login to that node). | Data file SIGS.DAT -- -- -- -- -- -- -- -- -- | | Telegard stores the user signatures in this file. When searching | for a users signature record, check UserRec.SigPtr and check there | for the signature offset. If it is not -1, then seek to this | record offset and read the record. If the signature record does | not match the correct userid, then set UserRec.SigPtr field to -1. Data file TIMELOG.DAT -- -- -- -- -- -- -- -- -- The TIMELOG.DAT data file contains the time tracking graphs that each of the nodes maintains. For single node systems, Record #0 is the only record that is maintained. For multinode systems, Record #0 is for Node#1, Record #1 is for Node#2, etc. The actual format of the data is not too significant, however: TimeLogRec.BusyPerHour contains the number of minutes in each hour breakdown that the node has been active for all days during the week. TimeLogRec.BusyPerDay breaks down that information into 20 minute chunks for separate days of the week. Both fields should be updated, rolling time past midnight if necessary. While it is possible to calculate the TimeLogRec.BusyPerHour fields from the TimeLogRec.BusyPerDay fields on the fly, that wasn't the way that I implemented the structure (grin). The TIMELOG.DAT data file can be deleted to restart the time tracking variables -- or alternatively the entire record can be set to null (fill all fields with chr #0). MULTINODE CONSIDERATIONS | Telegard 3.0+ opens all of its files in SHARE mode -- this means that for a multinode environment to exist, SHARE (or the OS | equivalent) must be loaded. Telegard 3.0+ opens most of its files in fmReadWrite+fmDenyNone mode, although occasionally Telegard uses fmRead+fmDenyNone and fmWrite+fmDenyNone. When files are being critically updated, Telegard will open the file in fmDenyAll mode (and will only keep the file open a short period of time). | A few circumstances arrive where Telegard 3.0+ actually LOCKS the file contents; JAM, Squish and the SYSTEM.DAT file are all cases where this is done. For JAM and Squish, follow their respective development documents, and for SYSTEM.DAT see the information earlier in the document. When opening files in a multinode environment, there are a few things to keep in mind: (a) Open the file in fmReadWrite+fmDenyNone unless there is a specific reason you need to open the file in another file mode. (b) All data files should be open as _little_ as possible (i.e. do not keep a file open while waiting for input for example). Utility authors should not ignore the existence of multinode systems -- if your utility can't co-exist in a multinode environment, please specify it is for SINGLE node systems or for systems which are shut down ENTIRELY for a maintenance event (i.e. no nodes are left active during maintenance). LANGUAGE SUPPORT | Telegard 3.0+ introduced multilingual support for the BBS -- this means that different language strings can be setup and chosen by the online user to use. While it is not necessary for Utility authors (esp. those with with interactive doors) to support multiple languages, keep in mind that your product could be used on several different language platforms, and multilingual support could be beneficial. There are two common ways to handle multiple languages; the first is to create a text file which is loaded on the fly (good for small amounts of text), and the second is a data file which stores different strings for different languages. Both methods are equally useful, and both have their drawbacks. If you intend on supporting multiple languages, the choice of which method to use is entirely up to you. You can load the current users language from their user record, or you can have it passed on the command line with MCI codes. The choice there is also up to the utility developer. Telegard language file structure is *not* distributed at the present time. If you need text strings from a Telegard language file, please get in touch with the development team for more information. The text strings are not always at the same file offset, so seeking into the file to get around the lack of structures will not consistently work properly. TEXT FILE DEFINITIONS | Telegard 3.0+ added the ability to import text file definitions for archivers, protocols and modem setups. This allows standard configurations to be freely distributed and installed without the hassle of playing with the setup, or having to input all of the field information. The text file definitions can be imported and exported by Telegard 3.0+, and the format is freely available so that distribution can easily occur. SysOps are encouraged to release new archive and protocol setups (provided they work, naturally) so that other SysOps can benefit from this new distribution method. The three formats (ARCHIVE, PROTOCOL and MODEM) are all discussed in their own document files (TEXT_ADF, TEXT_PDF, TEXT_MDF respectively). UTILITY DISTRIBUTION III _____________________________________________________________________ DISTRIBUTION TYPES When deciding to distribute your archive, there are several considerations you must make -- mainly dealing with what you want in return for your utility, etc. There are several distinct types of distribution, and several that fall in between categories - FREEWARE, SHAREWARE and COMMERCIAL SOFTWARE. There are too many misconceptions and other conditions to describe each type of distribution in this document; the best method is to determine what you want, find a similar product and look at how they marketed their product. THE DISTRIBUTION ARCHIVE Most distribution archives are packed in such a way that SysOps can quickly look at what is inside and determine that they are getting everything that they need. Some SysOps will not even look at your program if they are in any way suspicious of the contents of your distribution archive. The following hints are aimed at helping you as a utility developer get the most out of your efforts. THE ARCHIVE FILENAME Almost all utilities will require more then one release version, so version numbers come in handy so that SysOps and users know what version of the program the current archive contains. Thus, the version number normally finds its way into the filename of the archive, either as FFFFFFnn.EEE or FFFFFnnn.EEE depending on how the version numbers are represented (the first is for Vn.n, the second for Vn.nn -- your choice, personal freedom). The filename could also contain many variations on the above format, such as FFnnnFFF.EEE, etc. If at all possible, shorten your archive filename and add the version number to the name -- that is the best way to ensure that SysOps will take a look at your file. And if you generally think your utility will not have any more versions, give it another thought -- most do. THE ARCHIVE FORMAT The most popular archive formats are still ZIP and ARJ. Other formats, such as RAR and LZH have followings in certain regions, but may not necessarily be used everywhere your archive may travel. You are best to stick with the most popular archive formats -- if you really want to deviate, consider self-extracting forms of your archive choice. Also, if you are considering a self-extracting form, it is suggested that you first pack your archive with the self-extracting header, and then pack that .EXE file (and any release notes) in a .ZIP file. Your release notes should be separated from the your self-extracting .EXE file -- some SysOps read the release notes to determine if they want to look at the file further. If they only see an .EXE file, they might look the other way. THE ARCHIVE CONTENTS All pieces of your program should naturally be contained in your archive (this includes all necessary data files, text files and of course the executable files). Your archive should also contain some form of documentation. Even if your program doesn't do much, you should include documentation which at least covers the following topics: (a) a brief description of the program, (b) what files and versions of those files are required -- if any, and (c) contact information for reaching the author. The program documentation normally takes on the name of the executable file with the extension of .DOC, although this isn't a standard by any means. Some software distribution networks and BBSes refuse to carry files without documentation -- consider this, and even if it's only a small 1k file that you write, at least it ensures that your archive won't be immediately deleted. Last minute release information should be put into one of several files; README.DOC, UPDATE.DOC or RELEASE.DOC. These files are commonly searched and read before any other miscellaneous files are touched, so putting last minute information into these files is a good idea. If you wish to ensure that no files are added or removed from the archive, then a text listing (in both your main documentation and a separate file called PACKING.LST) of all the files in your archive is a good idea. If you wish to ensure that your archive is always described the way that you want it, a FILE_ID.DIZ file is a great idea (in fact, many popular BBSes now import the file as the file description, regardless of what the uploading user types in). The FILE_ID.DIZ file is a text file which describes your program. This file has a specific format; the maximum number of characters on any line is 45, and the maximum number of lines is 10. PLEASE DO NOT DEVIATE FROM THIS STANDARD - I REALIZE THAT OTHER AUTHORS HAVE AND JUST BECAUSE THEY DO DOES NOT MAKE IT RIGHT. IF YOU WANT TO ENCLOSE A FILE_ID.DIZ FILE, ENCLOSE IT IN THE PROPER FORMAT. Some authors like to time stamp all the files in their archive so that all the files have the same date/time. This makes it very easy for the author to check and see what files have been changed/added to the archive, by checking the file date when unpacking. This obviously isn't something that everyone needs to do -- personal preference again. Beyond that, archives are archives. Many authors have their own personal ways of building their archives and of setting up their distribution files. You can be just as free with your decisions, just remember that when it comes to distributing your archive, you may be limiting your distribution if you don't put together a complete package. DISTRIBUTION There are several ways of distributing your final archive. These ways will be discussed below so that you can determine the best way of getting your file out into the field. ADVERTISING The best place to advertise new utilities for Telegard is in a Telegard support conference. The participants of the conference are mostly Telegard SysOps or other interested parties, and thus advertising in the support conferences will ensure that you hit your best (and most interested) audience. TG_SUPPORT (our international Fidonet conference) is probably the best place to go for this advertising. Please note however that you should advertise your program only once or twice; advertising it daily will unnerve many people and may turn them off of your product. FILE REQUESTS/MAIN BBS DOWNLOAD Setting up your archive for File Request (mailer) or BBS download ensures that people will be able to get to your file. The problem with relying on this method alone means that your utility spreads only through direct contact (i.e. the interested SysOps must call your system to get the file). This results in a very systematic and slow diffusion of your program -- although it's still recommended that you do this, there are other methods you should also consider. SOFTWARE DISTRIBUTION NETWORKS Software distribution networks distribute your file for you; you send it in to a site, which sends it to connected systems, and those systems send it to connected systems, etc. This ensures that everyone who is connected to a specific SDN gets your file automatically (and in many cases, the file is automatically added to their file bases for download as well). This method of distribution means that you program will be spread in a very hierarchical way, but will spread quickly and automatically -- two things new releases generally benefit from. There are several different distribution networks out their to distribute your utility. Most networks have different categories for different types of files; so when choosing a network, make sure that they have an area for your type of utility (i.e. a Telegard area, or a BBS utility area). The Telegard Development Team has created a software distribution network on the fidonet filebone which provides national and international distribution through Fidonet channels. This area has the tag name TG_SUP and should be available from your Fidonet hub. If you would like your file distributed via this SDN, please see the section on DISTRIBUTION BY THE TELEGARD DEVELOPMENT TEAM. FTP (FILE-TRANSFER-PROTOCOL) SITES The internet provides a service called FTP, which allows for the collection of files over the internet. This ensures that people can access your file without cost (except the cost of their Internet provider, if any) and at any time of the day, without worrying about busy signals, etc. Although I'm not aware of most of the anonymous FTP sites, if you upload your utility to one of them into a /Telegard/ area, I am sure that you will get some internet users grabbing your file. The Telegard Development Team presently operates one official | FTP site, available through FTP at ftp://telegard.net/pub/tg. If you would like your file distributed via this site, please see the section on DISTRIBUTION BY THE TELEGARD DEVELOPMENT TEAM. DISTRIBUTION BY TELEGARD DEVELOPMENT TEAM The Telegard Development Team operates many distribution sites which can help distribution your utilities (TG_SUP, ftp site and the development sites). To have your file distributed via our distribution methods, please refer to the following "to-do" list: (a) Ensure your program works (test!) (b) Include complete documentation and contact information (c) Include FILE_ID.DIZ file (d) The preferred archive format is ZIP -- if you pack in another format, please consider double packing so that your file is ultimately in ZIP format. (e) Upload to a Telegard Development Site, let the SysOp know the file is there, and provide a simple one-line description (this is for the actual SDN distribution) (f) The file will be hatched into the TG_SUP as soon as humanly possible. Within 36 hours of that hatching, the file will be placed on the FTP server. OTHER QUESTIONS IV _____________________________________________________________________ If you have any further questions on topics discussed in this document, or even topics which were not discussed, please feel free to ask us in our support conference (preferred) or contact the author of this document or get in touch with one of the Telegard Development Team sites.