ARJ for OS/2 version 2.73 Release Notes August, 19, 2000 This product is an OS/2-hosted add-on for ARJ 2.7x for DOS. It may be used separately from the DOS package but it doesn't contain the documentation found in DOS release. *********************************************************************** *** *** *** THIS PRODUCT IS DEVELOPED SEPARATELY FROM THE MAINSTREAM ARJ *** *** PACKAGE AND THEREFORE IS NOT UPDATED SYNCHRONOUSLY WITH ARJ *** *** FOR DOS. SHAREWARE REGISTRATION FOR ARJ/2 IS NOT PROVIDED BY *** *** ARJ SOFTWARE, INC. *** *** *** *********************************************************************** LEGAL NOTICE No warranties of any kind are made. In no event should ARJ Software Russia be liable for any damage resulting from use or misuse of this product. You have a royalty-free right to distribute ARJ/2 package in any form. Commercial use of this product must be approved by ARJ Software, Inc. The information in this document is a subject to change without notice. INTRODUCTION As the OS/2 lacks a multifunctional compression utility, a version of ARJ for OS/2 has been developed by ARJ Software Russia. The aim of the porting was to retain the functionality and compatibility with ARJ for DOS ("ARJ/DOS" from this point onward) but several exceptions occur. They are covered later in this document. SYSTEM REQUIREMENTS * OS/2 version 1.2 or higher, or Windows NT v 3.10 or higher. For OS/2 systems: * A 80286 or higher CPU. * 512K of memory is needed for operation. For Windows NT systems: * 1024K of memory is needed for operation. DIFFERENCES WITH THE DOS AND WINDOWS NT VERSIONS OF ARJ ("-" indicates a missing feature, "*" - a different operation and "+" stands for a feature unique to ARJ/2) - There may be a significant performance drawback (compared to ARJ/DOS and versions of ARJ/2 prior to 2.62.08) when accessing files on volumes with no write-back cache. * This branch has a fix for security weakness of mainstream ARJ. It is possible to make ARJ enter an eternal loop when processing the archive, by placing negative values in the "compressed size" field of header. Now, if ARJ/2 encounters such a header, the processing will be terminated with a message saying that the header is most likely intentionally modified for some malicious purpose. The problem reported here may be fixed in subsequent releases of ARJ and ARJ32. * External garble modules are provided as OS/2 DLLs. You may not use DOS modules with ARJ/2, or vice versa. The encryption algorithm is the same, so ARJ/2 garbled archives can be processed under DOS. * ARJ/2 sets the "Host OS" to "OS/2" on its archives. When extracting any such archive with ARJ for DOS, some unusual behavior may occur. Versions of ARJ prior to 2.50 convert high-ASCII characters in filenames to 7-bit characters by ignoring the high bit. So, the filename "ąTESTą" will be converted to "1TEST1" on extraction. ARJ/DOS also strips high bit from archive comments, but only if the archive was created by ARJ/2. Archives created with ARJ/DOS and then processed under OS/2 are not affected. Beginning with version 2.72, ARJ/DOS considers the ARJSFXJR archives created under OS/2 to be hostile to the DOS version of ARJSFXJR and therefore cannot modify them. In such cases, it's advised to use the "-2d" switch under OS/2. * ARJ/2 uses the default OS/2 error handler that will normally display pop-up menus prompting for action. The -& option installs batch error and exception handler (new feature since ARJ/2 v 2.62.08). Remember that the ampersand character is used in OS/2 to start two processes consequently, so this switch will actually split the command line. Type -^& instead of -& to install the error handler. * ARJSFX and ARJSFXJR archives will display ANSI sequences if the ANSI support is enabled in the current OS/2 session (it's enabled by default in OS/2). * By default, ARJ/2 stores extended attributes among with files. To disable EA storage and thus revert to the traditional header format, use the "-2e" option. See a special dedicated section below for details on EA handling. * The "-hm" options are compatible with their DOS implementation hence a temporary swap file is created if the file list exceeds 3000 files. To keep the entire list in memory, use -hm65000. + The "-ha" has been improved. Now, when used in conjuction with an archiving command, it does not mark read-only files as such in archive. This simplifies archiving from CD-ROM media where virtualized read-only attribute is forced by respective IFS driver. + The "-h#" option has been improved to allow custom date/time formats. A custom format is specified by putting a format sequence right after the "-h#". The following characters represent date/time macros: Y = year, M = month, D = day h = hour, m = minute, s = second N = day of year (note that these are case-sensitive) All other characters, as well as those going beyond format limits (4 digits for year, 2 digits for all other fields), are treated as delimiters. Examples: ARJ a project- -h#YYYYMMDD (project-19991022.arj) ARJ a backup- -h#MM-DD_hh-mm-ss (backup-10-22_23-57-16.arj) ARJ a specs -h#YY (specs99.arj) ARJ a logs_ -h#NNN (logs_295.arj) ARJ a test -h#YYYYYYYY (testYYYY1999.arj) + The new "-2d" option will force the stamping of "MS-DOS" as host operating system for the archives created by ARJ/2. It may be used for compatibility, to avoid "binary file from another OS" warnings. "-2d1" will force only the main header to be stamped as MS-DOS one, so archive comments will not be discarded by DOS version. + "-2k" option forces 2-digit display of year in lists. This can be helpful if the 3-digit year format used for dates beyond 2100 is confusing. Alternatively, "-2k1" uses a non-ambigous format that is both easy to read and information-packed. The dates are represented by two digits if the year is 1970 to 2069, and in three digits if it's 2070 or beyond. + "-2p" can be used to set the priority of ARJ/2. The knowledge of OS/2 scheduling system is required to use this option. It can dramatically increase performance but slow down other processes. The can be equal to 1, 2, 3 or 4: 1 = idle-time priority 2 = regular priority 3 = time-critical priority 4 = fixed-high priority Priority delta values are not supported. We don't see a strong requirement for them. + The "-2a" option is implemented in ARJ/2 due to a popular demand from FidoNet system operators. Basically it acts like "-jo", with the exception that file names, and not the extensions, are "serialized". Consider having a file called "FILE995N.TXT" and an archive that contains the same file. If extracted with the "-2a" option, the file will be written to "FILE9950.TXT", if you extract it again, it'll be called "FILE9951.TXT", and so on up to "FILE9959.TXT", then "FILE9960.TXT". And after "FILE9999.TXT" ARJ/2 will start with "FIL00000.TXT". This option allows you to extract one file to 100000000 unique names. It's essential to system operators since multiple mail packets with the same name may come from different systems. NOTES: 1. It'll be wise to include this option in the script that unpacks the ARCmail packets and NOT in ARJ_SW environment variable. This option is a security measure for systems running in unattended mode, and will only confuse you if enabled by default. 2. There's a security hole: a file called "9999.XXX" or so, will not be overwritten. However, all subsequent writes will be redirected into file "0000.XXX". So, files with 9s in the beginning have less chances of being preserved. Hopefully such situation is unlikely for FidoNet systems. 3. There is another option, "-jo1", to serialize filenames, however its operation is different. The volume must support long filenames in order to use this option, moreover, it's not suitable for dealing with FidoNet ARCmail. + ARJSFXJR does not support long filenames under Win95 DOS. ARJSFXJR in ARJ/2 supports OS/2 long filenames. + With no ARJ_SW specified, ARJ/2 looks for a file named ARJ.CFG in its home directory. If found, this file will be parsed and used as a standard ARJ configuration file (see manual for details). EXTENDED ATTRIBUTES HANDLING Beginning with version 2.62.10, ARJ/2 can back up and restore extended attributes (also referenced as EAs) without needing any external utilities. This is achieved by compressing and storing EAs as a part of file header. Restrictions on EA support: * The multivolume restart feature (-jn) will not work if EAs are enabled. You'll have to disable EAs with -2d prior to using -jn, or to recreate the archive if the EAs are precious. * Hollow mode archives do not support EAs. A set of new options has been introduced to let the user control EA handling: * "-2c" restricts EA handling to critical EAs only. Archived non-critical EAs will not be restored. When an archive is created, only critical EAs will be saved. * "-2e" specifies EA inclusion filter. With no parameters given, it disables EA handling at all. Otherwise, an expression that follows it is interpreted as a wildcard that limits EA inclusion to a particular EAs. Multiple options can be entered to represent a set of EA names but list files are not allowed. Examples: ARJ a test In this example, all EAs will be preserved. ARJ a no_eas -2e ARJ x no_eas -2e EAs will neither be packed nor restored. ARJ a documents -2e.LONGNAME In this case, only .LONGNAME EAs will be handled. ARJ a test -2e.CLASSINFO -2e.ICON* .CLASSINFO and .ICON* (i.e. .ICON, .ICON1, .ICONPOS) EAs will be packed and restored. It's wise to specify "-2e.*" when backing up your OS/2 desktop or configuration files. The system EAs start with dot (".") while application EAs start with application name. * "-2l" allows to convert .LONGNAME extended attributes (these represent icon titles used in WPS) to file names, when possible. This feature simplifies moving document files away from an OS/2 system installed on a FAT volume. If the icon title (and so the extended attribute) contains line breaks, wildcard characters or other symbols, real filename will be used instead and the .LONGNAME EA will be preserved. This option is ignored during extraction. "-2e" and "-2x" have no effect on this option (but .LONGNAME EAs are not saved if .LONGNAME EA handling is implicitly or explicitly disabled). * "-2x" specifies an exclusion filter. It must be followed by an exclusion EA name specification. The rules are the same as with "-2e". Also, the two options may work together, providing both an inclusion and an exclusion rule. For example: ARJ a backup_ -r -p1 -h#2 -2e.* -2x.FED* c:\projects may be used to create regular back-ups of your work directory, including all system EAs but excluding EAs created with FED (Fast Editor Lite, an editor written by Sergey I. Yevtushenko, evsi@naverex.kiev.ua) - that program does not follow traditional EA naming conventions and uses system-alike EAs for anchor position marks. Extended attributes are also supported in ARJSFXV self-extractors where they are stored using the same technology as with usual ARJ archives. The presence of EAs is indicated by a "(EA: ...)" message when a file is packed. Note that this size may differ from the one given when the file is unpacked - the former is the EA structure size and the latter is the space allocated for EA storage. The number of EAs and the size of EA structure is also displayed when the archive is listed with "ARJ v" command. KNOWN PROBLEMS * On FAT volumes under Windows NT, both long filenames and short filenames may be used to specify a filename. If you refer to a file with its short name (e.g. "ARJ a lntest LONGFI~1.TXT"), the short filename will be stored as entered from console. So, no LFN equivalent is placed into the archive, and if a file with a similar long filename exists in the archive, it will not be replaced. * REARJ/2 may behave incorrectly if the command interpreter (specified with COMSPEC= environment variable) is not compatible with CMD.EXE. It's also unwise to rename REARJ.EXE since it calls itself recursively when converting archives within archives (/a). FREQUENTLY ASKED QUESTIONS Q: Third-party applications fail to handle ARJ/2 archives! A: Try to disable extended attributes (-2e) and HPFS DTA/DTC storage (-j$). Many applications are incapable of handling new archive format (although this format is fully compliant with documented guidelines). Known examples of such applications include File Commander v 2.11, Norton Commander v 5.00 and WinRAR v 2.60. Q: Extended attribute sizes reported by InfoZIP and ARJ/2 are different. What's the cause? A: As we have stated earlier, ARJ/2 reports the size of its internal EA storage structure as EA size when packing files. This may mismatch with space allocated for EAs by the system, and it may also mismatch with values reported by other OS/2 native archivers. Q: How can I back up my Workplace Shell folders, preserving the icons? A: Since folders are represented with directories, you'll have to enable directory storage with -a1 or -hbfd. Q: Why ARJ/2 is a 16-bit executable? A: There are several reasons for it. First, the Huffman encoding and decoding routines are optimized by introducing assembler code. That's why ARJ 2.x is many times faster than ARJ 1.x, and ARJ32 is quite slow compared to ARJ/2. Second, OS/2 16-bit binaries do NOT run in VDMs and are actually as fast and reliable as 32-bit ones. So-called 16->32 thunks are called when OS/2 API is accessed but it rarely happens: ARJ/2 does Huffman encoding/decoding most of the time, not involving any kind of external API. An "ARJ32 for OS/2" version was planned but we decided to stay with 16-bit code, adding ARJ32-specific features. Q: What is ARJLX? A: It's the same program as ARJ/2, compressed with LXLITE utility. It requires OS/2 v 3.0 or higher to run. You may encounter some compatibility problems with it (it's unable to register ARJLX with REGISTER.EXE, and so on...) Q: The EAs have vanished after I used ARJ/DOS to update an archive. A: Current versions of ARJ/DOS and ARJ32, as well as ARJ/2 prior to 2.62.10, strip the extended headers when any kind of archive update occurs. Q: I want to create single-volume self-extracting archive that supports EAs but ARJ/2 uses ARJSFX instead of ARJSFXV. A: You need to force use of ARJSFXV/2. The best way for it is to specify an arbitrary large value for volume size, e.g. -va. Q: How can I create an installer for my OS/2 product with ARJ? A: ARJSFX/2 is able to run OS/2 commands after unpacking archive. Try this: create a script you want to to be invoked after the installation completes. e.g., INSTALL2.CMD: ======= /* REXX */ if RxFuncQuery('SysLoadFuncs') then do call RxFuncAdd 'SysLoadFuncs', 'RexxUtil', 'SysLoadFuncs' call SysLoadFuncs end say "Installation has completed, creating desktop object..." call SysCreateObject .... ======== and so on... Now create an archive comment with the first line as follows: )) \InstallDir\ -b -x -y -!INSTALL2.CMD Call it, for example, CMT.ANS. Now create an ARJSFX archive: ARJ a PACKAGE.EXE -je -r -a -jm -zCMT.ANS -xCMT.ANS You'll make a self-extracting archive with an automatically invoked installation program. You may not commercially distribute these packages. Q: How can I distinguish ARJ/2 archives from ARJ or ARJZ ones? A: Use the ARJ V command. The "Host OS" field contains the type of archiver binding (DOS or OS/2). The "Revision" field may be used to determine the archiver version that added the file: 1 = ARJ versions earlier than 0.14, ARJZ compatibility mode (-md is less than 26624) 2 = ARJ v 0.14...0.20 3 = ARJ v 1.00...2.22 4 = ARJ v 2.30, X1 5 = ARJ v 2.39a, 2.39b 6 = ARJ v 2.39c...2.41 7 = ARJ v 2.42a...2.50a 8 = ARJ v 2.55...2.61, ARJ/2 v 2.61 9 = ARJ v 2.62, ARJ/2 v 2.62, ARJ32 v 3.00 10 = ARJ v 2.70 and higher, ARJ/2 v 2.62.10 and higher 50 = ARJZ with maximum distance up to 32K 51 = ARJZ with maximum distance up to 64K 100 = ARJ32 v 3.00b...3.01 101 = ARJ32 v 3.02 and higher Note: ARJ versions that created the Revision 1 header used a different method 4 compression. Support for it was dropped in versions 1.xx. Such archives may be incorrectly processed by ARJ/2. INFORMATION FOR DEVELOPERS ARJ/2 header format is 100% compatible with ARJ header format but ARJ/2 was the first to make use of extended header fields. Here's a brief summary of the EA storage technology. First, the extended header layout is as follows (all values are little-endian): Bytes Description ----- ----------- 1 Extended header ID (0x45, the character 'E'). Indicates a EA information header. This is only valid if the arj_nbr (header revision number) is 10 or greater. 1 Continuation flag. If set to 0, marks the end of block chain so the EA data can be concatenated and unpacked. Also it provides a way of checking for trashed EA blocks. ? Packed EA block. A standard CRC32 of the whole header, including the ID but excluding header size field, is appended to it. It is strongly recommended that the CRC is verified when any compressed EA processing occurs. The packed EA block is a complex structure that can span over multiple volumes. In case of such spanning, separate parts of the block are stored in separate extended headers on different volumes and they must be joined together when the last block is read (it's recognized by EXTFILE_FLAG being clear). The layout of the packed block follows: Bytes Description ----- ----------- 1 Compression method (0...4, may differ from the one found in the file header) 2 Unpacked EA data size in bytes 4 CRC32 of unpacked EA data ? Raw packed EA data The raw packed data may be decompressed using the original ARJ algorithms. In the case when the file is encrypted, the packed block is also encrypted (but the garble routine is reinitialized when compression of the EA occurs). The password modifier is the same as for the first file section. After decompression, the following EA structure will exist: Bytes Description ----- ----------- 2 Total number of EAs 1 fEA byte (may indicate a critical EA) 1 Size of extended attribute name 2 Size of extended attribute value ? Extended attribute name (not ASCIIZ) ? Extended attribute value (binary data) Even if the file is a text one, the EAs must be handled as binary data during compression and extraction. When EAs are spanned over multiple volumes, neither the packed block header is repeated nor the compression is restarted (actually, the whole block including its header is created in memory and later split to volumes). PRODUCT INFORMATION ARJ/2 has been developed by ARJ Software Russia with the most of original ARJ code left intact. It has been briefly tested on IBM OS/2 v 1.3, 2.0, 2.1, 3.0, 4.0 and 4.5, and Microsoft Windows NT Workstation v 3.51 and 4.0. The file systems chosen for testing include FAT, HPFS, Ext2FS and NTFS. The ARJ/2 development team: Programmed by Andrew Belov Testing lead by Viatcheslav Odintsov Source code is not available for public use. REVISION HISTORY (only general availability versions are listed) 2.61.10 21/11/1998 Initial release. 2.61.11 26/11/1998 ARJ/2 2.61.10 would cause exception 0Dh if ARJ$DISP is killed from the task list. The "clear screen" ANSI sequence now works. 2.61.13 13/12/1998 "ARJ M" fixup, decreased the size of EXEs, ARJ/2 now also works under Windows NT. 2.61.14 31/12/1998 Minor fixes. It is the last build of ARJ/2 v 2.61. 2.62.02 09/01/1999 G.A. version. Minor fixes in the message section. ARJSFX now properly supports the "-!" option. 2.62.03 12/01/1999 ARJ/2 v 2.62 crashed under Windows NT when creating directories. ARJSFX/ARJSFXV might come into loop on some LFN archives. ARJ/2 might crash if LFN goes beyond 131 chars. 2.62.04 24/02/1999 Various problems with ANSI comments were fixed. Some documentation enhancements. 2.62.05 01/05/1999 ARJSFXV v 2.62c has been introduced. 2.62.07 23/08/1999 ARJ i reported damage in earlier versions of ARJ/2. Modified file seek logic for newer IFS. The ARJ/2 distribution archive is now ARJ-SECURED. 2.62.09 04/09/1999 Fixed header comment corruption problem. Fixed problem with free space detection on FAT volumes larger than 1G. Enabled custom environment variable setting (-+). Added support for spaces in filenames. Changed default extension to lowercase ".arj". Resurrected the -& error handling option. Volume labels are now saved and restored. The "-h2" group of switches has been renamed to "-2" (although old names may still be used). Removed Windows NT host OS stamping to avoid confusion with native ARJ32 archives. Packaged a full set of accompanying utilities. 2.62.10 15/09/1999 Timestamps are now restored upon directory creation. Added support for extended attributes to ARJ and ARJSFXV. Fixed a problem with directories being mishandled by ARJSFXV. 2.62.12 26/09/1999 Added check for encryption when creating SFX archives. Fixed chapter mark creation with EAs enabled. Fixed to allow commenting of any ARJ chapters. Added check for files larger than 2 gigabytes. Fixed volume handling of "hollow" archives ("-j#"). Fixed handling of encrypted and dual-name archives with the "j" and "y" commands. Disabled volume prompting when a volume is not an ARJ archive. Changed multi-volume function to automatically handle volumes over 99. Implemented inline ARJ-PROTECT storage as in ARJ32 v 3.02. 2.70.01 08/10/1999 Changed error message display. Fixed to work in xterm of XFree86 for OS/2. Fixed possible loop when -hk is used with multivolume option. Fixed a problem with temporary archive not being deleted when no files are added. 2.70.02 25/10/1999 Fixed date/time stamping in ARJSFXJR. Fixed recursive repacking in REARJ. Added custom -h# format feature. Added comment compatibility option. 2.70.04 06/11/1999 Added day number option to -h#. Fixed extracted file postprocessing. Added automatic testing of ungarbled archives. Fixed chapter command parsing from the configuration file. Implemented default configuration file lookup. Fixed garble password prompting. Corrected to work with EXT2FS.IFS. 2.70.05 31/12/1999 Fixed rare "Can't find ..." errors when parsing list file. The SFX executables are now compressed with standard EXEPACK. Introduced new ARJ registration scheme. Added ARJ.KEY registration file lookup. Fixed "clear to end of line" ANSI sequence handling. Fixed "Out of memory" errors appearing after a long sequence of archive volumes. Fixed abnormal effects when extracting damaged archives. Fixed timestamp handling during extraction. Fixed recursive verification with /G in REARJ. REARJ/2 has been updated to version 2.42. 2.70.07 06/01/2000 Minor optimizations of the compression procedure. Added .LONGNAME EA to filename conversion on archiving. The "/J" option was not properly working in registered REARJ/2. 2.70.08 07/01/2000 Fixed a problem with valid registration keys being rejected by ARJ/2. 2.71.01 19/03/2000 Changed "-ha" option to skip read-only attributes on packing. Fixed century display for dates beyond 2000. Added "-2k" switch to skip century display in list commands. Fixed parsing of "-v...K" and "-v...M" size modifiers. Fixed license information display in ARJSFXJR. Changed REGISTER.EXE to use "-arj" and "-arj32" options instead of "-all". Changed the meaning of the "-s" option of ARJSFXV to enable file sharing. Added an alternate multivolume archive naming feature. 2.71.02 25/03/2000 Fixed file handle deficiency (occured in 2.70.09 and 2.71.01). 2.71.03 29/03/2000 Fixed abnormal behavior of -b and -b2 switches. 2.71.05 27/06/2000 Minor fixes and modifications. 2.72.06 18/08/2000 Commercial versions now never expire. Fixed a minor cosmetic display issue when using the "-t1" option. Fixed abnormal handling of "-*". Fixed damaged archives resulting from -m4. Added option checks when modifying an existing self-extracting archive. Modified ARJ version messages for compatibility with Winzip. Fixed multiple volume diskette updating to work properly when a work directory is specified. Fixed rare issue of ARJ building an empty archive as the last of a series of volumes. Fixed the ARJSFX self-extractor to detect and display an ARJ-SECURITY envelope. Fixed the "jc" extract count option to work with volume split files. Fixed a problem using "-js" in ARJ_SW/ARJ32_SW and on the command line. Changed the "l"ist command to show " 00" for year 2000 and "100" for year 2100. Added a "-2k1" switch for non-ambigous date formatting. Added "-hy1" and "-hy2" options to force OEM/ANSI codepage settings. Fixed garbling to accept "-hg" when garbling an existing ungarbled archive. Modified help display not to display milestone indicators. ARJCRYPT has been updated to version 2.1. 2.73.01 19/08/2000 Fixed looping on self-pointing headers. Minor changes in the message section. Added "-" symbol option to configuration file to work like "+" symbol except with prepending of its associated switch string. Fixed volume size handling during update of ARJ-PROTECTED volumes. Corrected total file count output on list command. Fixed the purging "k" of chapter volumes. Fixed volume series breakups. Fixed a problem with the "-vw" option when updating a volume. Fixed alignment on verbose list command. Corrected "-i3" and "-i6" total percentage counts for multiple volumes. Fixed the "-jc" extract count option to terminate prompting for unneeded volumes. Fixed looping when using "-va" on large partitions. End of document