JMODEM was first introduced about two years ago. It has en~joyed a steady increase in popularity around the world. It has even been explained in some detail in John Dvorak's book on PC communications; Dvorak's Guide to PC Telecommunica~tions, 1990, Osborne-McGraw-Hill, 2600 Tenth Street, Berkeley, CA.
JMODEM was first written in assembly language. Since this language is hard to read and use, it has been difficult for communications program developers to incorporate it into their programs directly. Instead it must be executed as an external protocol. Now JMODEM has been written in the C Language. C has become the de-facto standard for portable code development even though there isn't presently any code that is truly portable across many different machines. The writing of JMODEM in C will make it easier for software de~velopers throughout the world to use this very useful pro~tocol.
You can use this new version of JMODEM just as the older versions written in assembly. It has a new color sign-on screen and status-block windows that overlap. Although the new JMODEM.EXE code is larger than the assembly-language version, JMODEM.COM, the program still executes very fast because much effort has been taken to streamline the C code. JMODEM.COM required 64 k of RAM (one segment) to execute properly. JMODEM.EXE requires 79 k to allow the screens to be written properly, and 66 k of free RAM to execute without aborting although the previous screen content will be lost. Like all versions of JMODEM, this version is 100% compatible with all previous version including Beta version 1.00. From it's inception, the essential structure of JMODEM has never been changed.
How to install JMODEM.
JMODEM executes best from a batch file as an external pro~tocol for any of the communications programs that have ex~ternal-protocol capability. A typical communications program is TELIX.
«NB»
Here is a batch file used with TELIX for uploads:
@ECHO OFF
Rem * JMODEM TELIX Upload batch file.
C:\TELIX\JMODEM S1 %3
Rem | || |_________ file name (passed by TELIX)
Rem | ||____________ COM port (1 - 4)
Rem | |_____________ Send
Rem |_______________ Path and name of JMODEM
«BB»
«NB»
This is a batch file used with TELIX for downloads:
@ECHO OFF
Rem * JMODEM TELIX Download batch file.
C:\TELIX\JMODEM R1 %3
Rem | || |_________ file name (passed by TELIX)
Rem | ||____________ COM port (1 - 4)
Rem | |_____________ Receive
Rem |_______________ Path and name of JMODEM
«BB»
«NB»
This is a batch file used for PCPLUS (PROCOMM) uploads:
@ECHO OFF
Rem * JMODEM PCPLUS Upload batch file.
C:\TELIX\JMODEM S1 %1
Rem | || |_________ file name (passed by PCPLUS)
Rem | ||____________ COM port (1 - 4)
Rem | |_____________ Send
Rem |_______________ Path and name of JMODEM
«BB»
«NB»
This is a batch file for PCPLUS (PROCOMM) ownloads:
@ECHO OFF
Rem * JMODEM PCPLUS Download batch file.
C:\TELIX\JMODEM R1 %1
Rem | || |_________ file name (passed by PCPLUS)
Rem | ||____________ COM port (1 - 4)
Rem | |_____________ Receive
Rem |_______________ Path and name of JMODEM
«BB»
You can chop this text out with an editor and create your batch files quickly and correctly. PCPLUS passes the file-name as the %1 parameter and TELIX uses the %3 parameter. JMODEM does not need to know anything else about your system to operate. It does not even need to know the baud-rate or other communications parameters. All it needs to know about is the file name and the communications adapter port to use.
JMODEM doesn't care if you are using a 19,200 baud modem with the DTE locked at high-speed or a 300 baud modem on a lamp-cord. It is very smart about communications and handles lost carriers (hangup) and flow-control in a very simple way which will be explained in detail later.
If you only want to try JMODEM without having to install it, you need only to activate the JMODEM protocol on a BBS sys~tem, then "shell to DOS" and execute it manually from the DOS prompt. You will be pleasantly surprised at how rapidly it can transfer a file and how well it executes over net~works.
How JMODEM works.
JMODEM uses variable-length records called blocks. These blocks start with 512 data-bytes and increase in length to a maximum of 8192 bytes per block. There is a 6-byte overhead associated with each block so the percentage of overhead starts at a fairly high 0.1 percent and decreases to a very low 0.07 percent as the transmission progresses. The block length will increase in 512-byte increments as long as there are no errors requiring retransmission. Should an error occur, the block-size is cut in half. This continues until the block-size is as short as 64 bytes.
The blocks may actually be of any length but never exceed the maximum allowed. An attempt is made to reduce the amount of data that needs to be transmitted by compressing each block before transmission. Since much data is already com~pressed, being from ".ZIP" files and such, it is possible that "compression" may actually cause an increase in block-length. JMODEM will send a compressed block only if it is shorter than the non-compressed block.
Since it takes time to compress and expand data, only the simplest and quickest compression method is used. It is very effective with ".EXE" files that contain large blocks of binary nulls and most text files but it is not very effec~tive with highly compressed archive files. The compression method is called "run-length-limited" or more explicitly "how many of what kind". Basically, the block is searched for groups if similar characters. If many similar characters are found, compression consists of sending a sentinel byte of hexadecimal BB, followed by a two-byte amount, and this followed by the byte to be repeated.
In the following example we have a string of spaces (20 hex).
The blocks could actually get longer because the sentinel byte could be present in the data:
«NB»
Before compression:
( 9 bytes )
BB 00 BB AF EF BB 00 AE EF
«BB»
«NB»
After compression:
( 16 bytes )
BB 01 00 BB 00 BB 01 00 BB AF BB 01 00 BB AE EF
«BB»
As soon as the encoded length exceeds the data block length, compression is abandoned and the non-compressed block is sent. JMODEM sends a byte that tells the receiver if the data is compressed or not. This same byte tells the receiver when the end-of-file has occurred so JMODEM is able to preserve exact file-length.
The block length is a WORD (16 bits). This allows the blocks to be 65,535 bytes long although in practice the length is not allowed to exceed 8192 bytes (plus overhead). The block-length is the length of the entire JMODEM block, not just the data-bytes.
The Block number:
The block number is a BYTE. It starts at 1 and becomes zero again after block 255. It is used to make certain that all receiption errors are detected.
The Block type:
«NB»
This BYTE is bit-mapped to tell the receiver what kind of block has been sent. Presently there are three kinds of blocks:
00000000B Hex 00 ( Normal data )
00000001B Hex 01 ( Compressed data )
00000010B Hex 02 ( End of file )
«BB»
The CRC:
The CRC is a WORD (16 bits). It is not a "standard" type of CRC because, unless done with hardware, a standard 16-bit polynomial would take several seconds to calculate for a long block. Instead it uses a rapid rotate and sum algorithm that is probably just as effective as the more "standard" polynomials.
«NB»
The CRC is generated using this polynomial:
X = X + X^(2(n-mod 7))....... Where n = t(n-1)
And t = string length
«BB»
It has the advantage of simplicity in assembly-language programming and will detect errors with a probability of about one undetected error in 2^132 (which is a very large number). It does not correct errors so its not important to use some "standard" function to generate the CRC.
The C code shows how easy it is to create and check this kind of CRC.
«PG»
«FC»The JMODEM Communications
Hardware Control
«FL»
When JMODEM is first loaded for execution it checks the state of the modem-control leads. If there is no modem carrier detected, it assumes that you have connected two PCs together without a modem and will not bother to check for a dropped carrier during execution.
Since JMODEM must exercise flow-control so it can be used with high-speed modems with fixed baud rates, it also as~sumes that at the time at which it is first executed, the modem will be requesting data because there will have been very little I/O over the previous few seconds. JMODEM stores the state of RTS/CTS and DTR/DSR and uses this as a refer~ence. When transmitting data, should JMODEM find that the state of these modem-control leads has changed, it waits until the modem-control leads have reverted back to the initial state before sending any more data. This allows ANY modem to exercise flow-control with JMODEM, even those that use "pin 4" instead of "pin 20". When JMODEM is waiting, it checks to verify that the modem carrier has not been drop~ped. If the carrier is dropped, or if the user aborts, JMODEM will exit, setting a DOS ERRORLEVEL code.
You can abort JMODEM at any time by hitting Ctrl-Break or Ctrl-C. It will take several seconds for JMODEM to abort because it does not continually check these keys. Unlike previous versions, JMODEM erases the files from aborted downloads. Also, JMODEM will abort if it is unable to rename a file in the following example.
Suppose you wish to download a file called VIRUS.EXE. Sup~pose also that VIRUS.EXE already exists. Instead of over~writing your previous version of VIRUS.EXE, JMODEM has al~ways been nice and renamed it to VIRUS.OLD before creating a new file. With previous versions of JMODEM, if VIRUS.OLD already existed, JMODEM would have deleted it before renam~ing the present file to ".OLD". This no longer is done. JMODEM never deletes ANYTHING anymore except it's own abort~ed download. This should reduce the number of threats I have received!
«PG»
«FC»The C programmer's
Guide to JMODEM
«FL»
Included within this package should be the following files:
JMODEM This is a Microsoft MAKE file for JMODEM.EXE.
JMODEMTC Borland's Turbo C MAKE file for JMODEM.EXE.
JMODEM EXE The executable file
JMODEMTC RSP Link response file for Borland's Turbo C
JMODEM_A C Contains the _main() routine entry point.
JMODEM_B C Allocates memory, parses input strings.
JMODEM_C C All of the file I/O is in this file.
JMODEM_D C Data compression/expansion, CRC calculation
JMODEM_E C Communications I/O interrupt service, etc.
JMODEM_F C Screen I/O (windows, etc)
JMODEM H Contains JMODEM globals and data structures.
SCREEN H Screen definitions, prototypes, structures
UART H Definitions for the 8250 UART
TEST C This tests compression /expansion, disk I/O.
TEST This is the Microsoft MAKE file for TEST.EXE
TESTTC Borland's Turbo C MAKE file for TEST.EXE
SHOW C Source for verifying execution in a DOS shell
SHOW Microsoft MAKE file for SHOW.EXE
SHOWTC Borland's Turbo C MAKE file for SHOW.EXE
SHOW EXE Executes JMODEM from a DOS shell.
TEST C Tests JMODEM data-compression and file I/O.
TEST Microsoft MAKE file for TEST.EXE
TESTTC Borland's Turbo C MAKE file for TEST.EXE
TEST EXE Executable to test file I/O and compression
If you have MicroSoft 'C' version 5.0 or later, and if your environment and paths are properly set up, you should be able to type:
MAKE JMODEM
... and a brand new version of JMODEM will be created.
If you use Borland's Turbo C, you can use the "TC" make files to do the same.
Brad Smith from Jacksonville North Carolina created the MAKE files for Turbo C and modified the source-code to accom~mo~date the different libraries used in Turbo C. His revisions will be present in all subsequent versions of JMODEM to assure compatibility with Borland's Turbo C.
Brad is a Turbo C guru and runs a BBS system in Jacksonville.
Brad Smith
141 Riggs Street
Jacksonville, North Carolina 28540
BBS (919) 455-5972 12/24/9600+ 24 hrs
With this release, there seems to be a Null-Pointer assignment error when using Borland's Small Model in TCC Version 1.5. This is a compiler bug, or more specifically a bug in the C0.ASM code provided with the compiler that you can fix.
If you are a Turbo C programmer, don't use the small model when creating JMODEM.EXE unless you repair the C0.ASM file and create a new C0S.OBJ file using the MAKE-C0.BAT file that is provided with the compiler.
To fix the null-pointer problem in Turbo-C find the following code-fragment in \LIB\C0.ASM provided with your compiler:
IFNDEF __HUGE__
; Reset un-initialized datas
xor ax, ax
mov es, cs:DGROUP@@
mov di, offset DGROUP: bdata@
mov cx, offset DGROUP: edata@
sub cx, di
rep stosb
ENDIF
Add the following code:
push ds
mov ax,DGROUP
mov ds,ax
mov word ptr ds:[0],0
pop ds
Now find this next code-fragment:
xor ax, ax
mov si, ax
mov cx, lgth_CopyRight
cld
ComputeChecksum label near
add al, [si]
adc ah, 0
inc si
loop ComputeChecksum
sub ax, CheckSum
jz ExitToDOS
mov cx, lgth_NullCheck
mov dx, offset DGROUP: NullCheck
call ErrorDisplay
Modify the code so it looks like this:
; xor ax, ax
; mov si, ax
; mov cx, lgth_CopyRight
; cld
;ComputeChecksum label near
; add al, [si]
; adc ah, 0
; inc si
; loop ComputeChecksum
; sub ax, CheckSum
push ds
mov ax,DGROUP
mov ds,ax
cmp word ptr ds:[0],0
pop ds
jz ExitToDOS
mov cx, lgth_NullCheck
mov dx, offset DGROUP: NullCheck
call ErrorDisplay
After to have done this, execute MAKE-C0.BAT for the small model library from the DOS prompt:
F:\TURBOC\LIB> MAKE-C0 SMALL
This will create a new small-model startup object file.
Just as in the MASM language files, the source code is strong~ly-typed. All function prototypes are declared and no defaults are used. You can compile at warning-level 3, the most stringent level available with Microsoft compilers, and you will get no warning errors at all. The default warning level with Borland's compiler is even more stringent. You will get a single warning error when compiling TEST.C as the compiler discovers that dummy parameter "one" in the dummy screen routine isn't used at all. It has been my experience that if you get any warnings (in real code), they MUST be fixed or they will bite you later on.
Example:
*memory++ = foo;
... will bump the memory pointer after you put foo
into memory. Suppose you get to the end of the block
and wish to index backwards.
Do you use ...
--*memory = foo;
...?
You do NOT!! You must use:
*(--memory) = foo;
Little things like this (this is a BIG, BIG bug), could cause code to work <sometimes> and keep you awake nights. JMODEM has been carefully written to prevent bugs like this. It is impossible to be sure that there are no bugs in even simple code so be careful if you modify the source.
You may modify JMODEM for your own use, but P L E A S E do NOT distribute the modified version on BBS systems because I do not wish to support 222,500 versions of JMODEM! If you find a bug, or wish to improve or add something that will increase the value of JMODEM without making it incompatible with previous versions, please upload your improvements to my BBS system and I may include the revisions (with your name attached) in an upcoming version. Note that there are 6 "type" bits available in the JMODEM control-byte that could be used to tell the receiver that another file is coming, etc., (for batch). Things like this could be added without destroying compatibility with previous versions.
MicroSoft C seems to have several bugs one of which affects files that are very long (over 256k). My first attempts to use C files under MicroSoft for JMODEM used the stream-I/O (FILE *) type of files. This resulted in corruption of long files. Therefore I implemented the UNIX/DOS type of file-I/O that uses handles rather than file-control blocks. These work rather well.
Starting with Version V3.06, The JMODEM distribution file will no longer contain the old JMODEM.ASM source. This is to reduce the download time. The C version is now mature enough so that there is little or no difference between the per~formance of JMODEM.COM, the assembly version, and JMODEM.EXE, the C version. The original C version was about 25 k in length. It is now only about 12.
Floating point:
Originally no floating-point libraries were used. The de~fault is still not to use any because they result in doubl~ing the size of the code. Some persons have complained that the speed (cps) indication is not very accurate. This is true because of the granularity of the time returned from DOS (one second). If you add /DFTIME to the command-line used to compile JMODEM_A.C in the "MAKE" file, the compiler will use floating-point routines for the speed calculation. The resulting JMODEM.EXE will be about 35k in length, though, so this is a trade-off. If you wish accurate timing, you pay for it in much-increased code-size.
Modems:
When JMODEM gets control, it turns ON CTS and DSR just in case the BBS program has turned them off (WILDCAT does). Some modems "hiccup" when this occurs, and the RTS line will bounce. This caused JMODEM to "wait forever" for a RTS that never occurs. Therefore I added a 1/2 second delay between the time that I first set these bits and the time I start checking for RTS and RLSD for flow-control and a possible abort. This fixed this problem. The flow-control problem is complex because user's might wish to transfer files between two computers using only three wires. In such a case, there is no flow-control and there is no modem carrier. To accom~modate a universal solution to various hookups, JMODEM checks for the state of DSR/CTS and RLSD when it first gets control. If there is no carrier at this time, JMODEM will not check for an aborted carrier during transmission or reception. If JMODEM detects a CHANGE in either DSR or CTS, it assumes this is flow-control and waits for these bits to change back to whatever they were when JMODEM first obtained control. This protocol has been used successfully since the very first version of JMODEM. There is, however, a pos~sibility of JMODEM wrongly interpreting what it finds if there is "bounce" when it first gets control. That's the reason for the delay. In any event, the normal "timeout" trap will prevent a hung system.
BBS systems:
JMODEM still gets blamed for crashing some BBS systems even though it is not JMODEM's fault. A common problem occurs with systems that use compiled BASIC. Some BBS system pro~grams assume certain characteristics about external proto~cols and will blow up when the assumption is wrong. If you use BASIC code that works like this:
PROTOSEND$ = "JMODEM S1 "
COMSTRING$ = PROTOSEND$+FILENAME$
SHELL COMSTRING$
... Then JMODEM (and other protocols) will work okay.
However, If you attempt to BLOAD, CALL, or CHAIN to it, the results will be indeterminate. This is because JMODEM allocates memory from C runtime routines that get their memory from DOS (not BASIC). BASIC will not "know" that its memory has been used by the external protocol. If you exe~cute the "SHELL" command, then you force BASIC to give up all the memory that it can afford to spare. This free memory is recorded by DOS and can be used for the external proto~col.
BBS systems written is C should execute external protocols something like this:
strcpy(command_line,"JMODEM S1 FILENAME");
system (command_line);
The C "system" command executes DOS commands from within C programs by freeing memory and then executing COMMAND.COM. Some C compilers don't have to load an aditional copy of COMMAND.COM. They use an undocumented "back-door" to the existing command interpreter! This saves memory.
This is a sneak preview of the undocumented procedure:
MOV WORD PTR [SP_SAV],SP ; Put the SP in a safe place
MOV WORD PTR [SS_SAV],SS ; Save segment registers.
MOV WORD PTR [DS_SAV],DS
MOV WORD PTR [ES_SAV],ES
MOV AX,4A00H ; Free memory
MOV BX,OFFSET TOP ; Point to the top of code
MOV CL,4 ; Bits to shift
SHR BX,CL ; Div / 16
INC BX ; Round up
INT 21H ; Free some memory
;
MOV SI,OFFSET COMMAND ; Point to the command
INT 2EH ; Make the undocumented call
CLI ; No interrupts, cleaning up
MOV DS,WORD PTR CS:[DS_SAV] ; Restore all segments
MOV ES,WORD PTR [DS_SAV]
MOV SS,WORD PTR [SS_SAV]
MOV SP,WORD PTR [SP_SAV] ; Restore stack pointer
STI ; Allow interrupts
If JMODEM and other external protocols are executed from BBS software in this manner, then you will have no problems. JMODEM preserves the BBS system communications environment so BBS systems don't have to reinitialize anything when they get control. JMODEM does not even touch the baud rate! Its not necessary to alter any communications parameters so it doesn't.
A small program, SHOW.C is provided with this distribution. It is a C program that allocates a lot of memory, writes to the memory, then executes JMODEM.EXE using the "system" command. After JMODEM returns from the shell, SHOW then continues to write to memory before exiting. Since this procedure is properly written, no system crash will occur. You can load multiple copies of COMMAND.COM to "eat" memory. Eventually there will be too little memory available for JMODEM to execute. The system will still not crash.
Help:
JMODEM is now over three years old. I first wrote it in assembly and slowly it caught on. I spent last Christmas writing it in C. Since the C release, I have received over 500 comments from persons telling me how to make it better. Most of these comments were from so-called professional programmers who complained about various things. A synopsis follows:
"It should have been written UNIX-style..."
"It has too many comments and they don't make any sense..."
"The usage prompt should read: jmodem <r|s><port> filespec."
"It breaks my modem..."
"It is full of bugs which I have had to fix ..." (BBS sysop)
"It destroys my operating system..."
"It does not work..." (BBS sysop)
"I have never before seen such trash..." (College professor)
"It is very poorly written..." (Professional programmer)
"You are not handling interrupts correctly..."
"The MAKE file is amateurish..." (Professional programmer)
"You should not use pointers..." (Professional programmer)
"You don't use enough pointers..." (Professional programmer)
"You don't use pointers correctly..." (College instructor)
"You can't cast pointers to anything you like..."
"Microsoft C is not a professional C ..."
"You must never chain to the timer interrupt..."
"You will lose interrupts when you send 0x20 to the controller..."
"You must check the UART status in the interrupt service routine..." (Programmer at DIGITAL)
etc., etc...
There were no such comments when only the assembly-language version existed. You see, few know how to read assembly-language. It seems that E V E R Y B O D Y is expert in C.
I would like to answer each comment individually but don't have the time. I'm busy writing an advanced operating system at work. You see, I am a professional too.
If this seems too editorial, you haven't read anything yet! There are just too many persons who think they know every~thing there is to know about anything that comes along when, in fact, they know very little about their claimed field of expertise. For instance, one programmer "fixed" the UART transmit routine by enabling FIFO buffering on special UARTs. If we can assume that this "fix" won't hurt generic 8250 devices, what does it accomplish? The transmitter operates polled because it can't do anything else until it receives an ACK or NAK from the receiving party. Just think, all the MIPS of your processor is sitting there waiting for the UART to become ready to receive another byte! There is nothing, absolutely nothing that you can do to make the UART transmit faster than the baud-rate. Enabling FIFO buffering on special UARTs may help maintain the baud-rate speed on interrupt driven systems with a high interrupt-latency time (multitasking systems). It can't do a thing for us with our clones and terminal programs.
In the case of checking the UART status in the interrupt service routine, why?? The UART is enabled to interrupt only when a received byte is available. The program would not be executing the interrupt-service routine if a byte was not available so why check again? Just get the byte, put it in the buffer and get out as quickly as you can. That is what an interrupt service routine is supposed to do.
In spite of this, I welcome useful comments and enhancements to JMODEM. Just make the revision and upload it to my BBS. If I use the revision, you will be credited for your help. Please remember that an enhancement is not a "preference". If you prefer to do something one way or another then feel free to modify the code for your own use. If you find a better way to do something then please let me know.
«PG»
«FC»JMODEM revision history
«FL»
The revision history has been moved to JMODEM_A.C and to JMODEM.H (identical copies). This makes it easier to keep the documentation current.
