Simtel MSDOS 1992 June

home *** CD-ROM | disk | FTP | other *** search

/ Simtel MSDOS 1992 June / SIMTEL_0692.cdr / msdos / starter / transgid.txt < prev next >

Wrap

Text File | 1990-04-24 | 26.5 KB | 600 lines

[File; TRANSGID.TXT Revision date; April 23, 1990] A SHORT GUIDE TO NETWORKING AND FILE TRANSMISSION Erich Neuwirth Institute of Statistics and Computer Science University of Vienna Austria (A4422DAB@AWIUNI11.BITNET) GENERAL PRINCIPLES OF SENDING FILES IN ELECTRONIC NETWORKS Networking is mainly used in 2 ways: Electronic mail Sending (binary) files This paper tries to explain what some of the differences are and how one of the two transmission methods sometimes can be (mis)used for tasks which seem to belong to the other method. Electronic Mail Electronic mail means you are sending text from one computer site to another site. Letters of text are coded as numbers internally within computers. Problems arise from the fact that the same letter may be represented by different numbers on different computer systems and vice versa the same number may yield a different letter on different computer systems. Mostly we are concerned with two such representation systems for letters by numbers. ASCII (which is used on IBM-compatible PCs and on most non-IBM mainframe computers) EBCDIC (which is used on IBM (and compatible) mainframe computers) When you are sending text from one computer to another computer the computers "think" they only are sending numbers. People reading or writing text, on the other hand, expect characters, so some interpretation of the numbers producing the text must take place. Simply transferring the text file as a sequence of numbers (which is what it looks like to the computers involved) would result in an unreadable file on the receiving computer system. Therefore when using computers with different character representation systems the transmission usually involves a "translation process" which has the net effect of yielding a different "sequence of numbers" (= file) on the receiving machine, but this file usually gives the same letters when read as a text file. Usually these translation processes work quite well for letters (lowercase and uppercase) and digits. Quite often you will encounter problems with special characters like parentheses, brackets, tildes, carets and so on. If you are interested in merely transferring texts this is not much of a problem, because even if some special characters get scrambled it is usually not too hard to reconstruct the original text by normal editing. If you are setting up a new communications link it is a good idea to send a file containing all printable characters with descriptions and to test if they arrive at the other end as they should. At the end of this paper you will find an example of how such a test file could look. Of course such a file should be sent from both ends of the line because the scrambling process in many cases is asymmetrical, so different transpositions happen in the two different communication directions. Closely inspecting the file you receive will show you which characters are changed during the transmission process. Now three different events can happen: 1) You receive all the characters as they should be: Action: Don't worry, be happy 2) Some characters are not what they should be, but different characters still are different (even when not identical with their original) Action: Do worry, but not too much. In this case you can use the FIND and REPLACE function of your text editing program to restore the original meaning of the file. You even could program a macro in your text editor (if you don't know what that means just ignore this sentence) which automatically performs the "retranslation" process. 3) Some characters are scrambled and different characters in the source text file come out as identical characters at the receiving end. Action: Do worry, because this is the worst possible situation. It is not possible to construct an automatic "retranslation" process. As long as you are only concerned with text you will not have too many problems, because letters, digits, commas and periods usually are not scrambled when sent between different computer systems. If these characters also are scrambled the transmission process does not deserve the name "communication process" any more and you should talk to the technical people in charge of the transmission channel to take care of these problems. Things become more difficult when you want to send data files or program source files. Files of this kind usually contain special characters like parentheses and to reconstruct the original text of the file you usually have to edit the file you received by hand and to infer from the context the original meaning of a recognizably incorrect character. The automatic file transfer usually takes place between mainframe computers. So the most simple situation with text file transfer is that you use the editor on your mainframe computer to create your text and then you use the mailing program on the mainframe to send the text file (sometimes called e-mail or note) to its destination. At the destination site the receiver then can receive the file and read it with the help of the text editor program on the receiving mainframe computer. Sometimes the situation is more difficult. The file you want to send may exist on your PC, but not yet on the mainframe which is your entrance to the international computer networks. There is an important detail you have to take care of here. Usually you can write texts on a PC using two different kinds of programs to write with: Text editor programs or word processing programs Text files produced by text editing programs usually give no problems when you try to send them over a network. With most word processor files you will experience difficulties. But most word processing programs have a special way of saving your text as a "plain ASCII file". Remember to save your texts with this option if you intend to send them over networks. And if you are still considering which word processing program you should select for your personal use, only select a program which offers this option. If you do not know yourself how to verify the existence of such an option ask somebody more experienced than you to help you to find out. Now you have to find a way to transfer the file from your PC to your mainframe computer. For this purpose you need a file transfer program on the PC and on the mainframe. Different varieties of programs of this kind exist, but the prevalent program in an academic environment at the moment is KERMIT. To use KERMIT to transfer files you need the version of KERMIT for your PC and an installed version of KERMIT on the mainframe. The mainframe KERMIT is not your responsibility, you just have to find out from the staff of your computing center if they already have installed this program. If they have not done so yet you should tell them to do so because KERMIT is one of the very few hardware independent standards and it should be supported. Additionally, all KERMIT versions are in the Public Domain, so they do NOT COST MONEY. Your local computing center also should help you to find the version of KERMIT you need for your PC. KERMIT is a program used for 2 purposes; namely for using your PC as a terminal to your mainframe computer and for transferring files between these two systems. Now things start to be complicated (even more complicated? I hear you complain!). In this paper we will not deal with using KERMIT as a terminal emulator. There are many ways to do this and it mainly depends on which kind of mainframe you are using. You should try to get some help from the people from you local computing center who can show you exactly how to use KERMIT for this purpose. An additional remark: If you only want to use KERMIT as a "terminal emulator", which means using your PC as a terminal, you do not need KERMIT on the mainframe computer you are connecting to. The mainframe version is only needed for file transfer between the mainframe and your PC. Now things become really complicated! The PC KERMIT has only one way of transferring files. But the mainframe version usually has two ways (called "modes" by computer scientists). One way is text mode, the other way is binary mode. Text mode is used to transfer text files. E-mail consists of text files so it is this mode you need for downloading e- mail from your mainframe to your PC. Usually you need not care too much because practically all mainframe versions of KERMIT use text mode for file transfer if not told otherwise explicitly. So simply transferring a text file from your PC to the PC of somebody else you want to send it to can be done using the following steps: 1) Upload the text file from your PC to your mainframe with KERMIT in text mode 2) Use the mail facilities of your mainframe to send the text file as mail to the intended receiver 3) The receiver finally has to download this mail file (it still is text) with KERMIT in text mode to his/her PC In most cases the received file is identical with the original file. Letters and digits arrive as they should. The idea behind text mode of KERMIT is that the meaning of characters is preserved, so when transferring in text mode KERMIT automatically adjusts for different systems of character representations on the mainframe and on the PC. You might find that some of the special characters do not arrive as they should, but this usually is no problem when the text is only intended for reading and not as input to some computer program. Later we will see what you can do if you have to send a text file containing special characters and want to make sure that these characters arrive unchanged. TRANSFERRING NON-TEXT FILES It is becoming even more difficult in this section, but if you want to send programs and data files usable on other machines it is important that you understand this section. Networks can also be used to send PC programs over the network. If you want to send a program to somebody with the same kind of PC you have, the basic procedure is very much like the procedure for transferring text files from your PC via the network to somebody else's PC. The steps involved are: Uploading to a mainframe Using the sending facilities of the network Downloading from the target mainframe to the target PC The difficulties arising with program files are that programs contain more different symbols than text files. They especially contain lots of so called "nonprintable" characters. You can see this if you try to look at your program file with a text editor program or a word processing program. The simplest solution to transferring program files and like things (called binary files in computer terminology) is to use the binary transfer mode of your mainframe KERMIT to upload the program to your mainframe. Binary mode means that no translation whatsoever takes place while sending the file (remember, sending text files often involves a translation process). Now you can use the facilities of your mainframe for sending files over the network. Sending a file is not the same as sending a text as mail. Mailing implies that your text is put into the electronic equivalent of an envelope. Sending a files does not add the envelope, so the file being sent is (almost) identical with what you have on your PC. The receiver then can download the file to his/her PC also using the binary transfer mode of his/her mainframe KERMIT and the PC version of KERMIT. This file transfer quite often does not work. Some reasons may be: the two mainframes involved come from different manufacturers, some intermediate mainframe makes problems or the file is passing through different networks. One situation where it makes sense to try this way of sending binaries is when both mainframes are members of the EARN, BITNET or NETNORTH networks. It usually does not work when the mainframes belong to different networks like EARN and JANET. Now what can we do when we want to send a program or a data file from an EARN site to a JANET site? The main idea is translating your binary file (the one you cannot read because it contains nonprintable characters) into a file consisting only of printable characters. The most popular scheme for doing such a translation is the UUENCODE/UUDECODE process. It implies 2 programs, one usually called UUENCODE and the other one UUDECODE. UUENCODE takes a binary file and converts it into a file consisting only of printable characters. UUDECODE reverses this process and restores the original binary files from the encoded file. So what do you need these programs for? You UUENCODE the binary file and upload it to your mainframe (using the text mode of your mainframe KERMIT). Since it consists of printable characters only, you can incorporate it into a mail file you send. This mail file hopefully arrives at its destination and the receiver can download the mail from his/her mainframe to the local PC. Then it is mandatory to remove the "electronic envelope" from the mail file. An appendix will describe how an UUENCODEd file looks and how to recognize the parts forming the "envelope". Then the UUDECODE program can be used to translate the UUENCODEd version of the file back into its binary version. If you want to use this process you have to get hold of a copy of the UUENCODE and UUDECODE program. It is not possible (at least not in an easy way) to send this programs over networks if you have no experience with encoding and decoding binary files. These programs are binary files themselves and we cannot send unencoded binary files. So we would need the binary files already to translate the encoded versions into the binary version. It is a "who is first, the hen or the egg" kind of situation. There are ways of solving these problems, but the solutions involve a nontrivial amount of technical knowledge and also depend very much on the circumstances of the PCs and mainframes involved. (For the more technically inclined: we could send the source files of the translation programs as text files, but then we have to be sure that the recipient has a compiler for the programming language we are using.) So quite often the easiest way of setting up an environment where file transfer is possible involves sending a disk with the UUNCODE/UUDECODE programs to the sites involved. Once the programs are available file transfer can start. Now let us look what an UUENCODED file looks like: ------- the file starts directly below this line ------------ begin 644 erich.com MZV>0("`@("`@("`@("`@("`@4V\@>6]U(&UA;F%G960@=&\@555$14-/1$4@ M=&AE('1E<W0@9FEL92X-"B`@("`@("`@("`@("`@("`@("`@("`@0V]N9W)A ;='5L871I;VYS(2$-"B0:N@,!M`G-(;@`3,TA ` end ------ the UUENCODED file ended just above this line ------- The first line always contains the word 'begin' starting in the first column. The next item is a number which you can safely ignore and the last item is the name of the UUENCODEd file. The last line of the encoded file consists of the word "end" starting in the first column and nothing else. Some encoding programs add a line containing size information about the encoded file, but this is not really necessary. If you use the UUENCODing program on your PC the encoded version of the file usually has the same first part of the file name as the file being encoded and the file extension .UUE So encoding a program ERICH.COM would produce a file ERICH.UUE . This file ERICH.UUE is the one that should be uploaded and sent using the mail facilities of the network. At the receiving site the mail file sent can be downloaded to the PC. The downloaded file usually looks similar to the following example: ---------------- this line is not part of the file ----------- Date: Sat 14 Jan 89 06:51:59-EST From: John R. Somebody <SOMEBODY@SOMESITE> Subject: File transfer demonstration To: The catcher in the rye <CATCHRYE@MYSITE> begin 644 erich.com MZV>0("`@("`@("`@("`@("`@4V\@>6]U(&UA;F%G960@=&\@555$14-/1$4@ M=&AE('1E<W0@9FEL92X-"B`@("`@("`@("`@("`@("`@("`@("`@0V]N9W)A ;='5L871I;VYS(2$-"B0:N@,!M`G-(;@`3,TA ` end John R. Somebody 1/14/89 SOMEBODY@SOMESITE CATCHRYE@MYSITE 1/14/89 file transfer demonstration --------- this line does not belong to the file any more --- From this example it should be easy to see what the next step is: Every line above the "begin" line and every line below the "end" line has to be removed. The remaining file the can be decoded using UUDECODE. If no additional problems occurred the decoded program is identical with the binary program the sender wanted to send. Now for possible difficulties: UUENCODEd files contain special characters like brackets. Now when you are reading a text file you usually can recognize the intended special character even if it has been changed in a file transfer process. But it is not possible to recognize changed characters in an UUENCODEd file. So you have to find out if all the characters arrived unchanged. For this you can use the method described at the beginning of this paper, namely sending a file with all characters together with a verbal description of the characters. All remarks from the earlier part of the paper apply. Inspecting such a file closely might help you to find out which characters were changed and into what and with luck you can reverse this exchange process. The main problem with the UU scheme is that the set of characters being used contains special characters. So a variant of this method has been devised. It is call the XXENCODE/XXDECODE process. Essentially it functions like UUENCODE/UUDECODE, but the encoded file only contains letters, digits, and the plus and the minus sign. The advantage is that these characters usually are not changed when passed through different computers, so chances are higher that such a file will arrive unchanged. As with UUENCODE/UUDECODE you need the programs before you can start transmission of binary files. The XX scheme is relatively new, so usually it is easier to find programs for the UU scheme than for the XX scheme. It is important to be aware of the fact that UUENCODEd and XXENCODEd files are more than 30 percent larger than the original file. This is the price we have to pay for better transportability. There is one more important concept you should be aware of when transferring more than one file at a time and/or transferring big files. It is the concept of an archive. An archive essentially in one file created by pasting together and compressing one or more files. Usually when transferring a few files you use an archiving program which creates just one file out of a few files. This archived file also is smaller than all the "source" files together. In the archiving process you need two programs: the archiving program creating the archive and the dearchiving program reconstructing the original files. The advantages of using archives are: 1) It is impossible to forget a file belonging to a set of files when transferring copies of an archive 2) The amount of data to be transferred is smaller and therefore uses less disk space and less connect time for transferring them electronically. So if you want to send a few files belonging together it is quite common to create an archive, then to send the archive and then have the recipient reconstruct the original files by archiving. When you receive a file with file name extension ARC it is highly probable that it is an archive file. In this case the extension ARC denotes a special archiving (= pasting together and compressing) scheme. There is a new scheme around now which usually can be recognized by the file name extension ZIP. The 2 programs needed to be able to work with the ZIP scheme are PKZIP and PKUNZIP. Let us look at an example of how to use this set of programs. Let us assume we want to send 3 file named FILEA.TXT, FILEB.DTA and FILEC.COM. If we execute the command line PKZIP ARCHIVE FILEA.TXT FILEB.DTA FILEC.COM PKZIP will create a file ARCHIVE.ZIP. This file is our archive and contains all 3 "source" files in a condensed form. To reconstruct the original files we execute the command line PKUNZIP ARCHIVE which will create the 3 original file FILEA.TXT, FILEB.DTA and FILEC.COM. There are different programs around for the ARC variant of the process. ARC and ARCX are a pair performing essentially the same function as PKZIP and PKUNZIP, PKARC and PKXARC are another pair. There also is a program called LHARC which performs archiving and dearchiving functions with just one program. The difference is that PKZIP and PKUNZIP use the ZIP scheme whereas ARC, ARCX, PKARC and PKXARC use the ARC scheme and LHARC uses the LZH scheme. All these different schemes are incompatible. If you want to create an LZH-archive similar to the ZIP archive of the previous example you can do so with the following command: LHARC A ARCHIVE FILEA.TXT FILEB.DTA FILEC.COM This will create a file ARCHIVE.LZH. Extracting the files from the archive is done with the following command: LHARC E ARCHIVE There is a special variant of archive files, so-called self extracting archives. In this special case the archive and the dearchiving program are pasted together. The result is an executable file (usually with extension EXE) which, when executed, reconstructs the original files contained in the archive. It is not possible to recognize self-extracting archives from the file name extension, so you have to be told that a certain file is a self-extracting archive. So we have met two important concepts: Encoding for creating "mailable" files Archiving for creating smaller files It is quite common to combine these 2 processes. So if we want to send a set of files, first we create an archive containing all the files and then encode this archive. This hybrid product is sent via E-mail. The recipient first decodes the mail file into the archive file and then dearchives the archive into the original files. In this way we combine the advantages of compressing for reducing costs and of encoding to allow better transportability. APPENDIX A: CHARACTER TABLE Next is a list of all printable characters together with descriptions: Characters of the ASCII table blank ! exclamation mark " double quote # number sign $ dollar sign % percent sign & ampersand ' (closing) single quote ( left parenthesis ) right parenthesis * star + plus , comma - minus . period / slash digits 0123456789 : colon ; semicolon < less = equal > greater ? question mark @ at-sign uppercase letters ABCDEFGHIJKLMNOPQRSTUVWXYZ [ left bracket \ backslash ] right bracket ^ caret _ underscore ` left single quote lowercase letters abcdefghijklmnopqrstuvwxyz { left curly brace : vertical bar } right curly bracket ~ tilde ASCII 127 is nonprintable APPENDIX B: TECHNICAL DETAILS OF ENCODING AND DECODING The rest of the paper is very technical, so you should read it only if you have some knowledge of the mathematics underlying the functioning of computers. How do UUECODE and UUDECODE work? For UUENCODing, the bytes forming the file are grouped in groups of three. Every byte is an 8-bit binary number, so every group of three bytes is a 24-bit binary number. This number then is split into four groups of 6 bits each, i.e. into 4 6-bit binary numbers. The 6-bit binary numbers give all decimal numbers from 0 to 63. To every such 6-bit number 32 (decimal) is added, giving numbers in the range from 32 to 95. Every number then is replaced by the ASCII character associated with this value. (32 becomes (a blank), 33 becomes !,... 95 becomes _ (an underscore)). So the translation process converts each group of 3 bytes into 4 printable characters. Additionally every group of 45 bytes (giving 60 characters) is grouped into a line in the file to be sent. Then a leading character is added to this line. The leading character is calculated by using the encoding scheme we just discussed onto the number of bytes represented by the line. (45+32=77, so for a line representing 45 bytes the leading character is M (M is ASCII character 77)). Usually the last line is shorter and therefore the leading character of the last line also is different from M. Finally a first line containing "begin", a 3 digit number (giving access privileges on UNIX systems and meaningless on other systems) and the name of the original file and a last line containing the word "end" is added. The decoding program then mainly has to convert each group of 4 characters back into a group of three bytes (using the byte count given by the first character of each line for consistency checks). There are some problems with this scheme. We already discussed the possibility of special characters being scrambled. Additionally some "smart" mailing programs assume that trailing blanks always are unnecessary. Therefore they strip trailing blanks from every mail file. If it is only text you want to read you will not notice the difference. But an UUDECODing program will find out that the lines are too short (the first character of the line gives information about the line length!). There are different solutions for this problem. 1) Replace blanks by ` (the single opening quote having ASCII value 32+64=96) 2) Add an additional nonblank character at the end of each line 3) Make the decoding program smart enough to produce the missing blanks by itself. All the solutions are nonstandardized, so if you have some troubles when decoding you have to analyze them carefully. Solution number 2 usually works better than the two other solutions. So you should try to get an encoding program adding that additional character. Using an editor also makes it possible to transform the different "extended" formats of UUENCODEd files into one another. How do XXENCODE and XXDECODE work? XXENCODE uses the same splitting technique as the UU scheme (3 bytes into 4 6-digit binary numbers). Then every such number is converted into a character according to the following sequence: +-0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz So (decimal) 0 becomes +, 1 becomes -, (number) 2 becomes (character) 0, .... 63 becomes z. The mechanism for adding byte counts to lines is identical to the UU scheme with the difference the the numbers again are coded according to the above sequence of letter, digits, + and -. So it even is possible to convert UUENCODEd files into XXENCODEd files using the replace feature of a text editor. ACKNOWLEDGEMENTS The author wishes to thank Ted Werntz whose comments and suggestions helped enourmously to improve the paper.