Meeting Pearls 3

home *** CD-ROM | disk | FTP | other *** search

/ Meeting Pearls 3 / Meeting_Pearls_III.iso / Pearls / arc / Packer / StoneCracker / docs / stc.doc next >

Wrap

Text File | 1994-04-26 | 30KB | 816 lines

=-=-=-=-=-=-=-=-=-=-=-=> StoneCracker v4.10.3 <=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= This new version is major bugfix. No big changes or improvments have done. New much faster stc.library is also included! · Fixed some bugs: - When trying to save decrunched (lib)executable to a writeprotected disk, Stc won't crash anymore - Added '** StoneCracker: User aborted.. ' message, when user aborts single file crunching in commandline - If iconify fails Stc won't crash - Stc verified file overwriting only once with the same file. Now checked every time. · PAL/NTSC check - now larger filerequesters are possible under PAL · Dest dir (when presented) is also used with single file saving (huuray, I forgor to add this to previous version.. :( (Note that load/save requesters use the same filerequester) · Some slight modification to code.. Not visible to a user · New stc.library version 3.322 have faster decrunchers and _much_ faster crunching! Separate library for 68020/30/40 is not included! · Stc shows the library version it's using. --> INCLUDED AUTODOCS & INCLUDE FILES FOR PROGRAMMERS!!!! · Explained (professional) kill system decruncher a bit better. A4 register contains info about CPU type. See 2.2.2.3! =-=-=-=-=-=-=-=-=-=-=-=> StoneCracker v4.10.3 <=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Contents 1 General info 1.1 Disclaimer 1.2 Copyright & Distribution 1.3 Introduction to StoneCracker 1.4 Requirements 1.5 Contents of the package 1.6 The future 2 GUI Settings 2.1 Filetypes 2.1.1 Executable programs 2.1.2 Data files 2.1.3 Absolute programs 2.2 Decrunchers 2.2.1 E-Decrunchers for Executables 2.2.1.1 Build In 2.2.1.2 Library 2.2.2 A-Decrunchers for Absolutes 2.2.2.1 Normal 2.2.2.2 Plain 2.2.2.3 Professional 2.2.3 Absolute decruncher defs 2.2.3.1 Load address 2.2.3.2 Jump address 2.2.3.3 Decr address - decruncher address 2.2.3.4 USP address - user stackpointer address 2.2.3.5 SSP address - system stackpointer address 2.2.3.6 SR - status register 2.3 Packmode 2.4 Seclen - security length 2.5 DataID 2.6 Dest dir - destination directory 2.7 Save prefs - save preferences 3 Using GUI 3.1 Information gadgets 3.1.1 File name 3.1.2 File length 3.1.3 Processed length 3.1.4 Crunched length 3.2 Output window 3.3 Load file 3.4 Save file 3.5 Delete file 3.6 Iconify 3.7 Quit 3.8 Shortcuts 4 Using commandline 4.1 What I can do from commandline? 4.2 List of options 4.2.1 Filetypes -fe, -fel, -fa, -fap, -fak 4.2.2 Packmode -p0, -p1, -p2, -p3, -p4 4.2.3 Absolute decruncher defs 4.2.3.1 Load address -L 4.2.3.2 Jump address -J 4.2.3.3 Decr address - decruncher address -D 4.2.3.4 USP address - user stackpointer address -U 4.2.3.5 SSP address - system stackpointer address -S 4.2.3.6 SR - status register -R 4.2.4 Other options 4.2.4.1 Overwriting file -o0, -o1 4.2.4.2 DataID -i 4.2.4.3 Online info -d, ? 4.3 Multifile crunching 5 Advanced info 5.1 Tips for using StoneCracker 5.2 Decrunch info header & decrunching 5.3 Hunk preprocessor 5.4 Includes & autodocs for stc.library Don't get confused b'cos of my bad Engleesh!! =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Document 1 General info 1.1 Disclaimer The author cannot be held liable for the suitability or accuracy of this manual and/or the program(s) it describes. Any damage directly or indirectly caused by the use or misuse of this manual and/or the program it describes is the sole responsibility of the user her/him self. 1.2 Copyright & Distribution StoneCracker v4.10.3 professional - written in C (c) 1991-94 Jouni 'Mr.Spiv' Korhonen of StoneWare SoftWorks Stc.library v3.303 upwards - written entirely in assembly (c) 1993-94 jouni 'Mr.Spiv' Korhonen of StoneWare SoftWorks & Marcus 'Cozine' Ottosson (optimizations!) Really big thanks must go to Marcus who did real hardcore optimizations to both decrunchers and crunchroutine! Note! New address: Jouni Korhonen Salpausseläntie 8 B 10 11100 Riihimäki Finland Email: jikorhon@cc.helsinki.fi Reqtools.library (c) Nico François GadToolsBox (c) 1992-93 Jaba Development StoneCracker (just Stc for now on) (c) StoneWare SoftWorks. All rights reserved. This program is Public Domain as long as you don't make any money with it. If you are interested in using this program in your commercial products just contact me (use Email for faster reply). If you like this cruncher very much send some money, so I could purchase a real accelerator or A1200.. 8^} 1.3 Introduction to StoneCracker What Stc actually is? Stc is an executable/data/absolute file cruncher, not an archiver. Powerpacker, Imploder and the others do the same job as Stc but not so efficiently. Stc was designed to fulfil my own needs and includes those features I think useful. Algorithm used in Stc is called S404. Some LZ variant I think but entirely developed by me and you can notice it (slow & bad efficiency). The name is actually a verion number, where S means StoneCracker and 404 means v4.04. S404 uses 66Kb for a look ahead buffer and 24Kb for a hash table to speedup crunching. Although S404 can't compete with well written LZH, lh5, etc algorithms it still does the best result when compared to other single file crunchers (Powerpacker, Imploder, Spike, Crm, etc). It's faster and more efficient. Next version S405 already exists! And it crunches slightly better (1Kb better per 150Kb). There are two versions of stc.library: stc.library for 68000/010 CPUs and stc020.library for 68020++ CPUs. If your Amiga has 68020++ Stc tries first open stc020.library instead of stc.library. The difference between these libraries are that stc020.library uses new addressing modes and takes the advantage of odd addresses. No significant speed increasement though but faster anyway. You can use Stc either from commandline or GUI. From commandline multifile crunching with asterix '*' is possible and from GUI with multiselect thru filerequester. Stc detects all S403 & S404 crunched files automatically and can decrunch them. Cache support with all decrunchers. VBR check with kill OS decruncher. Fast decrunchers! Thanks to Marcus! I used A2000c 000/28MHz (poor man's accelerator) + kick2.04 + 5Mb ram + 140Mb HD. I had 3Mb but I ran out of memory when compiling the main C program 8^} (Køøl.. And the program size is just 31Kb) 1.4 Requirements Stc works ok on stock A500. 512Kb will do but atleast 1Mb and fast memory are more comfortable. There's no separate NTSC mode. The custom screen is always opened to fit into NTSC (sorry PAL users). Tested in KickStarts 1.3, 2.04 and 3.0. If you are using KickStart 1.3 remeber to use 1.3 version of reqtools.library and gadtools13.library. 1.5 Contents of the package This package should include atleast these files: · stc ; main program · stc.info · libs/stc.library ; any kick, any CPU · libs/reqtools.library ; kick2.0+ · libs13/reqtools.library ; kick1.3+ · libs13/gadtools13.library ; kick1.3+ · docs/stc.doc ; this file · docs/stcautodocs.doc ; about stc.library · sources/optidec.s ; optimised S404 decruncher · sources/cache.s ; set caches properly · include/clib/stc_protos.h ; for programmers · include/libraries/stc.h · include/libraries/stc.i · include/libraries/stc_lib.i · include/pragmas/stc_pragmas.h · include/proto/stc.h 1.6 The future If I ever release a new version of Stc, it will probably be based on some already existing algorithm. (I found some interesting sources from several ftp sites ;^) Of course it all depends on users. If there are no users there won't be new versions either. 2 GUI Settings 2.1 Filetypes 2.1.1 Executable programs Executable programs are normal program files that you can run from CLI or WorkBench. When you load any file, Stc first does a filetype check for that file. If hunk_header is found from the file Stc goes on with loading and hunk processing. If there aren't hunk_header, filetype is internally changed to data and Stc goes on with loading and crunching. Hunk processing means that the standard hunk structure of the file is converted to Stc's own shorter form. Note! If your program includes any debug, symbol, name or external hunks they are deleted. At the moment Stc's hunk processor can handle following hunk types: hunk_header, hunk_code, hunk_data, hunk_bss, hunk_reloc32, hunk_symbol, hunk_debug, hunk_external, hunk_name and hunk_end. Other hunk types will cause an error. For more info about Stc's hunk processor read 5.3. 2.1.2 Data files When filetype data is selected, Stc doesn't care about the contents of the file. Also no decruncher is attached to the crunched file. Standard fileheader is attached to the file. For more info read 5.2. 2.1.3 Absolute programs Absolute programs are sign of *really* *bad* programming habbit on Amiga. You better know what you are doing! Loaded file is handled as a data file and a decruncher is attached to the crunched file. The decruncher makes it possible to execute your absolute program and after loading decrunch it into certain memory location. Stc supports caches & VBR with absolute decrunchers. 2.2 Decrunchers 2.2.1 E-Decrunchers for executables 2.2.1.1 Build in decruncher attaches a decruncher and a relocator to the crunched file. Then it's possible to run crunched program as it wasn't crunched. After decrunching the relocator allocates all needed memory blocks for the program and the moves & relocates the program run there. Before running decrunched & relocated program, both instruction and data caches are flushed and the memory for crunched data is freed. Build in decruncher works with all kickstarts. 2.2.1.2 Library decruncher does everything that build in decruncher does but needs to find stc.library from your system (libs drawer). The advantage of the library decruncher is that it's much shorter (just the opening code for stc.library and one function call) and b'cos the routines in the library are highly optimized, whole decrunching & relocating process takes much less time. 2.2.2 A-Decrunchers for absolutes 2.2.2.1 Normal decruncher is quite simple. It just decrunches your program into certain memory location and runs it. The decruncher needs only load address (see 2.2.3.1) and jump address (see 2.2.3.2). After decrunching both instruction and data caches are flushed. Note! Exec function ClearCacheU() is used for that. So if you have taken over the OS... BanG may happen! 2.2.2.2 Plain decruncher is actually not a decruncher b'cos you can't run it from CLI or WorkBench. Plain decruncher doesn't include any hunks stuff (not runable..) or cachecode. Otherwise it works just like normal decruncher. 2.2.2.3 Professional decruncher is useful when you need to take over the OS or decrunch your (long) program into very low memory. Following things are done before decrunching: - CPU to supervisor (Exec function SuperVisor() ) - All interrupts off ($7fff to IntEna) - All DMAs off ($7fff to DmaCon) - All internal drives are turned off - If 68010++ exists VBR is set to 0 - USP is relocated (see 2.2.3.4) - SSP is relocated (see 2.2.3.5) - Decruncher code is moved (see 2.2.3.3) - If 68020++ exists caches are flushed (no OS anymore) And after decrunching: - If 68020++ exists caches are flushed (no OS anymore) - Status register is modified (see 2.2.3.6) - NOTE! A4 register contains info about the CPU. If A4 = $ffffffff your CPU is 68020/030/040 and if A4 = $00000000 your CPU is 68000/010. If decrunched program would overwrite even parts of the crunched data then cruncher data is move in memory so that decrunching is possible without lockups. 2.2.3 Absolute decruncher defs 2.2.3.1 Load address (in hexadecimales) is the memory location you want your program to be decrunched. 2.2.3.2 Jump address (in hexadecimals) is the start (run here) address of your program. 2.2.3.3 Decruncher address (in hexadecimals) is the memory location you want to place the decruncher code. Used only with professional decruncher (see 2.2.2.3). Don't place inside your stack memory or program memory. The code is 284 (=$11c) bytes long. 2.2.3.4 USP = user stackpointer address (in hexadecimals) is the top memory location for your user stack. Used only wirh professional decruncher. 2.2.3.5 SSP = system stackpointer address (in hexadecimals) is the top memory location for your system stack. SSP is used by the pro decruncher, interrupts, traps etc. whenever the supervisor bit is set in the status register (see 2.2.3.6). Used only with professional decruncher. 2.2.3.6 SR = status register is loaded with this value after decrunching. Used only with professional decruncher. 2.3 Packmode There are five packmodes available. The number 16K, 8K, etc means the maximum distance S404 crunchroutine tries to find equal strings. The higher the packmode value is the better the crunch result is. Though with small files lower packmode value may give better result than the highest value (testing.. testing..). The packmode doesn't affect to memory usage during crunching or decrunching but the lower the packmode is the faster the cruncher is. 2.4 Seclen - security length This value has quite important role. Better way to say security length is overlap distance. If source (crunched) and destination (uncrunched) data overlap, the destination must be atleast 'security length' bytes in higher memory than the source. It will guarantee 100% safe decrunching. Also if crunching fails at the begining of the file, increasing the security length may solve the problem. 2.5 DataID DataID is a string that is automatically added into every crunched data file. For example is dataID is 'argh' then '.argh' is added into crunched data files. If there isn't dataID nothing is added. 2.6 Dest dir - Destination directory Destination directory is only used when you select multiple files from the filerequester. Crunched & decrunched files are automatically saved to destination directory. Empty destination directory is equal to the directory you selected the files. 2.7 Save prefs - save preferences Save preferences save the currect state of GUI gadgets and dataID string into S: (s drawer) as stc.cfg. Stc.cfg is automatically loaded when GUI is used. 3 Using GUI GUI aka Graphical User Interface. The GUI was originally developed with Gadtoolsbox and after slight modifications it also works with kick1.3 (uses gadtools13.library then). Here's a short list of different gadgets and what you can do with them. 3.1 Information gadgets 3.1.1 File name shows the name of the loaded file (in buffer). If this text gadget is empty there's no file in buffer. 3.1.2 File length is the size in bytes of the file in the buffer. 3.1.3 Processed length shows the file length after precossing hunks. For data and absolute files processed length is the same as the file length. 3.1.4 Crunched length File length after crunching. The length also includes 16 bytes of decrunch info header (see 5.2). 3.2 Output window The output window (console) on the right side of the screen is mostly used to output some system messages. Some error messages include '(de:???)' where 'de' means 'dos error'. 3.3 Load file When you press this gadget a filerequester shows up and you can select one or more files to crunch or decrunch. If you selected multiple files they are automatically saved to destination directory. Also decrunching multiple files is possible. 3.4 Save file When you press this gadget a (save)filerequester shows up and you can select or write a save file name and path. Note that if you selected multiple files only the last crunched & decrunched file remains in buffer. The other ones are saved automatically to the destination directory (see 2.6). 3.5 Delete file When you press this gadget a filerequester shows up and you can select one or more files to delete. 3.6 Iconify Iconify will free all buffers, close some libraries, close window and screens and free used resources. A little window is opened and when you press right mousebutton over it Stc will restart. 3.7 Quit Exit this marvellous program. 3.8 Shortcuts Almost every gadget can also be used thru keyboard. Shortcut keys are marked with underscore. 4 Using commandline 4.1 What I can do from commandline? Stc can do everything from commandline that from GUI except for decrunching. Actually you have more possibilities b'cos you don't have to always select from predefined values. It's possible to crunch multiple files at one time. Multifile crunching is a bit limited though. Read 4.3 for more info. You can abort crunching any time with CTRL-C! The structure of commandline is as follows: stc [-<options>] [sourcefile(s)] [destdir] ^ ^ ^ ^ | | | | | | | +-----> Destination directory is | | | not needed if you want | | +--> Files to load. save crunched files into | | Asterix '*' may same directory you loaded | | be used. files from. | | | +--> These are case sensitive. None is compulsory. | +--> Program name Note! There can be only one sourcefile name at the time! And if there's no destdir, sourcedir is used as a destdir. 4.2 List of options Note! All options are case sensitive and may be placed everywhere in commandline. 4.2.1 Filetypes -fe (default) sets filetype executable and decruncher will be build in. -fel sets filetype executable and decruncher will be library. -fd sets filetype data. -fa sets filetype absolute and decruncher will be normal. Only load address (see 2.2.3.1 & 4.2.3) and jump address (see 2.2.3.2 & 4.2.3) are needed. -fap sets filetype absolute and decruncher is plain. Only load address (see 2.2.3.1 & 4.2.3) and jump address (see 2.2.3.2 & 4.2.3) are needed. -fak sets filetype absolute and decruncher will be professional. All absolute decruncher defs are needed (see 2.2.3 & 4.2.3). 4.2.2 Packmode -p0 (default) sets packmode 16K (14 bits). (see 2.3) -p1 sets packmode 8K (13 bits). (see 2.3) -p2 sets packmode 4K (12 bits). (see 2.3) -p3 sets packmode 2K (11 bits). (see 2.3) -p4 sets packmode 1K (10 bits). (see 2.3) 4.2.3 Absolute decruncher defs 4.2.3.1 Load address With -L option you define the decrunch memory location for your program. E.g -L3f002 will decrunch your program starting from address $3f002. Given value is always haxadecimal. (see 2.2.3.1) 4.2.3.2 Jump address With -J option you define the start address of your program that the decruncher calls after decrunching. E.g -J40000 jump into address $40000 after decrunching. (see 2.2.3.2) Note! If you don't define jump address, Stc assumes it's the same as load address. 4.2.3.3 Decr address - decruncher address This option affects only to the professinal decruncher! With -D option you define the location of the decruncher code. E.g -D100 moves decruncher to address $100. (see 2.2.3.3) Decruncher address defaults to $100. 4.2.3.4 USP address - user stackpointer address This option affect only to the professional decruncher! With -U option you define the location of user stackpointer. E.g -U20000 moves the top of USP to address $20000. The stack is reloaded with defined value before decrunching. The decruncher uses SSP - system stackpointer so it doesn't matter even if the decruncher code is over the stack area. (see 2.2.3.4) 4.2.3.5 SSP address - system stackpointer address This option affect only to the professional decruncher! With -S option you define the location of system stackpointer. E.g -S20000 moves the top of SSP to address $20000. The stack is reloaded with defined value before decrunching. The decruncher uses SSP during decrunching! (see 2.2.3.5) 4.2.3.6 SR - status register This option affect only to the professional decruncher! With -R option you define the state of status register. E.g -R2010 forces CPU to supervisor mode and sets C flag. Note! Reloading is done just before jumping into your program! 4.2.4 Other options 4.2.4.1 Overwriting file With option -o1 (default) Stc always checks the destination directory if there is a file with a same name as your file. If there is you are asked whether to overwrite it or not. -o0 option overwrites without any warning. Very useful if you are crunching multiple files. 4.2.4.2 DataID With -i option you can define the dataID string that is added to every crunched data file's name. Default string is 'stc'. Maximum string length is seven chars. E.g -ijouni adds '.jouni' to crunched data files. If you want to save data files without any ID string use -i alone. 4.2.4.3 Online info There are two short help texts included into Stc. With ? you get a quick reference of all commandline options and witd -d option you get a list of internal defults and an example how to use commandline. 4.3 Multifile crunching You can crunch multiple files in one shot. It's quite limited though. If the source filename includes asterix '*' Stc tries to replace the asterix '*' with any string, character or nothing. Note! There can be only one source filename at the time. The destination directory defaults to source directory. So you don't have to define any destination directory. If you define a destination directory Stc tries to find it and if the directory isn't found Stc asks you to generate one. 5 Advanced info 5.1 Tips for using StoneCracker · If you press enter in GUI load address string gadget will be activated. · Loaded file remains in buffer until you load a new one and start loading it. (GUI) · 'Abort crunching' gadget cancels all files if you had selected multiple files to crunch. (GUI) 5.2 Decrunch info header & decrunching Every file crunched with Stc4.02a or Stc4.10.3 has following header (16 bytes) at the beginning of the crunched data: "S404" or "S403" ; cruncher version - string Security length ; overlap size - longword Original length ; decrunched length - longword Crunched length ; crunched length - longword . ; crunched data starts . . Security length is always 16 + something. There are also two control words at the end of crunched data. For historical reasons it's quite wierd. I'll explain it with a following picture: <<- Lower memory Higher memory ->> +------------ Crunched length ------------+ | | InfoHeader|......................LastWord|BitCounter|MaxBits ^ ^ ^ ^ | | | | Crunched data --+ | | | | | | Last crunched word --+ | | | | How many used bits there are in LastWord --+ | | Efficiency (packmode - only in S404) --+ If both crunched data and destination memory overlap there must be atleast 'Security length' distance between the start of the crunched data and the start of the destination memory: <<- Lower memory Higher memory ->> <<<-------------- Decrunching direction -------------- InfoHeader|......................LastWord|BitCounter|MaxBits ^ | |<------------ Destination memory starts here .... | | +-- SecLen ---+ | +---------------> Crunched data starts here.. 5.3 Hunk preprocessor Every executable file is first preprocessed before crunching. Processed files are always shorter than originals. Preprocessor does the following to hunks: · All debug hunks generated by compiler are deleted (no way to get them back!). · There are no Hunk_ends · Hunk header info at the beginning of the file is modified to a shorter form: 1 longword = (total number of hunks)-1 (=n) n+1 longwords = hunksizes & memory type (as normally) · Hunk_code is converted to %01xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.L +-------- datasize>>2 -------+ · Hunk_data is converted to %11xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.L +-------- datasize>>2 -------+ · Hunk_bss is converted to %100000000000000000.W · Hunk_reloc start with %000000000000000000.W and then Repeat until (number of relocentries) = 0 1 word = number of relocentries (null if no relocs anymore) 1 word = relocate hunk number.. 1 longword = %bbbbbbbbxxxxxxxxxxxxxxxxxxxxxxxx.L ^ +----- First reloc ----+ | +--> Number of bytes to add to 'First reloc' 2 = 3 bytes per relocentry 4 or 6 = 2 bytes per relocentry 8 or 10 = 1 byte per relocentry ? = only one relocentry followed by (number of relocentries) %bbbbbbbb reloc- entries.. · End of processed hunks is %1111111111111111.W 5.4 Includes & autodocs for stc.library There are proper include files for standard assemblers (atleast DevPac and AsmOne understand them) and SAS/C. Btw.. include files for stc.library are a messish pile of shit! MOVEQ #0,d0 RTS ; succesful exit