Aminet 18

home *** CD-ROM | disk | FTP | other *** search

/ Aminet 18 / aminetcdnumber181997.iso / Aminet / dev / c / AsyncIO.lha / AsyncIO / asyncio.doc next >

Wrap

Text File | 1997-01-28 | 25.3 KB | 679 lines

TABLE OF CONTENTS asyncio/--background-- asyncio/--history-- asyncio/--changes-- asyncio/CloseAsync asyncio/OpenAsync asyncio/ReadAsync asyncio/ReadCharAsync asyncio/ReadLineAsync asyncio/SeekAsync asyncio/WriteAsync asyncio/WriteCharAsync asyncio/WriteLineAsync asyncio/--background-- asyncio/--background-- This documentation and source code was written by Martin Taillefer. Reading and writing data is crucial to most applications and is in many cases a major bottleneck. Using AmigaDOS' sophisticated file system architecture can help reduce, and sometimes eliminate, the time spent waiting for IO to complete. This package offers a few small routines that can greatly improve an application's IO performance. Normally, an application processes a file in a manner similar to the following: 1 - Open the file 2 - Read some data 3 - Process data just read 4 - Repeat steps 2 and 3 until all data is processed 5 - Close file Although the above sequence works fine, it doesn't make full use of the Amiga's multitasking abilities. Step 2 in the above can become a serious bottleneck. Whenever the application needs some data by using the DOS Read() function, AmigaDOS has to put that task to sleep, and initiate a request to the file system to have it fetch the data. The file system then starts up the disk hardware and reads the data. Once the data is read, the application is woken up and can start processing the data just read. The point to note in the above paragraph is that when the file system is reading data from disk, the application is asleep. Wouldn't it be nice if the application could keep running while data is being fetched for it? Most Amiga hard drives make use of DMA (Direct Memory Access). DMA enables a hard drive to transfer data to memory _at the same time_ as the CPU does some work. This parallelism is what makes the set of accompanying routines so efficient. They exploit the fact that data can be transfered to memory while the application is busy processing other data. Using the asynchronous IO routines, an application's IO happens like this: 1 - Open the file, ask the file system to start reading ahead 2 - Read some data, ask the file system to read more data 3 - Process data 4 - Repeat steps 2 and 3 until all data is processed 5 - Close file Immediately after opening the file, a request is sent to the file system to get it reading data in the background. By the time the application gets around to reading the first byte of data, it is likely already in memory. That means the application doesn't need to wait and can start processing the data. As soon as the application starts processing data from the file, a second request is sent out to the file system to fill up a second buffer. Once the application is done processing the first buffer, it starts processing the second one. When this happens, the file system starts filling up the first buffer again with new data. This process continues until all data has been read. The whole technique is known as "double-buffered asynchronous IO" since it uses two buffers, and happens in the background (asynchronously). The set of functions presented below offers high-performance IO using the technique described above. The interface is very similar to standard AmigaDOS files. These routines enable full asynchronous read/write of any file. asyncio/--history-- asyncio/--history-- 23-Mar-94 --------- - When seeking within the current read buffer, the wrong packet would be sent out to be filled asynchronously. Depending on the data read from the buffer, and how fast it was read, you could end up getting incorrect data on subsequent ReadAsync() calls. - There was actually bufferSize*2 bytes allocated for IO buffers instead of just bufferSize. This is fixed. So if you want the same effective buffer size as before, you must double the value of the bufferSize argument supplied to OpenAsync(). - MEMF_PUBLIC is now specified for the IO buffers. This is in support of VM hacks such as GigaMem. - A Seek() call had the mode and offset parameters reversed. The code worked, because both values were luckily always 0, but it was not very clean. - Now uses a typedef for the AsyncFile structure, and enums for the open modes and seek modes. 16-Feb-94 --------- - SeekAsync() now consistently works. It was getting confused when called multiple times in a row with no intervening IO. - WriteAsync() would produce garbage in the destination file if it had to bring up a "Disk is full" requester, and the user freed some room on the disk and selected "Retry". asyncio/--changes-- asyncio/--changes-- All changes described in this section were made by Magnus Holmgren. If you want to contact me about something, feel free to send a message to cmh@lls.se or "Magnus Holmgren", 2:203/512.10@fidonet.org. I've been in contact with Martin Taillefer, and he is aware of me releasing new versions of (t)his code. However, he is not informed about the actual details of the changes. Thus, any comments about the code (and especially the changes) should not be sent to him. Most of my changes are noted in the sources with a comment that starts with "MH:". The code was compiled using SAS/C. Earlier version were compiled with DICE 3.0. GCC should be able to compile it as well, without any (large) changes (although I haven't tested this). The DICE link libraries are created with the help of the LbMake utility in the DICE package. The file Lib.Def contains the definitions for them. Notes about the different link libraries: · The 'DLib' drawer contains DICE versions of the link libraries, and 'Lib' contains the SAS/C versions ('Libs' contains the shared library). · The asyncio.lib libraries should be used when creating standalone versions of the program. The ASIO_NO_EXTERNALS versions will have an extra 'e' "flag" in the name. · New link libraries with the base name asynciolib.lib are needed if you want to use asyncio.library, and don't use the DICE -mi option (or perhaps you want to have the autoinit code). Note that there is no corresponding SAS/C asynciolib.lib. Link with any of the asyncio*.lib link libraries to get the autoinit code, if needed (there is no glue available, as I don't want to write it by hand ;). · The include file "include/diceclib/asyncio_protos.h" is for DICE 3.0. This include file should go to "dinclude:clib" (you only need this to use the -mi option to use inline Amiga function calls), whereas the other ones should be copied to the respective drawers in "dinclude:amiga??/" (DICE) or "include:" (SAS/C). Notes about how to use the different link libraries: · To use the shared library: DICE - Define ASIO_SHARED_LIB and link with asynciolib#?.lib if you didn't use the -mi option, or you need the autoinit code. Remember that you must copy include/diceclib/asyncio_protos,h to dinclude:clib/ if you use the -mi option. SAS/C - Include <proto/asyncio.h>, and link with asyncio.lib if you need the autoinit code. Currently there is no glue code available, so you must include the proto-file. This proto file defines ASIO_SHARED_LIB for you. · To use the link library for standalone programs: Simply link with the appropriate link library, and make sure you only define ASIO_REGARGS when you want the regargs version. Warning for SAS/C users: If you link with the wrong library, the program will link properly, but will show strange errors when run! The reason for this "problem" is that I wanted to be able to use the regargs version from assembler, and then SAS/C does it this way (i.e. no difference in function names, as would normally happen for register argument functions). · To use the "no externals" version of the link library: You need to compile the library for starters. ;) Then define ASIO_NO_EXTERNALS. DICE - To compile the library simply type e.g. "lbmake asyncio s r e" and have dtmp: set to some temporary space. This creates a regargs, smalldata version. SAS/C - Type e.g. "dmake lib/asyncioer.lib" to get the regargs version. Notes: · When defining any ASIO_#? symbol above, it must be done before any asyncio include file is read. · I haven't personally tested all combinations above. The shared library have had some quick tests, while the "no external" versions of the link libraries haven't even been compiled. ;) · SAS/C users still need dmake (from DICE) to compile things. This since SMake lacks one feature in DMake I find very useful. Release 8 - 28-Jan-97 --------------------- · Yet another hole in SeekAsync() (hopefully) plugged. ;) Seeking in a double-buffered file is tricky business it seems. Thanks to David Eaves for finding it, and sending me a good report about it (including source to reproduce it). The problem was that when SeekAsync() really seeked past the end of the file, it didn't notice, and would try to reload the "last" buffer. · Finally finished up the ReadLineAsync() function. Only difference from dos.library/FGets() should be the return value. I hope it works properly, since I haven't tested it that much. ;) · Bumped the library version to 38, because of the new functions. Release 7 - 7-Jan-96 -------------------- · Files to use asyncio.library from E included. · Fixed yet some SeekAsync() problems. It could (still) get confused when called multiple times in a row with no intervening IO. I wonder if there are any holes left. ;) · Some more fixes to the shared library. · asyncio.library bumped to version 37.2. · Recompiled asyncio.library using SAS/C 6.56, and added general SAS/C support. · Cleaned up parts of this doc. Release 6 - 10-Nov-95 --------------------- · Bumped to version 6, since there was an "official" release 3 that I was unaware of. However, most of the changes were in the AIFF source (see below). Thus, not much was "lost" due to this. (In fact, the SeekAsync bugfix in the real release 3 was not correct. ;) · Sigh. The SeekAsync fix (to the performance problem) was buggy. :/ I think I've got it right now. · asyncio.library bumed to version 37.1. · Cleaned up this documentation. · Fixed some bugs in the include files. · Made some fixes to the shared library. Release 4 - 13-Sep-95 --------------------- · Oops. Forgot to include the include/asyncio.h file. Maybe as well, since I anyway had forgot to mention a few things in it.. ;) · Asyncio is now also available as a shared library (to be placed in libs:). This means that a couple of new include files were added, and a one include file was split up. The main reason for doing this was to simplify the use of it in other languages (only need to "port" a few includes). I have no other include files than those for (DICE) C at the moment, but feel free to send me headers for other languages as well. Note: I have not yet verified that the SeekAsync function works, but since read and write works just fine, I see no reason why SeekAsync wouldn't work. ;) Also, I haven't really used it in programs much either (only some testing), so the different defines and similar to use different versions may be a bit clumsy to use (see above for more information). Feel free to send me comments on how to improve this. · Changed so that a non-regargs version of the link library can be created. To use the regargs version now, simply make sure that ASIO_REGARGS is defined, and link with the proper link library. · Changed the name of the NOEXTERNALS define to ASIO_NOEXTERNALS. Define this before you include <clib/asyncio_protos.h> if you want to use that feature. Note that you need to create a proper link library yourself! (With DICE, all you need to do is "LbMake asyncio s r e" in the Src drawer.) · Modified OpenAsync() a little, to work around a problem when having SnoopDos (with SendARexx active), MungWall and Enforcer running. Not an asyncio bug as such, but.. ;) Release 3 - 12-Aug-95 --------------------- This version includes a couple of enhancements (most from asyncio code found in the sources to the AIFF datatype, by Olaf `Olsen' Barthel) and a couple of bugfixes to SeekAsync(): · SeekAsync() is now not unecessarely slow when doing some "kinds" of read-mode seeks (typically when seeking after some small amount of initial read). The problem was that it usually only considered the newly arrived buffer, not both buffers. This could make it discard both buffers, and restart reading, although one of the buffers already had the needed data. Note that the kind of seeks that caused the above problem may still seem to be somewhat slow, since the code must wait for both buffers to be loaded. This cannot (easily) be avoided. · SeekAsync() doesn't cause the read buffer to contain garbage after certain seeks any more. The problem was that ReadAsync() would read from the wrong buffer (the one currently being loaded). This made the files seem to contain garbage. This happened when the seek location was within the newly arrived buffer. · The code package is now supplied as a link library, rather than a single source module. The internal functions labeled "AS_#?" are private and should not be called directly by the application. · A few minor "cosmetic" changes were done, to either make the code more readable, or to make it slightly smaller. · Include file restructured a little (a public and a private part). · OpenAsync() now offers some new "options" (all from the AIFF code by Olaf Barthel): - Opening a file from an already open filehandle is now possible. - A "no externals" version may be compiled, that doesn't require any external variables to be available. - Each of the buffers will now be roughly bufferSize / 2 bytes large, rather than bufferSize bytes (rel 6 note: Really from Taillefers third release). - If there isn't enough memory for the requested buffer size, the code will try with smaller buffers (still properly "aligned") before giving up. asyncio/CloseAsync asyncio/CloseAsync NAME CloseAsync -- close an async file. SYNOPSIS success = CloseAsync( file ); d0 a0 LONG CloseAsync( struct AsyncFile * ); FUNCTION Closes a file, flushing any pending writes. Once this call has been made, the file can no longer be accessed. INPUTS file - the file to close. May be NULL, in which case this function returns -1 and sets the IoErr() code to ERROR_INVALID_LOCk. RESULT result - < 0 for an error, >= 0 for success. Indicates whether closing the file worked or not. If the file was opened in read-mode, then this call will always work. In case of error, dos.library/IoErr() can give more information. SEE ALSO OpenAsync(), dos.library/Close() asyncio/OpenAsync asyncio/OpenAsync NAME OpenAsync -- open a file for asynchronous IO. SYNOPSIS file = OpenAsync( fileName, accessMode, bufferSize d0 a0 d0 d1 [, sysbase, dosbase ] ); a1 a2 struct AsyncFile OpenAsync( const STRPTR, UBYTE, LONG [, struct ExecBase *, struct DosLibrary * ] ); file = OpenAsyncFromFH( handle, accessMode, bufferSize a0 d0 d1 [, sysbase, dosbase ] ); a1 a2 struct AsyncFile OpenAsyncFromFH( BPTR, UBYTE, LONG [, struct ExecBase *, struct DosLibrary * ] ); FUNCTION The named file is opened and an async file handle returned. If the accessMode is MODE_READ, an existing file is opened for reading. If accessMode is MODE_WRITE, a new file is created for writing. If a file of the same name already exists, it is first deleted. If accessMode is MODE_APPEND, an existing file is prepared for writing. Data written is added to the end of the file. If the file does not exists, it is created. 'fileName' is a filename and CANNOT be a window specification such as CON: or RAW:, or "*" 'bufferSize' specifies the size of the IO buffer to use. There are in fact two buffers allocated, each of roughly (bufferSize/2) bytes in size. The actual buffer size use can vary slightly as the size is rounded to speed up DMA. If the file cannot be opened for any reason, the value returned will be NULL, and a secondary error code will be available by calling the routine dos.library/IoErr(). INPUTS name - name of the file to open, cannot be a window specification accessMode - one of MODE_READ, MODE_WRITE, or MODE_APPEND bufferSize - size of IO buffer to use. 8192 is recommended as it provides very good performance for relatively little memory. sysbase - Library base needed for the "no externals" version of the library. dosbase - Library base, as sysbase. RESULTS file - an async file handle or NULL for failure. You should not access the fields in the AsyncFile structure, these are private to the async IO routines. In case of failure, dos.library/IoErr() can give more information. NOTES (by MH) Although stated that CON:, RAW:, or "*" cannot be used as the file name, tests indicates that the 2.0+ "Console:" volume is safe to use for writing (haven't tested reading). No guarantees though. SEE ALSO CloseAsync(), dos.library/Open() asyncio/ReadAsync asyncio/ReadAsync NAME ReadAsync -- read bytes from an async file. SYNOPSIS actualLength = ReadAsync( file, buffer, numBytes ); d0 a0 a1 d0 LONG ReadAsync( struct AsyncFile *, APTR, LONG ); FUNCTION This function reads bytes of information from an opened async file into the buffer given. 'numBytes' is the number of bytes to read from the file. The value returned is the length of the information actually read. So, when 'actualLength' is greater than zero, the value of 'actualLength' is the the number of characters read. Usually ReadAsync() will try to fill up your buffer before returning. A value of zero means that end-of-file has been reached. Errors are indicated by a value of -1. INPUTS file - opened file to read, as obtained from OpenAsync() buffer - buffer where to put bytes read numBytes - number of bytes to read into buffer RESULT actualLength - actual number of bytes read, or -1 if an error. In case of error, dos.library/IoErr() can give more information. SEE ALSO OpenAsync(), CloseAsync(), ReadCharAsync(), WriteAsync(), dos.library/Read() asyncio/ReadCharAsync asyncio/ReadCharAsync NAME ReadCharAsync -- read a single byte from an async file. SYNOPSIS byte = ReadCharAsync( file ); d0 a0 LONG ReadCharAsync( struct AsyncFile * ); FUNCTION This function reads a single byte from an async file. The byte is returned, or -1 if there was an error reading, or if the end-of-file was reached. INPUTS file - opened file to read from, as obtained from OpenAsync() RESULT byte - the byte read, or -1 if no byte was read. In case of error, dos.library/IoErr() can give more information. If IoErr() returns 0, it means end-of-file was reached. Any other value indicates an error. SEE ALSO OpenAsync(), CloseAsync(), ReadAsync(), WriteCharAsync() dos.library/Read() asyncio/ReadLineAsync asyncio/ReadLineAsync NAME ReadLineAsync -- read a line from an async file. SYNOPSIS bytes = ReadLineAsync( file, buffer, size ); d0 a0 a1 d0 LONG ReadLineAsync( struct AsyncFile *, STRPTR, ULONG ); FUNCTION This function reads a single line from an async file, stopping at either a NEWLINE character or EOF. In either event, UP TO the number of size specified bytes minus 1 will be copied into the buffer. It returns the number of bytes put into the buffer, excluding the null-termination. 0 indicates EOF, and -1 indicates read error. If the line in the file is longer than the buffer, the line will be truncated (any newline will be present). Any data left in the file upto the newline will be lost. If terminated by a newline, the newline WILL be the last character in the buffer. The string read in IS null-terminated. INPUTS file - opened file to read from, as obtained from OpenAsync() buffer - buffer to read the line into. size - size of the buffer, in bytes. RESULT bytes - number of bytes read. In case of error (-1 is returned) dos.library/IoErr() can give more information. EOF is indicated by a return of 0. SEE ALSO OpenAsync(), CloseAsync(), ReadCharAsync(), WriteLineAsync(), dos.library/FGets() asyncio/SeekAsync asyncio/SeekAsync NAME SeekAsync -- set the current position for reading or writing within an async file. SYNOPSIS oldPosition = SeekAsync( file, position, mode ); d0 a0 d0 d1 LONG SeekAsync( struct AsyncFile *, LONG, BYTE ); FUNCTION SeekAsync() sets the read/write cursor for the file 'file' to the position 'position'. This position is used by the various read/write functions as the place to start reading or writing. The result is the current absolute position in the file, or -1 if an error occurs, in which case dos.library/IoErr() can be used to find more information. 'mode' can be MODE_START, MODE_CURRENT or MODE_END. It is used to specify the relative start position. For example, 20 from current is a position 20 bytes forward from current, -20 is 20 bytes back from current. To find out what the current position within a file is, simply seek zero from current. INPUTS file - an opened async file, as obtained from OpenAsync() position - the place where to move the read/write cursor mode - the mode for the position, one of MODE_START, MODE_CURRENT, or MODE_END. RESULT oldPosition - the previous position of the read/write cursor, or -1 if an error occurs. In case of error, dos.library/IoErr() can give more information. NOTES (by MH) If you seek after having read only a few bytes, the function must wait for both buffers to be loaded, before the seek can be done. This can cause small delays. Note that the above case isn't the only one, but the typical one. SEE ALSO OpenAsync(), CloseAsync(), ReadAsync(), WriteAsync(), dos.library/Seek() asyncio/WriteAsync asyncio/WriteAsync NAME WriteAsync -- write data to an async file. SYNOPSIS actualLength = WriteAsync( file, buffer, numBytes ); d0 a0 a1 d0 LONG WriteAsync( struct AsyncFile *, APTR, LONG ); FUNCTION WriteAsync() writes bytes of data to an opened async file. 'numBytes' indicates the number of bytes of data to be transferred. 'buffer' points to the data to write. The value returned is the length of information actually written. So, when 'numBytes' is greater than zero, the value of 'numBytes' is the number of characters written. Errors are indicated by a return value of -1. INPUTS file - an opened file, as obtained from OpenAsync() buffer - address of data to write numBytes - number of bytes to write to the file RESULT actualLength - number of bytes written, or -1 if error. In case of error, dos.library/IoErr() can give more information. SEE ALSO OpenAsync(), CloseAsync(), ReadAsync(), WriteCharAsync(), dos.library/Write() asyncio/WriteCharAsync asyncio/WriteCharAsync NAME WriteCharAsync -- write a single byte to an async file. SYNOPSIS result = WriteCharAsync( file, byte ); d0 a0 d0 LONG WriteCharAsync( struct AsyncFile *, UBYTE ); FUNCTION This function writes a single byte to an async file. INPUTS file - an opened async file, as obtained from OpenAsync() byte - byte of data to add to the file RESULT result - 1 if the byte was written, -1 if there was an error. In case of error, dos.library/IoErr() can give more information. SEE ALSO OpenAsync(), CloseAsync(), ReadAsync(), WriteAsync(), dos.library/Write() asyncio/WriteLineAsync asyncio/WriteLineAsync NAME WriteLineAsync -- write a line to an async file. SYNOPSIS bytes = WriteLineAsync(file, buffer); LONG WriteLineAsync(struct AsyncFile *, STRPTR); FUNCTION This function writes an unformatted string to an async file. No newline is appended to the string. INPUTS file - opened file to read from, as obtained from OpenAsync() buffer - buffer to write to the file RESULT bytes - number of bytes written, or -1 if there was an error. In case of error, dos.library/IoErr() can give more information. SEE ALSO OpenAsync(), CloseAsync(), ReadCharAsync(), WriteCharAsync(), WriteLineAsync(), dos.library/FPuts()