home *** CD-ROM | disk | FTP | other *** search
- From: sergio@g386bsd.first.gmd.de
- Newsgroups: comp.sys.acorn.tech,comp.sys.acorn
- Subject: Disc map: explanation + FAQ
- Summary: An easy explanation of E-Format map and map FAQ
- Keywords: map, checkmap, compact, corruption
- Message-ID: <5238@bigfoot.first.gmd.de>
- Date: 24 Jun 94 07:42:57 GMT
-
-
- This file is aimed at all the people who want to understand more about HD maps
- without going deeply into technical speech. If you have just some questions you
- can try to read the "Possible Questions" section and ignore the rest.
-
- This file will only deal with new format maps (E-Format) since the old ones
- (D-Format) are completely out-of-date.
-
-
- E-Format map, an easy explanation (I hope!)
- -------------------------------------------
-
- Before starting with the map itself, we have to speak about files and disc
- structure.
-
- Each file on a disc (including directories) can be identified from a SIN
- (System Internal Number) that is stored together with all the other file
- informations (name, filetype, size, etc.) in the directory that contains the
- file. The SIN is made up by two parts, the file number and the sector offset
- (more on this next).
-
- The disc (HD or floppy) is physically made up by a number of heads (2 for
- floppies), tracks and sectors per track. RISC OS will generally ignore this and
- will consider the disc as a consecutive sequence of sectors.
-
- The map of an E-Format disc is divided into zones: each zone represents a
- number of consecutive sectors and can be considered as a stand-alone entity.
- So, unless specified, the map format can be applied in the same way to both
- single-zone discs (like 800K floppies) and multi-zone ones. Note that there are
- no relations between the physical disc structure (ie. number of heads, tracks,
- etc.) and the zones subdivision.
-
- On each disc there are 2 copies of the map (one is taken as a backup): quite
- ironically the 2 maps are (obviously!) always identical, so if the first is
- corrupted the second is of no use. The second map can be useful only if the
- first one has been physically damaged but (apparently) RISC OS always read the
- first and ignores the other...
-
- Each zone consists of a list of blocks of variable length, each one
- representing an occupied or a free part of the disc; the formers contain the
- file numbers (ie. the first part of a SIN) and the latters contain a link to
- the next free-space block. if you know the SIN of a file, you can easily find
- out where it is located in the zone and so you can know the exact position of
- the file on the disc (ie. the disc address). The same file number can appear in
- more than one block, in the same or in different zones, if a file is fragmented
- in small chunks; this way you can also have files that straddle across two or
- more zones.
-
- Sometimes two or more files have the same file number but different sector
- offset (ie. the second part of a SIN): this means that in the map they appear
- to be a single file (occupying only one block) but, knowing the SIN, it is
- possible to find out the right address of each one. This trick is useful to
- avoid a lot of disc space wasting: because of the organization of the map, each
- block must occupy a minimum fixed length (that generally depends on the size of
- the HD) and so, even if the file is really short, it has to use up some extra
- disc space. Thanks to the sector offsets, a lot of small files can share the
- same block (thus the same file number) but each one could only use the size of
- a physical sector.
-
- I tried to explain the map structure in a simplified way, without going deeply
- into the real data structure (bit streams, checksums, etc.), but I think I said
- enough to allow you to understand:
-
-
- What can go wrong
- -----------------
-
- Obviously the above structure is really rigid and there is no place for
- errors! Unfortunately errors always occur because of unexpected power down,
- system crashes, physical damages, (errors in FileCore module itself?), etc;
- this is a list of possible problems:
-
- - different file lengths in map and directory: if the allocated length in the
- map is more than the one in the directory you can ignore this (but you have
- some lost space) but in the opposite case RISC OS generally crashes if you
- try to load the file.
-
- - a file number is allocated in the map but not in the directory structure:
- you have some space occupied in the map by an inexistent file.
-
- - a file has a file number that isn't allocated in the map: generally RISC OS
- hangs the computer if you try to load the file.
-
- - the free space linked list is broken: there is some free space that isn't
- linked with the others, thus it is lost (it is no more free...).
-
- In these cases (except in the ones that make RISC OS to crash) *CheckMap
- reports an inconsistency.
- Unfortunately there is a worse case:
-
- - the checksum of one or more zones is wrong: in this case (apparently easy
- to fix) RISC OS isn't able to recognise the disc properly. Using a Disc
- Doctor tool it should be possible to manually fix the problem.
-
- Another quite common error (not directly related to the map) is:
-
- - broken directory: this error is issued when RISC OS finds a directory that
- has some 'strange' data in it (no directory identifier, wrong checksum,
- etc.). The only way to cure this is to remove the directory from its parent
- but unfortunately this is impossible using standard commands (wipe, delete,
- remove). Broken directories usually lead to inconsistent maps because all
- the files contained in them disappear from the directory structure but not
- from the map (see above).
-
-
- What can be wasted
- ------------------
-
- Because of the disc structure and the map organization, some space can be
- 'naturally' wasted.
-
- First of all, the sector bounds are a physical limit and so all the files must
- be 'sector aligned'. So, on a 512 bytes sector sized HD, a 1 byte file always
- uses (at least) 512 bytes, while a 513 bytes one uses 1024 bytes.
-
- But the real waste of space is caused by the map organization: as I already
- said, there is a minimun length for each map block (from now I will consider it
- to be 4K but note that this is only an example).
-
- This is not so bad because more than one file can share it, hence 4*1K files
- can fill up the same block wasting no space. The bad point is that only files
- in the same directory can share blocks, so 4*1K files in 4 different
- directories will use 16K.
-
- A directory always needs 2K of disc space to keep its data. Since a directory
- from the map point of view is just a file, it will still use 4K: fortunately it
- can share the remaining 2K with files contained in it (not with files from its
- parent). If a directory contains a 2K file or 2*1K files all the space is used
- but if it contains only files longer than 2K the space will be wasted.
-
- Wasted space isn't lost space: if you have an empty directory you will waste
- 2K. But when you delete it you will regain the whole 4K, no loss. If you save a
- 1K file in it, the free space on your disc will remain the same and the wasted
- space will be just 1K.
-
- During normal operations, there is no way to avoid wasting space: RISC OS does
- all the work and you don't have to worry about the file allocation strategy.
- Anyway, sometimes you can recover some wasted space by simply copying a file,
- deleting the original one and renaming the new one as the old one (usually it
- is better to copy the whole content of a directory to another and then rename
- the whole directory back) but don't expect too much gain...
-
-
- *CheckMap and *Compact
- ----------------------
-
- CheckMap performs quite a simple task: it just scans the directory tree and
- checks that all the file numbers in the map are actually contained in a
- directory. Considering that CheckMap should be used when something went wrong,
- it seems to be a very poorly written piece of code: sometimes it hangs the
- whole computer, sometimes it just prints the cryptic message "Map inconsistent
- with directory tree" and the only action that it should do to fix the situation
- is the swapping between the 2 copies of the map (but see above to understand
- why nobody has never been able to use this facility...).
-
- Compact has been a very useful command when the only way to format discs was
- D-Format: in fact with E-Format discs it is almost useless. Its function is to
- collect together fragmented files or fragmented free space inside zones (but
- not from one zone to another). With E-Format discs this operation is performed
- automatically when a new file is saved in that zone, but you can anyway use it
- 'manually' to speed up future savings (actually the speed gain is almost
- irrelevant). With D-Format discs the 'auto-compaction' doesn't exist and files
- cannot be fragmented: if there is enough space on the disc for the file that
- has to be saved but this free space is fragmented in small chunks, the error
- "Compaction required" is issued.
-
-
- Possible Questions
- ------------------
-
- 1) Is *Compact useful or it is dangerous?
- 2) Can I trust *CheckMap?
- 3) What can I do with a broken directory or an undeletable file?
- 4) What can I do if *CheckMap doesn't work or tells me that the map is
- inconsistent?
- 5) Are there some dangerous actions that can lead to a map corruption?
- 6) Does a map corruption imply that some data on the disc has been corrupted
- too?
-
- Possible Answers...
- -------------------
-
- 1) Is *Compact useful or it is dangerous?
-
- On E-Format discs *Compact isn't really useful but it can be used safely if the
- map is good. If the map is inconsistent I can't assure you of its behaviour
- (ie. I can't exclude that it can cause worse corruptions). Note that RISC OS
- can automatically run a 'subset' of Compact when it has to save a file
- (especially if the file is large and the free space is limited and fragmented).
-
- 2) Can I trust *CheckMap?
-
- I have to admit that CheckMap isn't a very well written command but if it says
- that the map is good, the map IS good and you have no lost space on your disc.
- If it crashes the computer or if it says "Map inconsistent with directory
- tree", your map or directory structure IS damaged and the damage can worsen if
- not cured soon.
-
- 3) What can I do with a broken directory or an undeletable file?
-
- A broken directory cannot be deleted using standard OS commands and it can just
- be considered as a 'normal' undeletable file. To remove it you must use some
- directory manipulation program (like Look System's Disc Doctor?) or the
- shareware program 'eliminate' (distributed with 'fsck').
-
- 4) What can I do if *CheckMap doesn't work or tells me that the map is
- inconsistent?
-
- Apparently the only thing you can do is to try the shareware program 'fsck'
- that performs the same task as *CheckMap but also tries to fix the damage.
-
- 5) Are there some dangerous actions that can lead to a map corruption?
-
- I am not the only person who thinks that having a nearly full HD (say, less
- than 1Mb free) can be dangerous, especially if you handle very large files.
- Using bugged programs that sometimes can crash the computer while it is writing
- to the HD is REALLY dangerous but fortunately there are very few programs of
- this kind around...
-
- 6) Does a map corruption imply that some data on the disc has been corrupted
- too?
-
- Generally no. For example if the corruption was caused by a delete operation
- that was interrupted after it removed the file from the directory but before
- the removal from the map, the map will be inconsistent but no data has been
- lost/overwritten. On the other hand, a 'Broken Directory' can be the evident
- sign that something actually has been corrupted on the disc, so you can find
- that some other files are damaged. In fact, RISC OS can easily notice the wrong
- directory header/checksum but it cannot really understand if a text file has
- been corrupted!
-
- ------------------------------------------------------------------------------
- I'd like to thank all the people who contributed to produce a 'working'
- version of this document.
- ------------------------------------------------------------------------------
- If you have any other question related to HD map, structure, directories, etc.
- you can contact me or post to comp.sys.acorn. Any interesting question will be
- added to the "Possible questions". Note that I can be wrong, inexact and vague,
- so any correction is welcome. If you think that this file is useless ignore it,
- if you think it is badly written, vague, etc. please contact me, possibly with
- some amendment.
-
- I can be contacted at the following address:
- pel0015@cdc8g5.cdc.polimi.it
- If you don't receive a reply (or if the message is bounced) I can also be
- contacted at:
- sergio@phantom.com
- or:
- sergio@g386bsd.first.gmd.de
-
-