MAPS-CD.txt 7.0 HOW TO BUILD APRS MAPS FROM THE USGS CD ROM 8 June 95 Note: This file used to be named CDROM.txt and was renamed. See also MAPS.txt and MAPMAKIN.txt New in 7.a: W4NMK, Dan Reilly has modified MAKEMAP1 to USGS2MAP.bas which works directly on USGS files obtained over the internet vice a local CD ROM. New in 6.8c: New MAKEMAP1.bas by KB4XF can make a 2,000,000:1 map almost hands free! See the 32 mile map in North Tennessee. New improved CONV100C.bas does NOT crash on city streets! New in 6.7: Brought README.100 into this file to reduce the # of txt files OVERVIEW: There are two areas of CDROM map development ongoing in the APRS community. One is using the relatively crude 2,000,000 CD ROM data to fill in the existing APRS map data with more detail on COUNTY lines, Streams, Railroads and Park boundaries on a typical 32 mile scale. The other is using the 100,000 CD data for making 2 and 4 mile detail maps of city streets. Notice that using 100,000 data to make maps larger than 4 miles generates TOO many points, and using 2,000,000 data below about 32 miles gives too little. For many applications, this detail is NOT needed. With a radio range of 20 miles, it would take 100 such maps to cover the area. At typical radio ranges and the variable accuracy of GPS, the 32 mile maps are ususally adequate for locating and identifying vehicle positions. There are 3 problems with the USGS data that causes excessive points in an APRS map. First, lines begin and end at EACH intersection. For a major road, for example, you end up with dozens of short road segments wasting 3 point pairs per exit. Second, these short segments are not always consecutive in the file, making it difficult and time consuming to try to join them. Third, the USGS data guarantees 100 points to the inch at the original maps scale (1/3 mile for the 64 mile maps) whether it is needed or not. A 10 mile long straight road is stored as 30 points in USGS vice 2 points in APRS. At the 100,000:1 scale, this puts a point every 100 feet! BIG MAPS: Information on using the original 2,000,000:1 USGS CD ROM has been moved to the end of this file, since by now, (fall 94) almost all of the US population centers have at least the ROADS from this data base. The 2,000,000 is also still useful for making 30 mile maps bringing in all of the rivers, streams, county lines, and railroads. You will not gain any more roads than what is already in most APRS 64 mile maps, but the other features can make a more colorful and geographically filled map. DETAIL STREETS: Making APRS maps from the 100,000:1 USGS CD ROMS should be considered EXPERIMENTAL and quite TIME CONSUMING at this point. Do not consider doing it, unless you accept a challenge or think you can help us find a quicker way to do it. For now, it is more fun to me to use a DIGITIZER (if you have a paper map) and take a full evening, than the time involved in the multiple passes with the tools that follow... If however, your digitizer is only 11 inches square and you have to re-place the MAP 4 or more times to get it all in, then I would say it is a toss-up. These days, the focus is on the use of 100,000:1 USGS CD ROMS that contain data on roads all the way down to your street. Because of the tremendous amount of data available in this data base, USGS sells the data on a set of 13 CD's. To see which CD covers your area, hit 1 on your APRS screen to get to the whole USA map. Then press MAPS-ALT command to see the CD.map file which is under the USA map. This display shows you the areas of the 13 different ROMS. To obtain one of these CD ROMS, send $32 for each ROM to the USGS Earth Science Info Center, 507 National Center, Reston, VA 22092. 703-648-6045 or call 1-800-USA-MAPS. BE SURE TO SPECIFY THE 100,000 OPTIONAL FORMAT! (vice standard) HOW TO BUILD A MAP FROM THE USGS 100,000:1 DETAIL CD ROMS The original APRS MAPFIX.bas program could build an APRS map from the separate files generated by the USGS XTRACT.EXE program provided on the 2,000,000 CD ROM. To take advantage of the MAPFIX.bas utility, W7KKE wrote a program called CONV100c.bas which takes data from the 100,000 scale CD ROMS and generates DOS files that 'look like' they came from the 2,000,000 data and therefore usable in MAPFIX. Here are the steps involved: 1. Run the USGS EXTRACT program and follow its instructions to output the appropriate map data for the 15 x 15 minute area that you are interested in. These files will be of the form XXXRDSnn for ROADS and xxxHYFnn for hydro where the nn is for multiple files to cover the area. There will be lots of ROADS files. 2. Run CONV100c.bas and once you give it one of the above file names, it will automatically generate a series of 10 output files of the form PREnRD.GRF for n different ROAD files and two other files preWB.grf and preST.grf for waterbodies and streams. CONV100c outputs these files in the same format as the original 2,000,000 that MAPFIX knows how to read. The PRE is a 3 or 4 letter prefix that you specify for this series to help you keep track of all the files. So far, the number 4, 9 and 10 maps in my area have been empty... but the pre8RD.GRF file is the largest by a factor of ten! It contains all of the neighborhood streets. 3. Run MAPFIX and load an existing APRS map that covers the area where you will be working. This map just makes it easy to visually get the cursor to the right area. Once you hit the ALT-U commmand, this existing MAP is no longer used. MOve the cursor to the center of the map you want to make, and hit the B command to establish the boundary of the area that you want to develop. Choose a scale of 3000, 4500, 6000 or 9000 that will cover the total area you eventually want covered. Next use the alt-USGS command to bring in the first of the above files. Then do the alt-SCRUNCH and the alt-JOIN commands to minimize the map. Then SAVE IT WITH A NEW NAME. Then repeat this step for all the maps that were output from the CONV100c.bas program. After you have saved each of these, you will now have almost a dozen maps each containing a different feature, but in APRS format. WARNING!: NEVER do a second ALT-U USGS BUILD on the same map, because it will clear all current points and start a new map from scratch. 4. Next, run Ken's DEFRAG.bas program on EACH of the files generated by MAPFIX in the above step 3. This program goes through the files over and over again, trying to join together all of the disjointed road segments (of the same class) generated by the USGS data base. Hopefully you may see a 20% or more reduction in points. Re-save each of these reduced maps under a new name so that you do not distroy your originals (if there is a problem. NOTE: Be sure to also review the steps at the end of this file on how MAPFIX is used to read the 2,000,000:1 CD ROMS. Since the 2,000,000:1 process was developed first and is the basis for MAPFIX routines, you might find some helpful details or clarifying information. W7KKE's CONV100c.bas program simply puts the 100,000:1 OPTIONAL data format into the GRAPHICS format which is then readable my MAPFIX.bas. His instructions are in README.100, NOW INCLUDED AT THE END OF THIS FILE. 4. Now, combine these maps into one APRS map, using the MAPFIX alt-IMPORT command. This command will bring just one, or all of the map features of one file into the current file. In an urban area, the level 8 road file will be larger than all the others combined. You can omit it to allow your map to cover a larger area or include it for the finest detail available. Good luck. I am just beginnning to play with this level of detail, and to this point, it is quite practical to use a digitizer! For more detail on the use of MAPFIX with CD ROMs, also read the 2,000,000:1 section below, since it evolved and was the basis for APRS maps before we began messing with the 100,000:1 data. ---------------------------------------------------------------------------- PROCEDURES FOR 2,000,000:1 CD ROM THAT COVERS THE WHOLE US Most of the initial 64 mile and larger maps were made by XTRACTING only the ROADS from this CD. But NOW (May 95) we have the new MAKEMAP1.bas program which makes these 2,000,000 type maps (including county lines and streams and railraods) relatively easily. The first such new map that includes ALL features on the 2,000,000 CD is in the N. Tennessee map, TNnorth.map. Note that it is quite full but only covers about 32 miles. MAKEMAP.BAS The new precedure uses the MAKEMAP program by KB4XF @ KC4ASF .#NOVA.VA that XTRACTS data directly from the CD and saves it in APRS format. You simply choose the state and specify a LAT/LONG and range for the center of the map. By choosing a small enough area (30 miles or so in the east, maybe 50 or more in the rural west) you get a resulting map with about 4 to 5000 points. THen load this into MAPFIX and use the alt-SCRUNCH command and finally go through each point and use alt-DELETE to get the number of points below 3000. This is a nice painless process. But remember that the detail on the 2,000,000 CD ROM is about the same (or worse) as looking at a typical state highway map through a magnifying glass!!! USGS2MAP.BAS: This is a modified version of MAKEMAP1.bas by Dan Reilly, W4NMK in North Carolina which will work directly on USGS files obtained over the internet. Get these maps from: http;//edcftp.cr.usgs.gov/pub/data/DLG/2M/{area of interest} /{transportation/hydrographic/political/railroads}/ Chose the area of interest and the features from the menu. You will then generate ROADS.GRA, WATER_BO.GRA, STREAMS.GRA, POLITICA.GRA and RAILROAD. GRA. Combine these all into one file. One way is with the COPY command: Copy ROADS.GRA+WATER_BO.GRA+STREAMS.GRA+POLITICA.GRA+RAILROAD.GRA ALLSTUFF.GRA Where the file named ALLSTUFF will be your new source file. Then follow the instructions in the program (just like MAKEMAP1). OLD MAPFIX METHOD: The following STEPS outline the older method which is pretty obsolete. This method uses the USGS XTRACT.EXE program to extract separate files for each of the major map categories of features. Then you use MAPFIX multiple times to combine the individual features. 1. Use USGS XTRACT.EXE program to extract the CD ROM data in the GRAPHIC format for your area from one of the dozen or so major USGS areas. Then select up to 6 categories for extraction onto your HARD disk, one for each of the data types, Political BOUNDARIES, ROADS, WATER BODIES, STREAMS, RAILROADS, and HYPSOGRAPHY (continental divide). These files will take about 5 to 10 MBytes. Note the filenames assigned by XTRACT. 2. Run MAPFIX.BAS program and load the best existing APRS map that covers the desired area. This map is only used to help you locate the area of interest. Next, move the cursor to the center of interest and select a SCALE using the BOUNDARY (B) command. This command will show you the maximum map possible for a given scale in Points/per degree. I have found 600 to be about right. 450 in sparse states and 300 in UTAH and NEW Mex. If you use any less, to get larger maps, you will end up with far too many points. I used 900 for a 35 mile map around Philadelphia. Making maps any smaller than this is probably not worth the effort, since there will be so few roads in it.. 3. When you have the B command set to the desired number of Pixels-per- degree, press the alt-U key. MAPFIX will allow you to round the ORIGIN values, to reasonable nearly whole numbers or fractions of a degree so that later manual operations with a map and ruler will have round numbers to deal with. 4. You do one type of data at a time. MAPFIX asks you for the type of file (RD, ST, WB, etc) and then the exact path to the XTRACTed USGS data. This path should include the first part ot the filename up to the (_) underline that separates the RD or WB etc final two characters. On my system it is usually D:\NEAST\S01_ and then MAPFIX adds the RD.GRF on the end. MAPFIX will then scan the entire indicated file for all points within the selected border. While converting the points to APRS format, MAPFIX will also compare the slope of each new point with the previous point and ignore the point if it has approximately the same slope, (within 1.2). The total number of points will be from 2000 to 4000. 5. Next use the alt-JOIN command. This will cause MAPFIX to scan through the map looking for consecutive map features that end and then start at the same point. These two features will be joined into one longer continuous feature. 6. Next use the MAPFIX alt-SCRUNCHER command which will run another slope comparison. You can select the SCRUNCHER factor. A factor of 1 will only throw out points that are exactly the same slope (few if any). A factor of 1.2 (best) will throw out points whose slope differs by less than 1.2. Bigger values will throw out more points and will tend to make shortcuts out of any long slow curves. Anything above 1.5 should be avoided. The safest approach is to do the alt-SCRUNCH at 1.1 twice, and then if needed to it again at 1.2. 7. At this point, you can save the map using Ctrl-S giving it a name which includes the feature type (..RD, ..ST, ..WB, etc) As long as you have not hit the SPACE bar or changed scales, the original MAP which was used for oreintation will still be on the screen with all the new dots. This map is NOT involved at all, but is still on the screen. Be sure you have saved the new map points; and then hit alt-U again to do the process again for another feature type (RD, WB, ST, etc). If you have not changed anything your ORIGIN values will be reused again. 8. Repeat steps 3 through 7 for all 6 data types, being sure to SAVE each type separately, but without re-drawing the map. When you are done, quit MAPFIX and run DEFRAG.bas on the RD file. (there is not much to join in the others, due to minimal intersections). This may remove some additional points by further joining lines. DEFRAG does not do LABELS, so if you run DEFRAG on a map with labels, they will be lost. 9. Now you have the smallest APRS maps that can be done automatically. Reload MAPFIX, reload the RD file and Move the MAP POINTER to the beginning FEATURE using the RESET command and use the gray +/- keys to cycle through EVERY point in the file. Use the alt-D key to delete all unnecessary points. Look for complete duplicates of some lines too. Often, both lanes of a 4-lane or interstate are in the file and will be EXACTLY the same! These steps should get you down to below 2000 or so. 10. Now simply use the Alt-IMPORT command to bring in each of the other feature maps. SImilarly go through and remove unnecessary points. Be sure the WHITE box is way out beyond the area of the maps, so there is no TRUNCATION going on with the IMPORT command (It only imports points within the white box.). There are usually LOTS of points that you can remove from the PB, AB, WB and ST files, with little loss of detail. NOTE: If you use the new MAKEMAP.bas/exe program, you will arrive at this step 10 in ONE step, and only need to do step 6 to begin reducing points. ALTERNATE PROCESS FOR WB, ST, BP, AB FILES: Another way to add some detail from the other non-road USGS files, is to use the MAPFIX USGS OVERLAY feature to temporarily overlay the USGS data on the MAPFIX screen without adding points. Then you use the NEW-FEATURE and ADD commands to draw APRS features over the top of these temporary points. The philosopy here is that 96% of amateur interest is on roads, and you can sure burn up points on waterways and county lines. For rivers, about the only points that are important are where the water goes under a bridge. You might want to first overlay on the 64 mile scale to see where things are, but then zoom into the 16 mile scale at least and do the overlay again to get enough resolution for drawing a particular feature. Usually, I sketch in rivers at the 32 mile range, because I do NOT try to get every kink in a river or stream... You can use the NEW, ADD and MOVE commands, but any other commands will redraw the map and wipe-out the overlay. Just hit the U command again to reproduce the same overlay. DO NOT CONFUSE THIS with alt-U which will WIPE-OUT your map. Use the Ctrl-R command to toggle the map REDRAW process on and off. This may help you to add lines without destroying the USGS overlay. You can do the overlay again and again. WARNING!: NEVER do a second ALT-U USGS BUILD without FIRST saving what you have, because it will clear memory and start an all new file. CAUTION: Do not expect too much from this 2,000,000 CD ROM. ALthough the 2,000,000:1 USGS database seems to have resolution down to 1/3 rd of a mile, remember that that 1/3rd of a mile was actually 1/100th of an inch on the original map! Your 64 mile section was only a two inch square!!!!! AND THE DATA WAS PROBABLY DERIVED BY A HIGH SCHOOL SUMMER STUDENT using a mouse! IT STILL TAKES AN EXCESSIVE AMOUNT OF MANUAL EFFORT TO REDUCE THESE MBYTES OF USGS DATA DOWN TO A reasonably quick loading 25K APRS MAP! IT TAKES ME 4 to 12 HOURS PER 64 mile MAP! About 3 hours for 32 mile maps.. By now (Fall 94) I have done most of the US to the state highway level of detail. IF YOU WANT ME TO DO A SPECIAL MAP OF YOUR AREA, to the resolution in the 2,000,000:1 ROM, SEND $$$ or convince me it is in the general best interest of all HAMS. FOR THE 100,000:1, I will only do these maps commercially, because the 3 days worth of effort only results in a map that covers a few miles. Only a few HAMS will even live inside one of these small areas. If you are a MAP-ology expert, or an ace programmer and can help us figure out a way to better automate the process, then we would love to make the whole process easier for everyone. Contact Ken Swaggart, W7KKE in Calif, or myself. ***************************************************************************** README.100 (as of 2/13/94) 1:100,000 CD ROM Map Making W7KKE Ken Swaggart w7kke @ ka6eyh.#nocal.ca.usa The 1:100,000 line segment maps contained on the USGS CD ROM uses a different format than the 1:2,000,000 CD ROM maps. My first utility, CONV100.BAS, converts the 1:100,000 map "Optional Format" extracted files to the .GRF file format which can be read by the APRS MAPFIX.BAS program. When you import these map files into MAPFIX you will notice they are extremely fragmented into small line segments which are not adjacent. The second utility, DEFRAG.BAS takes the saved MAPFIX file and defragments it. Here are the step by step instructions: 1. Create a directory on your hard drive where you will be work on your maps. Stick my programs in this directory and copy MAPFIX.BAS from the APRS directory into this working map directory. I use C:\APRSMAPS as my working map directory. 2. Extract "Transportation" and "Hydrography" for your area of interest from the 1:100,000 CD ROM using the extraction program which comes on the CD ROM. I zoom down to the 15' scale on the USGS extraction program before I extract the map file. Copy these files into your working map directory. 3. Run CONV100.BAS. The program will extract each feature, such as Interstates, US Highways, etc., as separate files with the filename extension of .GRF. 4. Run MAPFIX.BAS. Import the first converted .GRF file by using the @U command in MAPFIX.BAS. After the file is imported you and on your MAPFIX screen use the Join command (@J) to join adjacent line segments for this feature. Save this feature as its own .MAP file and repeat the process for each map feature you created. NOTE: Make sure you use the same origin and scale each time you import the .GRF file using the @U command. 5. Run DEFRAG.BAS to de-fragment the .MAP files you have created. If DEFRAG.BAS crashes, you can recover most of your defragmented segments. Look for SCRATCH1.TMP and SCRATCH2.TMP. Rename the larger of these two and delete the remaining .TMP file. 6. Use a text editor, such as DOS's EDIT, and examine your defragmented .MAP files and tweak as you see fit. You may find that some line segments consist of only one point - EDIT THOSE OUT 7. Run MAPFIX.BAS again and import your defragmented .MAP file. Check it over for any obvious errors, correct those, then run the scruncher function (alt-SCRUNCH) to pare down the .MAP file. Note: I generally do not run SCRUNCH on the water boundaries and streams, but hand edit to reduce the points. Another option which works well is to use the U command to overlay the water boundaries and streams, then manually draw in the rivers with the @new and @A commands. This results if far fewer points than the automatic method. 8. Save your scrunched .MAP file and import your next feature and repeat the process. (I only run the scruncher on each separate feature file so I can control what gets scrunched.) Sometimes I have to bump the scruncher factor way up to pare down the points to fit the total point limit for APRS. 9. Use a text editor, such as DOS's EDIT, to assemble the final map from all the defragmented and scrunched .MAP files created for each map feature type. BOB COMMENTS: I prefer to just use the MAPFIX @IMPORT command to bring these into the working map... 10. Run MAPFIX again and load the resulting complete .MAP file with all your features. If it is too large, which it probably is, delete points until you have a map which will fit within APRS limits. - There is some small errors (less than 1/4 mile) toward the north east and north west sections of the map. This is due to the conversion scheme I use to get from the UTM grid to lat/long. The two grids do not line up exactly. When you join two adjacent 15' maps, you will see that the roads are off by about 100 meters. Use mapfix to tweak the lines so they join. Please send me any program enhancements/fixes you make to either program. I'm sure there are a lot better ways to defragment the .MAP files - there MUST be! PS - watch out for "Streams" - when I extract the ones around here the resulting .MAP has thousands of points! Changes from CONV100B to CONV100C.bas are: Now does not crash on very large street files. Before, it would often crash when extracting class 4 streets (neighborhood type streets). Be prepared for very large class 4 files - over two megabytes for urban areas. I extract the source date for the data on the 1:100,000 CD ROM. This is to clue you that you might be working with old data - some in northern California are 12 years old!