Collection of Hack-Phreak Scene Programs

home *** CD-ROM | disk | FTP | other *** search

/ Collection of Hack-Phreak Scene Programs / cleanhpvac.zip / cleanhpvac / NEOCOG10.ZIP / NEOC.DOC < prev next >

Wrap

Text File | 1994-07-15 | 28KB | 619 lines

──────────────────────────────────────────────────────────── USER'S GUIDE NeoC Explorer Version 1.0 Copyright 1994 (C), Szabolcs Szakacsits Jozsef Attila University, Szeged, Hungary July 22, 1994 ──────────────────────────────────────────────────────────── 1. Introduction to NeoC Explorer ───────────────────────────────── The NeoC program is an implementation of Fukushima's Neocognitron neural network. It's purpose is to test the model, to facilitate interactivity for the experiments, and to make the Neocognitron neural net operation more understandable. Some substantial features: ■ friendly graphic user interface ■ explorer and tester operation modes ■ easy neural net construction ■ graphic or numerical displaying of cells and weights ■ training by patterns or modules step by step, or complete neural net training ■ displaying of the patterns and recognition results ■ recognition statistics ■ even more statistics of the tests in a .LOG file during tester mode The minimal system requirements to run NeoC are a 286 AT compatible computer, a VGA video card, and a mouse. A fast computer with a mathematical coprocessor is recommended because of the many computations. There are two utility programs for NeoC. One of them is the MAP (Make Pattern). Using it one can design new patterns easily and quickly in the graphic mode with mouse support. The other program - 1 - is the DIP (Display Pattern). It converts the binary pattern files to the standard output as text so that the text file can be interfaced with the user's own software. One can use this text file for one's own program (probably, the text file will have to be modified somewhat to conform to the user's own file type), and in this way the experiments can be done with the same patterns comparing NeoC and other neural nets. For this purpose, another solution can be found in the MAP.DOC file. NeoC uses three main type file formats. They specify the ■ neural net structure and parameters (files with extension .ndf) ■ complete neural net (files with extension .net) ■ train and input patterns (files with extension .ptt) The first two are text files, so one can create them using a word processor. However I suggest doing this with NeoC because it's much easier. The structure of a .ndf file is, for instance, the following: 2 1 8 1 6 6 3 6 4 3 3 2 3 3 1 2 0.100000 10 1.000000 1.000000 99.000000 99.000000 The first line is the number of net modules, the next lines are the U0 (input layer), US1 (S-layer of module 1), UC1 (C-layer of module 1), US2 (S-layer of module 2) and UC2 (C-layer of module 2). These lines consist of three columns, i. e. the number of planes in that layer, the size of planes in the current layer (eg. the above second line means that the input layer has one plane with 8 x 8 cells), and the mask size (reception field). Note the size (N) means actually a real size N x N field. The following lines are the alpha parameter, the decision parameter (see later), the r parameters in each and every module, and the last line that contains the q parameters. The .ptt binary files are created by the MAP program. The NeoC command line structure is shown below. NeoC [-m] [file1 [file2 [file3]]] The file1 is a neural net definition file (.ndf), file2 is a train file (.ptt), and file3 is an input file (also a .ptt file). If the command line does not contain any parameters (except for the -m option), the program will suggest a net structure, and it may be modified and saved later. If one tests a particular net structure - 2 - with the same train and input file regularly, then it is useful to make a batch file which consists of the above described lines, and to run it. If only a monochrome monitor is available, then one should run NeoC with the -m option. Information can be obtained on command line structure, if the ?, -?, h, or -h options are used. Usage of NeoC consists of six components: Main Menu, 'Explorer Panel', 'Net Input' and 'Identify as', NeoCognitron Neural Net, Structure, and 'Tester Panel' windows. 2. Main Menu ───────────── Main Menu structure of NeoC: ■ Structure ■ Load ■ Save ■ Change ■ Net ■ Load ■ Save ■ Reset ■ Operation ■ Explorer ■ Tester ■ DOS ■ DOS shell ■ Exit ■ Help ■ NeoC Explorer ■ About The neural net structure cannot be changed in the NeoCognitron Neural Net window because of dynamic memory allocation of the net structure. If one desires to change it, then one has to use the Structure/Change menu. The Net/Reset menu can be used if one is in the Explorer mode (it is automatic in the Tester mode). By selecting it, one may reset the weights of the current module or every module. The value of the weights will be set between 0 and 1 randomly. The 'Explorer Panel' window will appear if one uses the Operation/Explorer menu (this is the default mode). Here are the most possibilities to manipulate and examine the net. The 'Tester Panel' window will appear instead of the 'Explorer Panel' if the Operation/Tester menu is used. Here, the net performance can be measured and analyzed. Some of the menus will not always make sense, should this be the case an information message will be displayed on the screen. - 3 - 3. Explorer Panel ────────────────── The white texts are static and the red ones are dynamic in the windows and window fields. The red texts can be given a new value by pointing and clicking on them with the left, or right, or both buttons. The left button on the mouse will increase the value by one, the right button will decrease the value by one, and pushing both at the same time will cause a small window to appear on the screen. The new value may be written into it. This method works in the case of text variables also. The boxes in view may be left by pressing the ESC key or by pushing both of the mouse buttons at the same time if the new value given is not desired. If the new value isn't correct, a message will indicate the error. The 'Train file' and 'Input file' fields determine which pattern files will be trained and recognized. These fields must be filled in, and the file types must be in the .ptt file format. The pattern size, in both files, and the input layer size of the net must be equal. If these conditions aren't granted, the program will display an error message. The numbers after the file names, are the number of patterns in the current (train or input) file. The net weights and planes (ie. the sets of cells) can be viewed by using the 'VIEW' button which is on the bottom right part of the 'Explorer Panel' window. The 'Display type' field can be: ■ Graphic ■ Numerical The fields which are below the 'Display type' field, determine what kind of elements will be displayed (S-planes, C-planes, weights). If a trained net exists, choose a pattern from the train or input file in the 'Net Input' window by using the 'PAGE' button, and click on the 'VIEW' by using the mouse button. At this time the current values of the net cells and/or weights will be calculated and afterwards displayed. When using the numerical display mode, one can page forward by clicking with the mouse button or one can exit with the ESC key. When using the graphic mode, the solid yellow squares mean positive activity of the cells in the S-planes or C-planes, and they mean trained weights in the reception fields, i.e. the learned masks are shown. The NeoC program provides information on the recognition in the bottom right corner of the Explorer Panel window. The +, -, ? columns mean the percentage of correctly, incorrectly and undecided recognized input patterns. In this version of NeoC, a pattern recognition is undecided if there is no active output or there are more than one active outputs. Since there is no explicit determination of the pattern categories in the .ptt file one has to follow the rules shown below exactly, if the proper recognition statistics are requested: - 4 - ■ If the number of train patterns is N (they will be treated as different category patterns), then the number of input patterns has to be M x N , where M >= 1 . ■ One has to have M patterns from each train pattern category in the input file. ■ The order of categories has to be the same in both files (in the train file and in the input file). For example, if the structure and the pattern categories of a train file are XYZ (X,Y and Z mean a pattern and category simultaneously), then the structure of an input file can be XYZ, or XXYYZZ, etc. It's not necessary to do this if the correct statistic is not desired because a recognition result can be viewed in the 'Identify as' window by using the 'PAGE' button in the 'Net Input' window. The left mouse button will page forward, while the right one will page backwards. The 'Train type' field in the right top corner of the Explorer Panel determines the type of the train. These types can be: ■ PATTERN ■ MODULE ■ NET These types will be discusses at the description of the TRAIN button. The field can be changed by using one of the mouse buttons. The 'Net input' field under the 'Train type' field, specifies either, what the net will learn when the 'TRAIN' button is selected or the pattern type (train or input) through which the information on the net will be displayed when the 'VIEW' button is used. The values of this field may be: ■ TRAIN PTT (train pattern) ■ TRAIN FILE ■ INPUT PTT (input pattern) ■ INPUT FILE By changing this field then one can see either one pattern of the train file or one pattern of the input file in the 'Net Input' window. There is a difference only in that case between 'TRAIN FILE' and 'TRAIN PTT', and between 'INPUT FILE' and 'INPUT PTT' if the 'RECOGNIZE' button is used. At this time, the net will try to recognize either all the patterns of the current file successively (in case of 'TRAIN FILE' or 'INPUT FILE'), or only the current pattern that is situated in the 'Net Input' window (if the 'Net input' field is 'INPUT PTT' or 'TRAIN PTT') One can give the number of the epoch of a pattern at the 'Num. of epoches' field. This value is valid in all the modules. Since the Neocognitron learns module by module, the number of the - 5 - pattern presentations is equal to the number of modules multiplied by the value of 'Num. of epoches' field. If q values are large (e.g. q1=100, q2=10000), then the recommended value of this field is MAX( ROUND_UP( |S-planes in module i| / |train patterns| ) ) where i is 1,2, ...to the last module. If one uses smaller q values and a bigger epoch number, then one will get the same result but more slowly because this implementation is a supervised neural net although it can also work in an unsupervised mode with a little modification in the source. The 'Current module' field is important when one activates the 'VIEW' or 'TRAIN' buttons. This field determines the current module if the net train is started and the 'Train type' field is 'PATTERN' or 'MODULE'. If the 'Train type' field is 'NET', then this field does not have an influential effect during the training because the train is automatic. If one uses the 'VIEW' button, then this field specifies what module (ie. its S-planes, C-planes and weights) will be displayed with regard the pattern in the 'Net Input' window. Information will be displayed about the values of some part of the net graphically or numerically by using the 'VIEW' button. The display is influenced by the alpha parameter, the decision parameter, the r parameters, the pattern in the 'Net Input' window and the 'Current module' field. The mouse cursor will disappear during the computing. The process can be canceled by hitting the ESC key on the keyboard. Finishing this action, the program will show the required information relating to the 'Display type' fields. The 'TRAIN' button. The learning done by the net can be done in three ways. These ways are the net learning by patterns, net learning by modules (from module to module) or full net learning. In the first case (train by patterns), the 'Net input' field has to be set to 'TRAIN FILE' or 'TRAIN PTT', and the pattern, that it is desired that the net learn, has to selected and placed in the 'Net Input' window. These settings are automatic in the other two cases. What is important before the net can start to train in the three different cases? 1. Training by patterns: The alpha, r and q parameters, the S-column, the pattern in 'Net Input' window, the 'Display type' fields, the 'Current module' and the 'Num. of epoches' field are important. The mouse cursor will also disappear during the computing. The process can be canceled by hitting the ESC key on the keyboard. This cancellation method is valid in the next two cases too. 2. Training by modules: The same things must be set-up as above except for the 'Display type' fields and the 'Net Input' window. In this kind of training, these two - 6 - elements have no meaning because all of the patterns will be trained. 3. Training of the complete net: Just the 'Num. of epoches' field and the parameters of the 'NeoCognitron Neural Net' window are important exclusive of the decision parameter. If the 'RECOGNIZE' button is used, then the program will categorize either the current pattern (situated in the 'Net Input') or each and every pattern file in the current .ptt file. Before recognition, one can set in the 'Net input' field what is desired to be categorized: the train file (TRAIN FILE), a train pattern (TRAIN PTT), the input file (INPUT FILE) or an input pattern (INPUT PTT). After a training session, it is worth while, to make sure of that the train file was able to teach the net: that the net learned all of the train patterns correctly. In the next step, one can try to make the net recognize the input patterns. If the 'Net input' field is 'INPUT PTT' or 'TRAIN PTT', then the net tries to recognize the pattern that is situated in the 'Net Input' window. This may be necessary if one had to alter the net parameters because the answer of the net was incorrect for that pattern in the former recognition attempt. This may or may not be a desired final step. It depends upon the curiosity of the user. 4. 'Net Input' and 'Identify as' windows ───────────────────────────────────────── One can view the patterns of the train file or input file in the 'Net Input' window which is related to the 'Net input' field which is situated in the 'Explorer Panel' window. 1. By clicking with the left or right mouse button on the 'PAGE' button at the bottom of the 'Net Input' window, the user will cause a train or input pattern to be displayed in the window, and the results of the previous recognition attempt will appear in the 'Identify as' window. 2. If the 'Identify as' window is empty, then either there was no active cell in the last C-layer of the net, i.e. the net couldn't identify the pattern is situated in the 'Net Input' window, or the result of the former recognition was cancelled by the effect of another event (e.g. new training, modification of the 'Net input' field, loading of a new .ptt file, etc.). 3. If a 'big solid square' appear in the 'Identify as' window, this means that, there were more than one active cells in the output layer of the net. - 7 - 5. NeoCognitron Neural Net window ────────────────────────────────── The white texts are static, and the red ones are dynamic here too. See the treatment of these items above (first part of Explorer Panel). In the left column, are the names of the net layers i.e. U0 is the input layer, US1 is the S-layer of the module 1, UC1 is the C-layer of the module 1, etc. Features of the layers are in their lines successively. Specifically ■ The number of the planes ■ The size of the plane ■ The size of the mask (reception field) ■ The size of the S-column ■ The r (sensitivity) parameter ■ The q (learning) parameter The first three features can be modified in only that case if the Structure/Change menu is called. I found that the size of the S-column was not too significant. I recommend small values (2-4). The role of the r (sensitivity) parameter is very important. It influences the result of the comparison between the reception fields of the S-cells and the learned mask. If this value is small (e.g. below 2 depending on the size of the mask), then the net is more tolerant of the pattern differences. If the r parameter is large (e.g. above 8), then the net can identify only the learned mask in the reception field, and it sets to zero the activity of the other S-cells. One of the experiment subjects can be the determination of the optimal r parameters. The q parameter controls the speed of the train. If this value is large then the learning is faster. In this case, the net will learn a part of the patterns otherwise it can learn a pattern that doesn't exist in any pattern (e.g. a combination of a part of two or more patterns). I suggest big values (q1=100 in the first module, q=10000 in the others) in this version of NeoC. The first version of NeoC was the unsupervised model but the result of the learning was difficult to evaluate. One can tune the tolerance of pattern deformation of the patterns by using the alpha parameter. I have detected no effect so far. In the most cases, I used the 0.01 value. A train pattern category can be elicited from the net by the decision parameter if the net was not able to categorize a pattern at the first identification attempt (no active cell or more active cells than one in the last layer). If its value is 0, then it doesn't influence the recognition (original Neocognitron model). The recommended value is between 0 and 20. - 8 - 6. Structure window ──────────────────── This window can be called up by the Structure/Change menu. One is able to set-up the net structure here. The fields mean the same as in the 'NeoCognitron Neural Net' window. The value of the 'Num. of modules' field can be between 2 and 5. One must invariantly follow the next rule. The 'Plane number' field of the C-layer of the last module must be equal to the number of the train patterns. The first output cell will identify the first train pattern category. The second one will identify the second train pattern category, e.t.c. Recommended but not obligatory: the 'Plane number' of the module i has to be a multiple of the number of train patterns. For example, if there are ten train patterns, then the 'Plane number' of the module i can be 10, 20, etc. If large values are set, then the net will work more slowly. If small values are set, then the net won't learn uniformly from each and every train pattern. The values of the 'Plane size' field must be reduced layer by layer from the input layer U0 to the last layer. The 'Plane size' is always 1 x 1 in the last layer. The 'Mask size' is optimal when it is equal to the 'Table size' of the previous layer minus the 'Table size' of the current layer plus one. In this case the weights of the cells of the current layer cover exactly the planes of the previous layer. Of course, this field can be smaller or larger. 7. Tester Panel ──────────────── The 'Tester Panel' window can be selected by the Operation/Tester menu. This work mode is the most complex and efficient part of the NeoC program. One can test a previously created net construction. The neural net initialization, learning and recognition will happen continuously and automatically as long as neither the number of the tests reach the value specified in the 'Num. of tests' field or the ESC key is hit. The meaning and managing of the train and input file, preparation of the right statistics, and the meaning of the 'Num. of epoches' field are the same as in the 'Explorer Panel' window. A logbook file (recommended extension is .log) will be made during the time that the program is running. If this file is already in existence, then further information will be appended to it. The logbook file reports the recognition in each test, the recognition percentage of every input pattern, and recognition percentage of input patterns that are in the same category. The logbook file name can be added in the '.LOG file' field. - 9 - A discussion of the structure of a .log file is next. The net parameters are the first part of the .log file. The pattern categories ( 0 1 2 etc.), i.e. the train patterns are between the lines that are made with the character '-'. Below the line, the result of the recognition test follows on every line. At the end of lines, the '*' means the current best performance while the '=' means a performance result equal to the best performance. The pattern categories ( 0 1 2 etc.) are again under the next line. The recognition percentage of every input pattern proceed as far as the line composed of the character '='. The last table line contains the recognition percentages of the same category patterns and the average performance. Not only a .log file is made but a neural net is stored which has given the best performance. The neural net name is determined by the '.NET file' field. Thus one can load these net in the 'Explorer Panel' window and view the NeoC results after a recognition process. 8. Epilog ────────── This software is FREEWARE. It may be freely duplicated and distributed. It was created for education and research purposes. A few words about the new versions: ■ running under Windows 3.1 operation system ■ pattern categories => evaluation of tests will always be possible ■ add unsupervised learning ■ add built and designed masks ■ new transfer functions If you find NeoC Explorer useful or interesting and you are interested in the next versions, then please register it. Your registration will contain NeoC and also MAP. You can do so by sending a small donation to my address, and in this way you will support updates of production, and will receive the next upgrades. See the details in the REGISTER.TXT file. If you have any questions, comments or suggestions, please feel free to write to me. ADDRESS: Szakacsits Szabolcs Kodaly 2. 2. lph. H-8800 Nagykanizsa HUNGARY E-MAIL: jzp0119@huszeg11.bitnet jzp0119@vm.cc.u-szeged.hu jzp0119@alfa.cab.jate.u-szeged.hu FAX: +36-93-310-311 - 10 -