Beginner's introduction to the atree release 2.7 adaptive logic network (ALN) simulation software. Welcome to logical neurocomputing and the atree release 2.7 software! This package will help you experiment with learning networks in ways that will instruct, and perhaps surprise you. The package might even be adequate for building some industrial applications, but it is really intended for the researcher and experimenter. Goals of neurocomputing. The field of neurocomputing dates back to the 1940s, at least, and has many branches. One particularly important division is into two groups: those whose goal it is to explain how animals' brains and nervous systems really work, and those who pursue other goals. The latter group direct their efforts to solving artificial intelligence problems, like the control of robot motion or the exploitation of information in images or sounds, using whatever computing tools seem appropriate, whether biologically relevant or not. It is the latter group that atree is meant for, those who want to engineer "intelligent" systems. The word "intelligent" is placed in quotes here, simply because we have very limited expectations of what computers can "understand" today. It would be exciting if ALNs held the secret of building "advanced neural network processors", as referred to in the recent film "Terminator II", but, alas, the most we can hope for is that logical neurocomputing may provide one small component of such future systems. This might analogous to knowing how to fabricate transistors on a chip, but not knowing how to put them together to get a calculator. Since people are very sensitive to "hype" in the area of neural networks, it is desirable not to promise anything until you have done it. Fortunately, there are many demonstrations of practical application of neurocomputing today, so the field can exist without the hype of unrealistic promises. How neurocomputing is applied today. If you have just heard the term neural networks, but have never used one, then the following description will give you an idea what to expect from using this software. Many problems cannot be completely solved by mathematical techniques, or by standard software development techniques, even though there may be a wealth of empirical data available. Just having this data is often a good step towards solving the problem, and neural networks are an important tool in linking the data to the solution. For example, consider a medical application where some measurement data related to symptoms, treatments and history are given for each person in a sample, together with a value indicating whether a certain disease was found present or not (after costly tests). Suppose we want to find a simple relationship between the given data and the presence of the disease, which could lead to an inexpensive method for screening for the disease, or to an understanding of which factors are important for disease prevention. If there are many factors, with complex interactions among them, the usual "linear" statistical techniques may be inappropriate. In that case, one way of analysing data is by the use of neural networks, which can, in principle, find simple relationships by means of an adaptive learning procedure in very general circumstances. Since the method uses only empirical data, very little human intervention may be required to obtain an answer, making the approach very easy to try out. The MOSQUITO example included in the release, is an illustration. Beyond applications in data analysis, such as the above, ANS are being used in an increasing number of applications where high-speed computation of functions is important. For example, to correct an industrial robot's positioning to take into account the mass of an object it is holding would normally require time-consuming numerical calculations which are impossible to do in a real-time situation during use of the robot. An ANS can learn the necessary corrections based on trial motions, and can perform the corrections in real time. It is not necessary to use all possible masses and motions during training, since ANS have the capacity to extrapolate smoothly to cases not presented in training. Extremely high speed response is needed in electronic systems, when parameters have to be adjusted, as in automatic equalizers for communication links. Other applications are in pattern recognition, sensor fusion, and sonar signal interpretation. Can ALNs compute only logic functions? No. Although the networks are constructed using logical operations, it is important to realize that they can also be applied to functions of real values or integers by using appropriate encodings of reals or integers into logical vectors. The results of logical computations are then decoded to obtain the real or integer results. This feature has been included and improved in release 2.5. Comparison between ANS in general and ALNs. The most prevalent approach to neural network learning is the backpropagation algorithm and its variants. Backpropagation depends on fast multiplications and additions, as well as table lookup of functions called sigmoids. Special processors have been built to do ANS computations at high speed. Some systems resort to analog chips for greater speed. The ALN is a type of ANS that has been developed at the University of Alberta, following earlier work at Bell Telephone Laboratories and the Universite de Montreal. It uses only simple logical functions such as AND, OR, and NOT. Execution hardware is just an ordinary combinational logic circuit, a digital logic circuit without cycles or delay units. A learning system of this kind we call an adaptive logic network (ALN). The atree software is a simulator for ALNs. The ALN approach is an extremely simplified version of the backprogagation approach -- with some enhanced heuristics for learning. Though simple, it can replace backpropagation in many situations. Where we expect ALNs to be of particular use is in two situations: when one needs to solve some pattern recognition problem at very high speed, or when one is restricted to using a computer with very limited processing power (for example a personal computer with no math co-processor). We think all PC users will be pleased with the potentially faster learning and execution ALNs offer them. Comparisons with the description of a commercial processor using standard ANS suggested that hardware based on ALNs could evaluate functions several orders of magnitude faster, and would be in the trillion connections per second range. (A connection is a multiply-add operation, and in a NAND (NOT_AND) gate, this is a special case which can be done at very high speed, e. g. 2 nanoseconds or less. The computation time for a tree is proportional to its depth, not its size.) Another advantage of the logic networks is that most of a computation can often be left out. For example if a logical zero is input to an AND-node in a logical tree, then the output of the node can be computed without computing the other input, or even knowing the inputs which give rise to it. This produces no speedup in a completely parallel system, however in systems running on ordinary processors, or in systems which re-use any ANS special hardware sequentially (the usual case), it is of critical importance for speed. Systems which combine special hardware parallelism with the possibility of leaving out unnecessary computations are using "parsimonious parallelism". A small amount of such parsimony applies to ordinary ANS, but in a logical network, the speedup produced can amount to many orders of magnitude, which confers a great advantage on this approach vis-a-vis the usual one. The "fast-tree" feature, new to release 2.5, turns pattern recognition problems into traversal of a binary decision tree. The backpropagation technique for training standard ANS is quite slow. The simulation of ALN learning should prove fast enough in many situations, and we hope will prove satisfactory even on personal computers. Parsimony plays an important role in learning too. There is a hardware architecture for training adaptive logic networks which would make on-line learning quite feasible. A high-speed adaptive processor will soon, we hope, eliminate long waits for convergence of training for all but the most difficult tasks. It will process patterns at a rate of 25 million per second! All the learning tasks we have tried so far would finish in a small fraction of a second. This possibility will undoubtedly open up completely new fields of application for ALNs. Acknowledgment. The production of this package was done mainly by Andrew Dwelly, Rolf Manderscheid and Monroe Thomas, to whom I am very grateful for their dedicated work. They only sought other employment when faced with the alternative of starvation. Important contributions have been made by many others, including Scott Reynolds, Dekang Lin, and Jiandong Liang. Financial support came from the Natural Sciences and Engineering Research Council of Canada. It has supported this type of work on and off for over twenty three years, which I acknowledge with thanks. Future developments. What we have been able to incorporate into release 2.5 just scratches the surface of the possibilities for ALNs. Unfortunately, all my current research money has been spent on the above high-quality programming talent, and those people have now moved on to other jobs. Rolf Manderscheid and Monroe Thomas have kindly agreed to help with getting any remaining bugs out of release 2.5 for the Unix and PC versions respectively however. Release 2.7 will be the last research-oriented one. Because the difficulties of getting adequate support for research into ALNs in Canada appear insurmountable, it is time for this technology to leave the research nest and fly on its own. I am convinced it is a winner, and that some of you will take up the challenge of making it work for you. If someone succeeds in developing a commercial prototype based on the software, I expect commercial licensing of that derived program would be possible, but details would have to be discussed with the authors and the lawyers at the U. of A. If someone wanted to develop a more advanced system, which some of our research has shown to be possible, that is a question of providing funding. The development of the high-speed hardware adaptive processor, the "jewel" of the entire ALN technology, is up for grabs, and would be along the more advanced lines alluded to above. If there is some government agency or commercial organization that would be interested, something could be done there. Until the advent of commercial software and harware, I extend my best wishes for success in using the atree release 2.7 package! Please try a few experiments by just changing the given sample programs first. After you gain confidence, I hope you will try some really exciting projects. Please let me know about your experiences, so future adaptive logic networks can be better. Bill Armstrong Address: Professor William W. Armstrong Department of Computing Science University of Alberta Edmonton, Alberta Canada T6G 2H1 Tel. (403) 492 2374 FAX: (403) 492 1071 email: arms@cs.ualberta.ca Note: If you can't obtain atree release 2.7 from a friend or through ftp from menaik.cs.ualberta.ca [129.128.4.241] in pub/atre27.exe, you can order it by mail. The programs for both Unix (TM AT&T) and PC versions and documentation, including previously published papers and some unpublished ones, will be mailed to you upon payment of $150 (Canadian), made payable to the University of Alberta.