The Amiga Foundation Layer

With the impending first release of the Software Development Kit (SDK), we will be launching the first new Amiga product in over 5 years. The SDK will provide all the software elements required to develop for the new Amiga. We want to share with you some of the features and concepts our pathfinding developers will encounter, some insight into why Amiga has made some of its choices, and some idea of where we intend to go over the short, medium and long terms. The Amiga has always been about enjoyment, and its secret weapon has always been the smile. Too often these days people and technology collide in a multiple pile-up of confusion, frustration and frowns, whether it be someone wanting to record a program, add a new purchase to their house or developing a new audio codec. The eastern philosophy of Taoism teaches the principle of Wu Wei, which is (poorly) translated as effortless effort. This has always been and will continue to be the guiding principle of the Amiga philosophy--the creation of products that empower the user, customer and developer while being highly tuned to the tasks that they want to perform. They are thus able to do what they want to do without having to concern themselves with the underlying system. Because they are able to succeed in what they want to do, they find themselves smiling. The flow of Tao into the Amiga is now more than just a philosophical reference. We looked at many potential partners and technologies before finally settling on the provider of the Amiga Foundation Layer. At issue was ensuring that we found a product that matched our very different technical and cultural view of the digital future. Since we see part of the problem as originating in traditional architectures, it was very important to us that we picked a partner who not only saw the future as we see it but who "walked the walk" as well as they "talked the talk;" in other words, a partner who not only got the future but who had built a product designed for that future.

The Tao Group's story starts with Chris Hinsley, a games developer throughout the 1980s, the golden age of the home computer. He would develop a game for the Amiga, then be asked to port it to the Atari ST or to the PC. After one or two ports it occurred to Chris that there had to be a better way, so he began experimenting. First he came up with a heavily macroed assembler; this soon evolved into something more--A virtual processor that could be simply and quickly implemented on top of multiple CPU families to provide the holy grail of software development: Binary portability without sacrificing high performance or memory efficiency, both prerequisites given the specification of hardware in his market space. With such a cutting edge concept Chris was able to attract a lot of talented developers, and from the virtual processor rolled out other powerful features, including effective dynamic binding and transparent multiprocessing capabilities. This was rolled into TAOS, introduced to most of us for the first time in the excellent Byte article by Dick Pountain <http://www.byte.com/art/9407/sec6/art1.htm>. That was five years ago. Since then the Tao Group has been improving, adding to and spreading its product. Its second-generation is called intent, and combines a wealth of functionality that includes a series of multimedia libraries and the intent Java Technology Edition. The latter is fully Sun certified and the first implementation of the PersonalJava specification. It is this product we have selected to be the Amiga Foundation Layer. The core of the Amiga Foundation Layer (AFL from now on) and of the whole new Amiga architecture is the Virtual Processor. As befits something that is virtual, it doesn't actually exist. It is an abstraction to which source code can be compiled to create binaries. The virtual processor is a 32 bit little endian RISC processor which provides 5 banks of registers: integer (i0-i(n) - 32 bit), long (l0-l(n) - 64 bit), float (f0-f(n) - 32 bit), double (d0-d(n) - 64 bit) and pointer (p0-p(n) - 32 bit). The developer accesses these registers using the macro assembler language VP, which assembles into VP Code, the binary portable object code. VP Code has been designed to allow the developer to take the small instruction set and grow it into powerful expression trees, giving the benefit of compact binaries while also allowing for the application of powerful optimization techniques. VP provides not only register and memory access through the basic instruction set but also provides a rich and powerful set of macros covering assignment, condition, iteration and some higher level functionality that will be instantly familiar to C programmers. Most attempts at the creation of portable binaries have ended up sacrificing portability for speed through the use of interpreters.

VP Code is NOT interpreted. It follows the same pipeline from source to machine code as do compiled architectures, with a single modification: It stretches the pipeline and inserts VP Code between the compiler back end and the creation of native (i.e. CPU specific) machine code. VP Code thus becomes the distributable rather than the native machine code. The final piece of the compilation pipeline takes place not on the developer's machine but on the user's machine. When the user process requires that piece of code be executed, the VP Code is loaded and compiled dynamically into the machine code of the host processor. This native machine code, which is then executed, providing the same speed as classic compiled code... And sometimes more. How is that possible? VP Code has been designed to be as small as possible, and often the VP version is much smaller than the native code version. While there is the obvious overhead of translating VP Code to machine code, this is much less than the overhead of reading the larger machine code version from storage (perhaps a hard drive). Adding to this the fact that the translators themselves are often less than 100K in size (and therefore stay in the cache on many processors), and when the total cost equation is calculated, the VP Code solution is often faster. Because it is translated dynamically on the target environment, VP Code can make intelligent decisions in real time--something far beyond the scope of traditional architectures, which must prepare a general distributable and add code for all possible variants. For example, the translator may take the VP Code and generate different machine code versions on an i386 system as opposed to a Pentium system because the translators for each are smart enough to provide optimizations. Thus the path from development to execution is: High Level Language is compiled, into Low Level Assembly Language (VP), which is assembled, into Binary Portable VP Code, which is distributed and installed on a system that calls for the execution of that piece of VP Code, which is then loaded from storage, into The Translator, which then optimizes and translates the VP Code into native machine code for the host CPU, which is then Executed as normal on the host CPU. Note: Java is compiled into Java byte code, which is translated at load time into 3.

The Amiga advantage is thus both binary portability and speed. What was always seen as a compromise can now be a dual benefit. The more perceptive of you may have seen a potential flaw in this architecture. If you have to translate a whole library or class every time it is required, then the overhead will be tremendous. The answer to this question leads nicely into another key feature of the Amiga Foundation Layer, one with which VP programmers will become very familiar. The AFL utilizes a tool-based architecture. A tool is a small piece of code that performs a single operation. To understand this better, consider classic C or C++ development, which leads to the creation of libraries, each containing a set of functions, or to the creation of classes, each containing a set of methods. When VP is used, libraries and classes are replaced by a set of tools, one for each function or method. Thus, the AFL provides a very finely grained environment with tools often fitting into less than a kilobyte of memory and enabling toolkits (sets of tools targeted at a specific functional area) to offer a lot of power in a small size. For instance, the standard kernel is 62 KB, the AVE is 152 KB, the complete ANSI/POSIX library is 80KB and the Pentium translator is 88KB. When a tool is called, the AFL first checks to see if it is already available in memory as a previously translated tool. If it is, then execution flows directly into it, otherwise the tool is pulled from storage, translated and then placed as a native executable into memory. The advantage of this dynamic binding is clear. Not only can targeting be done at run time using state information, but the AFL only has in memory the tools it needs. Compared to this granularity, the loading of an entire library or class just to use a few functions or methods seems positively archaic. Those who asked the first question may now ask a second question: "Ahh, if this virtual processor is so portable, what about processor specific features such as AltiVEC, SSE or 3DNow? The virtual processor doesn't know about them." Well, not only do the translators have the intelligence built into them to be able to take advantage of these extra instructions, but a developer can code both a VP Code tool for general release and then a native tool, both with the same name and providing the same function. If a particular tool is called, then the system will first look for a native tool specific to that system (which of course requires no translation and is optimized to that system) and only if one isn't available will it utilize the default VP Code tool. The AFL does this by using the naming convention, toolname.nn where nn represents a specific processor target and 00 represents the virtual processor. Tools have a name, an entry point, one or more exit points and their own set of registers, and are fully re-entrant. There are two type of tools: Main tools and secondary tools. Main tools are directly startable, and secondary tools are callable only from a main or another secondary tool.

One of the strengths of the Amiga Foundation Layer is that it fully supports both static and dynamic binding. Static binding is provided through the SYSGEN utility and is mainly used for compiling to embedded systems, where all references are known before time. For the Amiga Foundation Layer, we will concentrate on using dynamic binding. This is where tools are loaded as they are referenced and it can happen either when a process starts or when another tool makes a call. In general, the binder performs this function. Tools call each other via qcalls or quick calls, of which there are 3 types. 1. Normal qcall - the referenced tool is found at load time and the calling tool is fixed up so that execution jumps directly to the called tool's entry point. This is the fastest possible call. The tool remains in memory until the calling tool is removed. 2. Virtual qcall - at bind time, the calling tool is fixed up to call a kernel function with the name of the tool required. At run time, the kernel function finds the referenced tool, loads it and then calls it. The referenced tool is de-referenced when the call returns. This is slower than a straight qcall but is very efficient on memory. 3. Virtual + Fixup qcall - at bind time, the calling tool is fixed up to call a kernel function with the name of the tool required. At run time, the kernel function finds the referenced tool, loads it and then calls it. The tool then remains in memory for further calls. Thus, the first call is as slow as a virtual qcall on its first call but as fast as a normal qcall on subsequent calls. Tools are the building blocks of the Amiga Foundation Layer and of applications written for the new Amiga. At run time, they are accessed via processes and threads. A process is a set of threads and a thread is a route through the code in a set of tools. This is different from the common UNIX model and care should be taken to understand the difference. In the AFL, processes are very lightweight and memory efficient. Threads can share system resources such as the file table, the signal table, etc., but they are effectively just processes with some additional attributes. In addition, process sovereignty is not enforced; thus processes running on the same processor can access one another's data.

We at Amiga feel that memory protection is an overhead that punishes good code because bad code exists, and this is an attitude that is becoming increasingly prevalent. Indeed, the classic Amiga lasted for 15 years without any memory protection at all and has been praised as one of the most stable systems ever created, a credit to both the system and the developers themselves. Instead of placing this overhead on the system, we intend to push it up to its rightful place; at the code level itself. Protection should be at the language level, giving a choice of safe and unsafe languages. Those who are comfortable writing in unsafe languages such as VP and C should be able to take advantage of the extra speed and features. Those who aren't should use a "safe" language such as Java, or restrict their development to those niches where stability is a secondary consideration Amiga foresees markets for both digital consumers who want stable systems with software that works safely, and pathfinders who like to get down and dirty with the best system possible and consider the occasional instability a price worth paying. We will thus work hard to promote VP as our system level language of choice, but will encourage the use of Java as the safe development language. This does not mean that those who use Java will be restricted to Java. Far from it, in fact. Not only is the Tao Group's J-Engine one of the fastest personal Java implementations in existence but it also offers a very fast and lightweight JNI solution. Thus, all of the functionality written in VP will be made available to the Java developer through Amiga packages. So, for our digital consumer markets, we will closely monitor who is allowed to develop in VP, while offering Java as the safe development environment for all. For our pathfinder markets, users are free to develop and use any applications they like in any language they like with the caveat "Buyer beware!" The advantages of Java are already well documented and, over the last two years, the disadvantages have grown fewer to the point where it is now a very safe, powerful and simple language to use. With the special Amiga packages that we will create to encapsulate our system level functionality, Java developers no longer have to be the poor relatives.

The advantages of VP should hopefully be apparent. While it is an assembler level language it is far more than just assembler, and a good C developer will feel at home in a very short time. Its chief advantages over other languages are simple. It is blazingly fast and it is the language that the Amiga Foundation Layer speaks. There is a learning curve, as there is with any language, but our development team and those who have been kind enough to test the AFL for us have gone from pain to pleasure in a very short time. Of course C is also provided via the GNU compiler but this simply compiles source down into VP Code anyway. Why buy TV dinners when you can cook yourself? We see the C route as being used primarily as a porting and learning path, with developers turbo charging their applications via VP. Since C programs can call VP tools and vice versa, this will become a powerful feature for accessing the AFL functionality as it grows. We at Amiga are as excited as those thousands of developers out there who have been emailing us constantly for the last 4 months asking us when they can get hold of the new Amiga. Indeed, a recent semi official poll on the Be website asking what system developers want to get their hands on next found the new Amiga the clear winner. We are excited because we are all standing together at the beginning of a new path, a new adventure, just as the original Amiga team was in 1985. There is a lot more to do, a lot to build and a lot of work involved but it is also going to be a lot of fun, and we are going to do it together. As a teaser, we are pleased to present the famous but slightly modified Hello World program in VP in such a form that you can code it when you get your SDK home and have it installed on your development machine. The first thing that you will notice is that we are shipping the SDK with the Amiga Foundation Layer running on top of Linux. Why? Ignoring the fatuous answer, "because we can," Linux at the present time provides a far richer driver and applications base for developers. While we are looking forward to Amiga native products appearing soon, we feel it would have been intensely cruel to limit developers to VI and clawhand syndrome.

But why can we? Because the AFL allows for multiple levels of abstraction. Not only can it sit on top of multiple processor families, it can also sit painlessly on top of other operating systems and provide added value to them while taking advantage of features within that host OS. Yes, there is a penalty, and we are looking forward to creating our first pure Amiga system, but as an exercise in demonstrating the flexibility of our foundation layer, it succeeds admirably. A point I should make is that the AFL can run either on its own on top of hardware (the pure version), as an OS on top of another underlying OS with only the AFL visible (dedicated hosted), or as an application on top of another OS (application hosted), in which case you can use both the host OS and the AFL at the same time. For the SDK we are shipping the AFL as an application. I have been using PFE as an editor in Linux to type my programs. The AFL uses a mirrored filing system that allows the Linux filing system to see the Amiga file system and so on. To create my Hello World program, I simply wrote the following VP program and saved it into my developer directory within the Amiga file system. See Figure 1. As you can see we have an include file for the VP macros as the first line. Then we define the tool proper, in a tool/toolend structure. In the tool initiation statement we give the tool location and name, the language it is in (VP), the fact that it is a main tool, the stack size and a global data area size. The next statement defines the entry point, the registers to be passed in and the registers to be passed out. Multiple values can pass in and out, providing a lot of flexibility for the system. Remember that each tool gets its own bank of registers specifically for it, as can subroutines within tools (but that is a more advanced feature). In this example, nothing is passed in or out. Then we have 3 loops and some pretty standard code that should warm the hearts of C developers everywhere. (i0) is the first integer register, and we can have as many as we like. VP also supports register naming, where (i0) could be called (count) for added readability. Finally, a tool must always have at least one exit point as specified by the ret command. With this program saved in the Amiga file system as /demo/example/HelloAmiga.asm, I can flip to the Amiga shell and do the following. See Figure 2. As you can see, most of this is pretty self-explanatory. (-v) on the assembler (asm) gives the verbose mode, and writes out demo/example/HelloAmiga.00. Remember, the (.nn) represents the processor family, with 00 meaning that my tool is targeted at the virtual processor. It's as easy as that. In one fell swoop, I now have a tool I can run on any processor for which there is a VP translator (over 20 and counting). Hopefully this example will mean that anyone buying the SDK will be able to get at least one new Amiga program running. Of course, this isn't the full extent of our developer support program. Gary Peake, our director of developer support is constructing a comprehensive set of resources including web resources, forums and technical books. We encourage all developers to register, take advantage of, and more importantly become a part of the new Amiga developer community.

Fig. 1 .include 'taort' tool 'demo/example/HelloAmiga',vp,f_main,8192,0 ;tool written in vp, main tool, stack size,global data ent -:- for 5,i0 printf "Hello World, Amiga is back !!! (%d)/n",i0 next i0 cpy.i 5,i0 while i0 != 0 printf "Hello World, Amiga is back !!! (%d)/n",i0 dec.i i0 endwhile cpy.i 5,i0 loop breakif i0 == 0 printf "Hello World, Amiga is back !!! (%d)/n",i0 sub 1,i0 endloop ret toolend

Fig. 2 Amiga0.1:/$ Amiga0.1:/$ cd /demo/example$ Amiga0.1:/demo/example$ Amiga0.1:/demo/example$ ls H* HelloAmiga.asm Amiga0.1:/demo/example$ asm v /demo/example/HelloAmiga Elate Assembler 2.93 demo/example//demo/example/HelloAmiga.asm Written 212 bytes to demo/example/HelloAmiga.00 Amiga0.1:/demo/example$ /demo/example/HelloAmiga Hello World, Amiga is back !!! (5) Hello World, Amiga is back !!! (4) Hello World, Amiga is back !!! (3) Hello World, Amiga is back !!! (2) Hello World, Amiga is back !!! (1) Hello World, Amiga is back !!! (5) Hello World, Amiga is back !!! (4) Hello World, Amiga is back !!! (3) Hello World, Amiga is back !!! (2) Hello World, Amiga is back !!! (1) Hello World, Amiga is back !!! (5) Hello World, Amiga is back !!! (4) Hello World, Amiga is back !!! (3) Hello World, Amiga is back !!! (2) Hello World, Amiga is back !!! (1) Amiga0.1:/demo/example$ ls H* HelloAmiga.00 HelloAmiga.asm Amiga0.1:/demo/example$

We know that we won't succeed on our own, or by telling developers what they can and cannot do. We will only succeed in partnership. We are on a new adventure. There is much here already, and more to come: Full audio, streaming, 3D, some great partner announcements and a pure system. In addition, there is our next generation architecture, codenamed Amie, which is being built as we speak and which will, we hope, be an important player in the digital world of tomorrow. We look forwards to working with you, playing with you, and smiling with you. Fleecy Moss (VP Technology) on behalf of the whole Amiga Team, Amiga Inc, Snoqualmie, Washington USA