Java Virtual Machine Security

This chapter will answer the following questions:

 


Overview

Before exploring the Java Virtual Machine, we will explain some of the terminology used in this chapter. First, the Java Virtual Machine (JVM) is the environment in which Java programs execute. It essentially defines an abstract computer, and specifies the instructions that this computer can execute. These instructions are called bytecodes. Generally speaking, Java bytecodes are to the JVM what an instruction set is to a CPU. A bytecode is a byte-long instruction that the Java compiler generates, and the Java interpreter executes. When the compiler compiles a .java file, it produces a series of bytecodes and stores them in a .class file. The Java interpreter can then execute the bytecodes stored in the .class file.

Other terminology used in this chapter involves Java applications and applets. It is sometimes appropriate to distinguish between a Java application and a Java applet. In some sections of this chapter, however, that distinction is inappropriate. In such cases, we will use the word app to refer to both Java applications and Java applets.

Finally, it is important to clarify what the word Java really stands for. Java is more than just a computer language; it is a computer environment. This is because Java stands for two inseparable things: the design-time Java (the language itself) and the run-time Java (the JVM). This interpretation of the word Java is a more technical one. Interestingly enough, the practical interpretation of Java is that it stands for the run-time environment-not the language. When you say something like "this machine can run Java," what you really mean is that the machine supports the Java run-time environment -more precisely, it implements a Java Virtual Machine.

Why is the Java VM Necessary?

It might seem strange that a truly portable language should need a specific "machine" to run anywhere; in other words, how can Java run on any machine, and yet not be able to run without the JVM? The answer is quite simple: for a language to be truly portable, it must meet the following requirements:

As for the first requirement, Java's language specification is well defined... as it stands, there is only one flavor of Java, and there exists a standard library for it (Sun's JDK). As for the run-time environment requirement that is taken care of by the JVM: the JVM is the run-time environment. That means that having Java programs run under the JVM guarantees a common run-time environment. Even though there are different implementations of the JVM, they all must meet certain requirements to guarantee portability; in other words, whatever differs among the various implementations does not affect portability.

What Are the Main Roles of the JVM?

The JVM is responsible for performing the following functions:

Throughout the remaining chapter, we will focus on the last function: security.


Java VM Security

One of the JVM's most important roles is monitoring the security of Java apps. The JVM uses a specific mechanism to force certain security restrictions on Java apps. This mechanism (or security model) has the following roles:

In the following sections, we will see how these security roles are taken care of in Java.

The Security Model

In this section, we will look at some of the different elements in Java's security model. In particular, we will examine the roles of the Java Verifier, the Security Manager, and the class loader. These are the components that make Java apps secure. In addition, we will see how the Java language specification is an important factor in making Java a secure environment.

The Java Verifier

Every time a class is loaded, it must first go through a verification process. The main role of this verification process is to ensure that each bytecode in the class does not violate the specifications of the Java VM. Examples of bytecode violations are syntactic errors, and overflowed or underflowed arithmetic operations. The verification process is handled by the Java verifier, and it consists of the following four stages:

  1. Verifying the structure of class files.
  2. Performing system-level verifications.
  3. Validation bytecodes.
  4. Performing run-time type and access checks.

The first stage of the verifier is concerned with verifying the structure of the class file. All class files share a common structure; for example, they must always begin with what is called the magic number, whose value is 0xCAFEBABE. Following the magic number, are four bytes representing the minor and major versions of the compiler. At this stage, the verifier also checks that the constant pool is not corrupted (the constant pool is where the class file's strings and numbers are stored). In addition, the verifier makes sure that there are no added bytes at the end of the class file.

The second stage performs system-level verifications. This involves verifying the validity of all references to the constant pool, and ensuring that classes are subclassed properly.

The third stage involves validating the bytecodes. This is the most significant and complex stage in the entire verification process. Validating a bytecode means checking that its type is valid, and that its arguments have the appropriate number and type. The verifier also checks that method calls are passed the correct type and number of arguments, and that each external function returns the proper type.

Finally, the verifier ensures that all variables are initialized correctly.

The final stage is where run-time checks take place. At this stage, externally referenced classes are loaded, and their methods are checked. The method check involves checking that the method calls match the signature of the methods in the external classes. The verifier also monitors access attempts by the currently loaded class to make sure that the class does not violate access restrictions. Another access check is done on variables to ensure that private and protected variables are not accessed illegally. Also, some runtime optimizations are performed at this stage, such as replacing direct references for indirect ones.

From this exhaustive verification process, we can see how important the Java verifier is to the security model. It is also important to note that the verification process must be done at the verifier level, and not at the compiler's, since any compiler can be programmed to generate Java bytecodes. Clearly then, relying on the compiler to perform the verification process is dangerous, since the compiler can be programmed to bypass it. This point illustrates why the JVM is necessary.

The Security Manager

One of the classes defined in the java.lang package is the SecurityManager class. This class is used to define the security policy that specifies certain security restrictions on Java apps. The security policy's main role is to determine access rights. Here's an overview of how this works: every Java app loaded into the JVM exists in its own namespace. An app's namespace defines its access boundary. This means that the app cannot access any resources beyond its namespace. Before an app can access a system resource, such as a local or networked file, the SecurityManager object verifies that the resource is inside the app's namespace. If it is, the SecurityManager object grants the access right; otherwise, it prevents it.

The SecurityManager class contains many methods used to check whether a particular operation is permitted. The checkRead() and checkWrite() methods, for example, check whether the method caller has the right to perform a read or write operation, respectively, to a specified file. The default implementation of all of SecurityManager's methods, assume that the operation is not permitted, and they prevent the operation from taking place by throwing a SecurityException. Many of the methods in the JDK use the SecurityManager before performing dangerous operations.

In order to specify your own security policy, you need to subclass SecurityManager. Once you have your own SecurityManager class, you can use the static System.setSecurityManager() method to load it into the environment. Now, whenever a Java app needs to perform a dangerous operation, it can consult with the SecurityManager object that is loaded into the environment.

The way Java apps use the SecurityManager class is generally the same. An instance of SecurityManager is first created in the following way:

SecurityManager security = System.getSecurityManager();

The System.getSecurityManager() method returns an instance of the currently loaded SecurityManager. If no SecurityManager has been set using the System.setSecurityManager() method, System.getSecurityManager() returns null; otherwise, it returns an instance of the SecurityManager that was loaded into the environment. Now, let's assume that the app wants to check whether it can read a file. It does so as follows:

if (security != null) {
  security.checkRead (fileName);
}

The if statement first checks whether a SecurityManager object exists, then it makes the call to the checkRead() method. If checkRead() does not permit the operation, a SecurityException is thrown and the operation never takes place; otherwise, all goes well.

Note: Keep in mind that although by default all of SecurityManager's methods automatically throw a SecurityException, unless you use System.setSecurityManager() to specify a SecurityManager, attempting to instantiate SecurityManager will always return null that means that the SecurityException will never be thrown and all access operations will be permitted.

The class loader

The class loader works alongside the security manager to monitor the security of Java apps. The main roles of the class loader are summarized below:

Each class is associated with a class loader object. Before a class can be loaded into a certain package, its class loader must check which package the class belongs to. The class loader achieves this by calling SecurityManager.checkPackageDefinition(). Once loaded, the class loader resolves the class, which means that it loads every other class that the class references. Resolving a class involves: verifying that the class has the right to access the classes it references, and ensuring that referenced classes are not loaded from invalid sources.

Java's Safety as a Language

So far, we've seen how the Java verifier, the SecurityManager, and the class loader work to ensure the security of Java apps. In addition to

these, there are other mechanisms not described in this chapter, such as the encryption and signed classes, which add to the security of Java apps. All of these mechanisms end up overshadowing the security of the Java language itself.

There are a number of things that make Java's language specification secure, including:

What About Just-In-Time Compilers?

It is appropriate to include a brief discussion of Just-In-Time (JIT) compilers in this chapter. JIT compilers translate Java bytecodes into native machine instructions to be directly executed by the CPU. This obviously boosts the performance of Java apps. But if native instructions are executed instead of bytecodes, what happens to the verification process mentioned earlier? Actually, the verification process does not change because the Java verifier still verifies the bytecodes before they are translated.


Summary

What was covered in this chapter: