NGWS Runtime Application Domains

This document introduces the concept of an Application Domain and describes how the NGWS runtime uses domains to isolate applications running in the same Win32 process.

© 2000 Microsoft Corporation. All rights reserved.


Introduction to Application Domains

Operating systems and runtimes typically provide some form of isolation between applications running on the system. This isolation is necessary to ensure that code running in one application cannot adversely affect other unrelated applications. In Windows, this isolation has historically been achieved using process boundaries. In this model, a process runs exactly one application and that process scopes the resources that are available for that process to use. For example, memory addresses in Win32 are process relative - a pointer in one process is meaningless in the context of another process. The NGWS runtime has similar needs for isolation, but there are many scenarios where the process boundary is too heavy weight. Additionally, the NGWS runtime can rely on the fact that user code can be verified to be type safe to help provide isolation at a lower cost than the process boundary.

The NGWS runtime allows multiple applications to be run in a single Win32 process. A construct called an Application Domain is used to isolate those applications from one another. In many respects, Application Domains are the NGWS runtime equivalent of a Win32 process.

Specifically, isolation means:

  • Applications can be independently stopped and debugged.
  • Code running in one application cannot directly access code or resources from another application. Doing so could introduce a security hole.
  • Faults in one application cannot affect other applications by bringing down the entire process.
  • The application has control over where code loaded on its behave comes from and what version it is. In addition, configuration information is scoped by application.

The primary goal of this document is to describe how the NGWS runtime uses Application Domains to achieve these goals. In addition, the document also describes how domains are created and manipulated by hosts, and how threads, GC heaps, remoting, and class loaders relate to domains.


Application Domain Scenarios

The following examples describe scenarios in which it is useful to run multiple applications in the same process:

  1. ASP+ runs multiple web applications in the same process. In ASP/IIS, application isolation was achieved by process boundaries. This proved too expensive to scale appropriately.
  2. IE runs code from multiple sites in the same process as the browser code itself. Obviously code from one site should not be able to affect code from another site.
  3. In the future, it is likely that productivity apps like word processors and spreadsheets could run macros written by users in their processes.
  4. Database engines need to run code from multiple user applications in the same process.
  5. Application server products may need to run code from multiple applications in a single process.


NGWS Runtime Hosts

The NGWS runtime is typically started and managed by environments like ASP+, IE or the Windows Shell. These hosting environments need to run managed code on behalf of the user and take advantage of the application isolation features provided by Application Domains. In fact it is the host that determines where the Application Domain boundaries lie and in what domain user code gets run in.

The NGWS runtime provides a set of classes and interfaces used by hosts to create and manage Application Domains. See the specification entitled Hosting Interfaces for a description of how to use these classes and interfaces.

Examples of current NGWS runtime hosts include:

ASP+

ASP+ creates Application Domains to run user code. Domains are created per application as defined by the web server.

IE

By default, IE creates an application domain per site. However, this behavior can be customized by the developer.

Shell EXE

Each application that is launched from the command line runs in a separate app domain.

VBA

VBA runs the script code contained in an Office document in an AppDomain

WFC Forms Designer

The WFC Forms Designer places each form the user is building in a separate AppDomain. When the user edits the form and rebuilds, WFC shuts down the old domain, recompiles the code and runs it in a new domain.

SQL Server

SQL Server allows user-provided functions to be written in any NGWS runtime language. These functions can then be called from SQL statements. In addition, SQL generates and runs IL to evaluate expressions in SQL statements.


Application Domains and Isolation

This section describes how the NGWS runtime uses Application Domains to isolate applications as described in the overview.


Cross Domain References

As described above, isolation requires that code running in one application cannot directly access code or resources from another application. One way the NGWS runtime enforces this is to not allow direct calls between objects in different Application Domains.

Objects that pass between domains are either copied or accessed via proxy. The mechanism for determining how objects are access across domains is determined by the agility of the object. There are three cases:

  • Unbound objects are marshaled by value across domains. That is, a copy is made and the receiver manipulates the copy, not the original object.
  • AppDomain bound objects are marshaled by reference across domains. As such, cross domain access is always through proxies.
  • Context-bound objects are also marshaled by reference across domains. In fact, they are marshaled by reference between contexts in the same domain as well.


Fault Isolation

The NGWS runtime relies on the fact that code is verifiably type safe to provide fault isolation between domains. By relying on the type safe nature of the code, application domains provide fault isolation at a much lower cost than the process isolation used in Win32. Because isolation is based on static type verification, there is no need for hardware ring transitions or process switches.

Preventing direct access between domains is also required for security reasons. If a caller in one domain could compromise code in another domain, the potential exists to maliciously breech security.


Unloading and Threads

Application Domains represent the unit of code unloading in the NGWS runtime. We do not provide for unloading of code at a more granular level than a domain, such as a class or an assembly.

Each domain is a process can be unloaded independently of the others using the Unload method on System.AppDomain.

A new domain runs on the same thread as the code that created it. There is not a one to one correspondence between threads and domains. Web server scenarios can require hundreds of domains per process, whereby the ideal number of threads is some small multiple of the number of processors on the machine.

At any given time, every thread is executing in one domain. The NGWS runtime keeps track of which threads are running in which domains. This information, coupled with the fact that direct references are not allowed between domains, allows the NGWS runtime to provide clean shutdown. When a shutdown request is made, the runtime unwinds the thread until it gets to the domain boundary. At this point, it can "detach" all proxies to objects in other domains.


Debugging

Domains can also be debugged independently. Stopping at a breakpoint in one domain will not affect the other domains in the process. NGWS SDK debuggers provide a list of currently running domains and allow the developer to choose which one to debug.


Scoping Application Behavior

The preceding descriptions of isolation have centered on preventing applications that are already running from seeing or manipulating each other's memory space or other resources. An equally important aspect of isolation is the ability to allow an application to control where the NGWS runtime finds code to load on its behalf. This is important to prevent code placed on the machine by one application from inadvertently affecting other applications.

The ability to scope resolution requests by application differs significantly from the way the current platform works. In Windows today, the resolution scope is effectively "the machine" in that any application can use any code as long as its described in the registry or placed in a well known location like the Windows system directory. In fact, sharing in this manner is the default - there is no way for an application to explicitly describe its pieces, their dependencies, and where to find them. This behavior of default sharing leads to the dll conflicts that user's of our platform experience today.

In addition to scoping requests to load code, it is also important to scope configuration information to an application. For many configuration settings, this isn't possible today. For example, consider the case of configuring the remote machine on which to run an NGWS class. Setting the RemoteServerName key in the registry for a particular class affects all applications that use that class. As with unintended sharing or code, the unintended sharing of configuration data prevents an application from having complete control over its own behavior.

A key goal of the deployment features in the NGWS runtime is to provide the form of application scoping described above. To provide this form of isolation, each assembly must be able to explicitly express dependencies on the other assemblies it uses. This dependency information must include version requirements so the NGWS runtime can load the appropriate version of the dependency. This information is captured in the assembly's manifest. For all references made from the current assembly, the information in the manifest is used to govern how the reference is resolved. There are three cases of interest:

  • A requestor in the current assembly could make a request for another item in the same assembly. In this case, the manifest provides the location of the bits to load.
  • A requestor in the current assembly could make a request for an item in another assembly. In this case, the current assembly supplies binding rules, including version requirements, used by the NGWS runtime to locate the referenced assembly, but defers to that assembly to supply the rules for how to locate the bits to load.
  • A request comes in from outside the current assembly for an item that the assembly has exported. In this case, the manifest provides the location of the bits to load.

While the information in the assembly manifest describes the versions of the assembly's dependencies, it doesn't provide any explicit information about how to locate the assembly. Specifically, manifests do not record dependencies between assembly in terms of paths or filenames. The information about where to look for assemblies can be controlled by the Application Domain. In addition, the Application Domain also provides the configuration information to be used by code running in that domain.

More accurately, search rules and configuration data is provided by the host when the application domain is created. The host is responsible for supplying this data because the meaningful values vary by scenario. For example, the configuration data for ASP+ applications is stored per application, site and machine, while the configuration data for an exe is stored per application, user and machine. Only the host knows the specifics of the configuration data for their particular scenario.

In terms of searching information, the host provides a search path when creating the Application Domain. This path is used to find dependent assemblies that are deployed in subdirectories underneath the application. These assemblies are typically private assemblies, but shared assemblies can be deployed here as well. Note that for shared assemblies, the NGWS runtime will always look in the global assembly cache as well as on the search path.

In addition, the configuration information supplied with the domain may provide explicit information about where to find particular assemblies. The path is supplied using the appropriate parameter to AppDomain::CreateDomain:

public static AppDomain CreateDomain(
     string friendlyName,
     IIdentity[] securityInfo,
     ContextPolicy[] policies,     // optional
     string AppRelativeSearchPath, // optional
     boolean ShadowCopyFiles,      // optional
     long SnapshotID);

The combination of the dependency information captured as part of each assembly manifest and the searching and configuration data maintained by the domain, provide the information that the NGWS runtime uses to properly scopes requests to load code and to access configuration data to a particular application. The following diagram summaries where each piece of information is stored in a running domain:


Application Domains and Security

Hosts can affect how the the NGWS security system assigns permissions to code running inside the domain. Specifically, hosts can supply domain specific code access security policy that is evaluated by the security system along with the per-user and machine wide security policy. Also, hosts can play a role in supplying the security system with evidence about code that is loaded into the domain.

The details about how security works with Application Domains are described in the Hosting and Loading Code specification.


AppDomains, Type Identity, and Shared Assemblies

By default, a type is loaded into the same domain as its caller. This means that if the same assembly is referenced by code in two domains it will be jit'd twice and each domain will maintain separate copies of all the loader and EE data structures used to describe the class. In addition, these types have different runtime identities. TypeA loaded in DomainX and TypeA loaded in DomainY are not equivalent (the == operator will fail).

There are cases where the extra memory overhead introduced by this scenario is unacceptable. This is often the case for large assemblies that are typically shared by all the domains in a process. Examples of such assemblies are generally "system" assemblies like the Base Class Libraries, NGWS runtime, the ASP+ libraries, and so on. One could also imagine an ISV wanting to load an assembly that is shared across a family of products this way.

To address these issues, the NGWS runtime supports the notion of "shared assemblies". When a type from a shared assembly is loaded the first time, it is automatically mapped into all Domains instead of being loaded into each domain separately. As a result, shared assemblies consume fewer resources if multiple domains refer to them. Shared assemblies also speed up the creation of domains since the NGWS runtime doesn't need to re-jit the code or build multiple copies of the EE data structures used to represent the types. Shared assemblies are not unloaded until the runtime leaves the process.

While shared assemblies do reduce memory usage, they do result in slightly larger jitted code. In addition, since statics are scoped by domain, references to statics must be indirected when the assembly is shared instead of being accessed by direct memory reference. As a result, access to statics will be slower when the assembly is loaded shared than when it is not.

Assemblies can be loaded as “shared” by setting a flag on the calls to AppDomain::Load and Assembly::Load.


Assembly Overview

This document provides an overview of assemblies in the NGWS runtime. Before describing what an assembly is, we start by describing the two primary problems assemblies are designed to solve. Next, we define what assemblies are and how they are implemented. The remainder of the document describes how assemblies are used at design time and at runtime to scope types and resources, express and enforce version contracts, and contribute to building side-by-side, reusable, shared components.

The two primary problems assemblies are designed to solve are:

  • The versioning problem
  • The Complexities of application installation


The Versioning Problem

The versioning problems encountered with Windows applications today exist because there is no way for an application developer to express versioning rules between pieces of an application and have those rules honored and enforced by the system. Instead, our current approach to versioning is based upon developers following "common-sense" coding practices. These practices state that interface definitions are static once published, and that a single piece of code must maintain backward compatibility with previous versions. Furthermore, code is typically designed so that only a single version of it can be present and executing on a machine at any given time.

The versioning problem is further complicated by the fact that there is no way to maintain consistency between the set of components that were built together and the set that will be present at run time.

The manifestation of these versioning problems is one aspect of what we refer to as DLL Hell. Specifically, installing one application can inadvertently break an existing application because a certain software component was installed that was not fully backward compatible with a previous version. Once this situation occurs, there is no support in the system itself for diagnosing and fixing the problem.


Application Installation Complexities

Installing Windows applications today is a multi-step process. In the typical case, installing an application involves copying a number of software components to the disk and making a series of registry entries that describe those components to the system. The separation between the entries in the Registry and the files on disk makes it very difficult to determine exactly what needs to be removed to uninstall an application.

In addition, the relationship between the various entries required to fully describe a COM class in the registry is very loose. These entries often include entries for coclasses, interfaces, typelibs and DCOM app ids, not to mention any entries made to register document extensions or component categories. Often times these entries require manual inspection to verify consistency.

Finally, a registry footprint is required to activate any COM class. This drastically complicates the process of deploying distributed applications because each client machine must be touched to make the appropriate registry entries.


Assemblies Defined

An assembly is a reusable, versionable, self-describing deployment unit for types and resources – it is the primary building block of a NGWS application. Just as we gain benefits from having self-describing types in the NGWS runtime, we also gain benefits from self-describing assemblies. Assemblies build on the benefits of self-describing types by providing the infrastructure we need to allow the Runtime to fully understand the contents of an application and to enforce the versioning and dependency rules defined by the application.

An assembly consists of two logical pieces: the set of types and resources that form some logical unit of functionality and metadata that describes how these elements relate and what they depend on to work properly. The metadata that describes an assembly is called a manifest. Manifests are physically represented using the same metadata technology used to describe components. The only difference is the schema used to represent the information.

The following information is captured in an assembly manifest:

  • Identity. An assembly's identity includes it’s simple textual name name, a version number, an optional culture if the assembly contains localized resources, and an optional public key used to guarantee name uniqueness and to “protect” the name from unwanted reuse (see the later description of shared names).
  • Contents. Assemblies contain types and resources. The manifest lists the names of all the types and resources that are visible outside the assembly, along with information about where they can be found within the assembly.
  • Dependencies. Each assembly explicitly describes other assemblies that it is dependent upon. Included in this dependency information is the version of each dependency that was present when the manifest was built and tested. In this way we record a “known good” configuration that we can revert to in the face of failures do to version mismatches.
  • Requested Permissions. As an assembly is being built, the assembly author records the set of permissions that the assembly requires to run. This set of requested permissions is stored in the assembly manifest.

In the simplest case, an assembly is a single dll. This dll contains the code, resources, type metadata and assembly metadata (manifest). In the more general case, however, assemblies consist of a number of files. In this case, the assembly manifest either exists as a standalone file, or is contained in one of the PE files that contain types and/or resources. The following picture shows an assembly called 401k that consists of two files. One file contains the manifest and all the types, while the second file contains the assembly's resources. This particular assembly depends on one other assembly called Tax.


Private and Shared Assemblies

There are two “kinds” of assemblies: private assemblies and shared assemblies. Private and shared assemblies are the same structurally and there is really no difference in the types of functionality they can provide. Instead there are differences in how these assemblies are named, the locations to which they can be deployed, the level of version checking done when they are loaded and how they are referenced.

A private assembly is intended to be used only within one application. Private assemblies are typically authored by the same group or organization that authors the application in which it is used. As a result, the application author has a high degree of control over the naming and versioning of the assembly. In fact, private assemblies must be deployed in the same directory as the application or a subdirectory thereof. This allows the NGWS runtime to enforce that no other application can reference this assembly.

Private assemblies have relatively simple naming requirements – their name just has to be unique within the application and concerns about name spoofing are limited. Private assemblies have what we call a private name, which is simply a descriptive text string that makes up the assembly’s identity along with version and culture.

Because private assemblies are under the control of the application author, and are deployed along with the application, the NGWS runtime does not enforce any versioning or naming rules.

In contrast, shared assemblies are meant to be used by several applications on the machine. Typically a shared assembly is authored by someone other than the author of the application that uses it. Examples of shared assemblies include a UI grid control that is used by several applications or system class libraries like those shipped by Microsoft or other library vendor. The naming requirements for shared assemblies are much more stringent than they are for private assemblies. For example, the name of a shared assembly must be globally unique. In addition, the author of the assembly needs some protection of the name. That is, they need to be guaranteed that a third party can’t release a subsequent version of an assembly they’ve produced and have it picked up by users unknowingly. The NGWS runtime supports these naming requirements through a feature called shared names. Shared names use cryptographic key pairs to ensure name uniqueness and to prevent name spoofing. In addition, shared names also provide the caller of the assembly with information about the identity about the publisher of the assembly. Callers can use this information to ensure that the assembly they use at runtime comes from the same publisher that supplied the assembly they compiled against. More details about shared names can be found in the Shared Name Spec.

Shared assemblies are typically deployed to the global assembly cache. The assembly cache provides for side by side storage of multiple versions of an assembly and provides performance benefits in terms of lookup and code sharing. However, not all shared assemblies must be deployed in the assembly cache. There are several scenarios where an application may want to deploy a shared assembly in its application directory to ensure that version gets loaded.

Unlike references to private assemblies, references to shared assemblies include version checking and the application of version policy as described throughout this document.


Building Assemblies

Typically, the assembly manifest is emitted as part of the build process along with the types, resources and type metadata. By default the output of each Visual Studio compiler is an assembly.

However, there are situations in which a user wishes to build individual modules and decide later how these modules should be grouped into assemblies. To support this model of development, each VS compiler provides command line switches to emit modules. We then ship a tool in the SDK called Alink that can be used to generate an assembly manifest for a set of dll’s, exe’s and resource files.

There are two sets of interfaces that can be used to emit assembly manifests: the unmanaged metadata apis (IMetadataAssemblyEmit) and the set of types in the System.Reflection.Emit namespace can also be used to generate assemblies. The VS compilers and several third party compilers use the unmanaged metadata interface to emit manifests. Tools like managed script engines commonly use the types in System.Reflection.Emit to build assemblies on the fly.


Assemblies are not:

To further clarify the role of assemblies in the NGWS runtime, this section provides some examples of problems assemblies explicitly DO NOT address.

  • Packaging and Deployment Technology. A common misperception is that assemblies are a packaging format or a deployment technology. However, assemblies are not a packaging or distribution technology in and of themselves. Instead, assemblies can be packaged for deployment in a variety of ways. As mentioned, an assembly's files are represented as NGWS PE's and resource files. These files can be distributed as is, or they can be packaged into some other distribution unit. For the first release of the NGWS runtime the additional distribution formats are .cab, and .msi.
  • Name Lookup Service or Registration Service. Assemblies do not provide a lookup service for locating NGWS classes, or a registration service that describes where all classes can be found.


Assemblies in NGWS

This section describes how assemblies are used by the NGWS runtime at runtime.

Assemblies are core to the NGWS runtime's ability to locate and securely execute code. In particular, assemblies are used by the NGWS runtime to:

  • Provide a scope for type resolution
  • Locate types to load
  • Establish security identity and the set of requested permissions for code.
  • Establish a visibility boundary for types


Type Resolution

Assemblies are the construct used by the NGWS runtime to locate and load types and resources. Just as type metadata removes the need to look in the Registry for the description of a type, the assembly metadata removes the need to look in the Registry to locate the type.

The assembly metadata combined with type metadata provides all the information the NGWS runtime needs to find and load types. Each time a type is loaded, the manifest for the assembly containing the type is loaded as well (unless it was loaded before, of course). The manifest is then used to resolve all references to types made from within the assembly. References to types will always resolve to one of two places: the same assembly as the caller, or in a dependent assembly. In both cases, the manifest provides the information needed for the NGWS runtime to resolve the reference.

Because the NGWS runtime class loader relies on the manifest, the assembly establishes a resolution scope for types. In classic-COM, the resolution scope was the registry in that a caller could activate any type that was visible in the Registry. In contrast, the NGWS runtime uses the assembly to scope the set of types that can be referenced at runtime. The target type must either be in the same assembly as the caller or in dependent assembly. As mentioned, information about all dependencies, including the version of the dependency the assembly was built against, are fully described in the manifest.


Security Identity and Requested Permissions

The assembly is the unit that the NGWS security system uses to determine the set of permissions that code is granted at runtime. As an assembly is being built, the assembly author records the set of permissions that the assembly requires to run. For example, if the developer writes code that accesses the disk, they must request “File IO Permission”. The set of requested permissions are stored in the assembly manifest.

At runtime, evidence about the assembly is used as input to security policy. Security policy is typically established by the administrator and is used to determine the set of permissions that are granted to the code within the assembly. Examples of evidence in V1 include the publisher of the assembly if it is signed, the web site and zone (in Internet Explorer terms) the assembly was downloaded from, or the assembly’s name if that name has been secured with a key pair (termed a “shared name”).

For example, an administrator may establish security policy that says all code downloaded from the internet and signed by “Joe’s Software” cannot access my disk drive, but may display user interface on my machine.

To allow policy to be based on either Authenticode signatures or shared names, the the NGWS runtime loader must be able to guarantee that the contents of the assembly has not been changed since it was built. That is, we must be able to guarantee the integrity of each assembly we run. We do this through hash verification: the assembly manifest contains a list of all files that make up the assembly, including a hash each file as it existed when the manifest was built. As each file is loaded at runtime, its contents are hashed and compared with the hash value stored in the manifest. If the two hashes don’t match, the assembly fails to load.


Type Visibility

Assemblies establish a boundary for the visibility of types. As part of its declaration, a type states whether it is visible to callers outside its assembly, or whether visibility is restricted only to callers within the same assembly. These visibility rules are enforced at runtime.


Solving the Versioning Problem

The preceding sections of this document have defined assemblies and described how they are used by the NGWS runtime. This section describes how the assembly concept is used to solve the versioning problem described earlier.

To solve the versioning problem, the NGWS runtime must:

  • Allow developers to specify version rules between different software components and have these rules enforced at runtime,
  • Provide the infrastructure to allow multiple versions of a software component to run simultaneously, and
  • Isolate running applications from others running in the same process.


Recording and Enforcing Version Policy

Version checking in the NGWS runtime is done at the assembly level.

As mentioned, a version number is part of each assembly’s identity and each assembly manifest includes information about all dependencies, including version numbers

The dependency information in the manifest records the “known good” configuration. That is, the versions of the assemblies that were built and tested together. By default, this same set of assemblies will run together after they are deployed, subject to QFE policy. However, we provide a set of version policies that can be set to alter which version of an assembly the NGWS runtime will load at Runtime. These version policies are expressed in xml configuration files that are associated either directly with the application, with a particular user, or the machine as a whole. The NGWS runtime uses the version information recorded in the manifest and any policy expressed in the configuration files to determine which version of a dependency to load.

One version policy of particular interest is what we call safe-mode. This policy is used to cause the runtime to bind to the exact versions of its dependencies that are recorded in its manifest (i.e all four parts of the version number is matched exactly). Safe-mode is used to return to the “known good” configuration that was recorded at build time. This can be used to ensure consistency between development and runtime and to revert to a know working state in the event that an update of a shared assemblies inadvertently broke an application.


Side by Side Execution

The ability to run multiple versions of the same assembly simultaneously is referred to as side-by-side execution. The NGWS runtime provides the infrastructure to allow multiple versions of the same assembly to be run on the same machine, or even in the same process!

Supporting side-by-side execution is the basis for the robust versioning in the NGWS runtime. Code that is side-by-side capable has more flexibility it terms of providing compatibility with previous versions. Code that cannot run side-by-side must remain completely backward compatible with all previous versions because only one version (typically the latest) can be running on the system at the same time. Given the robust versioning support in NGWS, side-by-side components do not need to maintain strict backwards compatibility. Consider the situation where ClassX supports side-by-side and makes an incompatible change between version 1 and version 2. Callers of ClassX that expressed a dependency on version 1 will always get version 1 regardless of how many later versions of ClassX are installed on the machine. The caller would only get version 2 if it specifically "upgraded" using version policy.

Writing code that is capable of running side by side obviously requires some awareness on the part of the developer. For example, side by side components can not use shared state without some form of synchronization.


Isolation and Application Domains

The terms assembly and Application Domain are often confused. While assemblies are used as the basis for type resolution and versioning, Application Domains are the basis for application isolation in the NGWS runtime. Application Domains are a construct internal to the NGWS runtime that is used to isolate applications running in the same process from each other.

There are at least two aspects to this isolation. First, we must prevent running applications from seeing or manipulating each other's memory space or other resources. Faults in one application cannot affect other applications by bringing down the entire process. An equally important aspect of isolation is the ability to allow an application to control where the NGWS runtime finds code to load on its behalf. This is important to prevent code placed on the machine by one application from inadvertently affecting other applications.


Simplifying Application Installation: Deployment and Configuration

This section describes how assemblies are used to simplify the process of deploying and installing an application.

Perhaps the biggest single advantage to be gained from self-describing types and self-describing applications is ease of deployment. Rather than requiring complex registry entries, deploying a NGWS application can be as simple as copying the files that constitute the application to a directory structure on disk. The NGWS runtime can launch the application and resolve all references directly from the file system. Because each assembly describes where to locate the classes it contains, there is no need to make registry or Active Directory entries of any kind.


Locating Assemblies

Of course, in order to find a type within an assembly, the NGWS runtime must be able to find the assembly itself. Without a global registry that can be used to find all assemblies, we must establish a set of default locations and search rules that we use to find assemblies. These search rules can be summarized as follows:

  1. The xml configuration files can be used to specify an explicit path, or codebase, to an assembly. If this information is provided, there is no need for the NGWS runtime to search – we just load the assembly directly from the supplied codebase.
  2. If no codebase is supplied, the NGWS runtime looks on the file system in the application directory for the assembly. A set of search rules are applied, based on the name, version and culture of the assembly.
  3. The final place the NGWS runtime will look is in the global assembly store. Each machine has a store used to hold assemblies that are meant to be shared by multiple applications on the machine.


Packaging Assemblies

As mentioned in section 4 above, assemblies are not a packaging format or a deployment technology. Instead, assemblies can be packaged for deployment in a variety of ways. Perhaps the most common distribution format is the assembly files themselves – the dll’s, exe’s and resource files that make up the assembly. However, there are situations where more specialized packaging may be required. In V1, we support two such additional packaging formats: cab files and Windows Installer files. Cab files are typically used in web download scenarios to gain the benefits of file compression for faster download. Windows Installer files are used to distribute your application with the Windows Installer.


Configuration

During deployment, an administrator can add configuration details to an assembly. While the assembly manifest describes the assembly at the time it was built, certain types of information cannot be supplied until the application is deployed. For example, information regarding physical deployment locations and remoting information is not known when the application is built. In addition, the administrator may want to override information kept in the manifest. For example, an administrator may want to apply specific version policies to control which version of a given assembly is loaded by the NGWS runtime.

Configuration information is supplied in xml configuration files that are either associated directly with the application, with a specific user, or with the entire machine.


Uninstall and Replication

This section has primarily focused on how assemblies reduce the complexity of installing applications. However, an equally important aspect of assemblies is their ability to help with uninstall and replication. In several scenarios, uninstalling an application is as simple as deleting a directory from the disk. Also, in many cases, an application can be replicated by xcopy’ing a directory from one machine to another. Of course, if the application uses a more sophisticated install technology like the Windows Installer, simply deleting the directory will not completely remove the application.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Asynchronous Programming Model

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Asynchronous Programming Model Concepts

This section describes the Asynchronous Programming Model concepts.


Introduction to Asynchronous Programming Model

Asynchronous programming is a feature supported by many areas of NGWS, namely:

  • File IO, Stream IO, Socket IO
  • Networking: HTTP, TCP
  • Remoting Channels: HTTP, TCP
  • ASP+, ASP+ WebServices,
  • Messaging Message Queues over MSMQ
  • Delegates

As a core concept, the NGWS frameworks and runtime provide an important common design pattern to avoid confusion among developers. This document details this design pattern and demonstrates the pattern using Async delegates.

The basic ideas behind this pattern are as follows:

  1. It is the caller that decides if a particular call should be asynchronous.
  2. It is not necessary for a called object to do additional programming for supporting asynchronous behavior by its clients. The NGWS runtime infrastructure should be able to handle the difference between the caller and called object views.
  3. The called object can choose to explicitly support asynchronous behavior either because it can implement it more efficiently than a general architecture or it wants to support only asynchronous behavior by its callers. However, it is recommended that such called objects follow the asynchronous design pattern for exposing asynchronous operations.
  4. Type safety. For Async delegates, the compiler generates type safe method signatures for BeginInvoke and EndInvoke.
  5. The NGWS frameworks and runtime provide services needed for supporting asynchronous programming model. An example partial list of such services is:
    • Synchronization primitives such as Monitors and Reader Writer Locks.
    • Threads and Thread pools.
    • Synchronization constructs such as containers that support Waiting on objects.
    • Exposure to the underlying infrastructure pieces such as IMessage objects and Thread Pools.


Asynchronous Programming Model Design Pattern

As an example, consider the class in the following section.

Example Class

This class factorizes a number into its two prime factors.

// Class the factorizers the number
public class PrimeFactorizer
{
   public bool Factorize(
                int factorizableNum,
                ref int primefactor1,
                ref int primefactor2)
   {
      primefactor1 = 1;
      primefactor2 = factorizableNum;

      // Factorize using a low tech approach
      for (int i=2;i<factorizableNum;i++)
      {
         if (0 == (factorizableNum % i))
         {
            primefactor1 = i;
            primefactor2 = factorizableNum / i;
            break;
         }
      }

      if (1 == primefactor1 )
         return false;
      else
         return true   ;
   }
}

Caller defines a delegate for the method

The following code snippet shows the caller defining a pattern for the invoking Factorize method asynchronously:

// Define the delegate
public delegate bool FactorizingCallback(
         int factorizableNum,
         ref int primefactor1,
         ref int primefactor2);

// Creating an instance of the Factorizer
PrimeFactorizer pf = new PrimeFactorizer();

// Creating a delegate on the Factorize method on the Factorizer
FactorizingCallback fd = new FactorizingCallback(pf.Factorize);

Compiler and NGWS Runtime support

When the compiler emits the FactorizingCallback delegate class after parsing its definition at the first line above, it will generate the BeginInvoke and EndInvoke methods in addition to the Invoke method.

class FactorizingCallback : delegate
{
   public bool Invoke(
         int factorizableNum,
         ref int primefactor1,
         ref int primefactor2);

   // Supplied by the compiler
   public IAsyncResult BeginInvoke(
         int factorizableNum,
         ref int primefactor1,
         ref int primefactor2,
         AsyncCallback cb,
         Object AsyncObject
         );

   // Supplied by the compiler
   public bool EndInvoke(
         ref int primefactor1,
         ref int primefactor2,
         IAsyncResult ar);
}

Client side programming model

The following code snippet demonstrates the client-side programming model for invoking the Factorize method asynchronously.

Example

The class receives the results of the Async call on a callback.

public delegate bool FactorizingCallback(
         int factorizableNum,
         ref int primefactor1,
         ref int primefactor2);

// Class that receives a callback when the the results are available
public class ProcessFactorizedNumber
{
   private int _ulNumber;

   public ProcessFactorizedNumber(int number)
   {
      _ulNumber = number;
   }

   // Note the qualifier one-way.
   [OneWayAttribute()]
   public void FactorizedResults(IAsyncResult ar)
   {
      int factor1=0, factor2=0;

      // Extract the delegate from the AsyncResult
      FactorizingCallback fd = (FactorizingCallback)ar.AsyncObject;

      // Obtain the result
      fd.EndInvoke(ref factor1, ref factor2, ar);

      // Output results
      Console.WriteLine("On CallBack: Factors of {0} : {1} {2}",
                    _ulNumber, factor1, factor2);
   }
}

Async Variation 1 – call

// Async Variation 1
// The ProcessFactorizedNumber.FactorizedResults callback
// is called when the call completes.
public void FactorizeNumber1()
{
   // Client code
   PrimeFactorizer pf = new PrimeFactorizer();
   FactorizingCallback fd = new FactorizingCallback(pf.Factorize);

   // Async Variation 1
   int factorizableNum = 1000589023, temp=0;

   // Create an instance of the class which is going
   // to called when the call completes
   ProcessFactorizedNumber fc = new ProcessFactorizedNumber(factorizableNum);

   // Define the AsyncCallback delegate
   AsyncCallback cb = new AsyncCallback(fc.FactorizedResults);

   // Can stuff any object as the state object
   Object state = new Object();

   // Asynchronously invoke the Factorize method on pf
   IAsyncResult ar = fd.BeginInvoke(
                        factorizableNum,
                        ref temp,
                        ref temp,
                        cb,
                        state);

   //
   // Do some other useful work
   //. . .
}

Async Variation 2

// Async Variation 2
// Waits for the result
public void FactorizeNumber2()
{
   // Client code
   PrimeFactorizer pf = new PrimeFactorizer();
   FactorizingCallback fd = new FactorizingCallback(pf.Factorize);

   // Async Variation 1
   int factorizableNum = 1000589023, temp=0;

   // Create an instance of the class which is going
   // to called when the call completes
   ProcessFactorizedNumber fc = new ProcessFactorizedNumber(factorizableNum);

   // Define the AsyncCallback delegate
   AsyncCallback cb =
   new AsyncCallback(fc.FactorizedResults);

   // Can stuff any object as the state object
   Object state = new Object();

   // Asynchronously invoke the Factorize method on pf
   IAsyncResult ar = fd.BeginInvoke(
                     factorizableNum,
                     ref temp,
                     ref temp,
                     null,
                     null);

   ar.AsyncWaitHandle.WaitOne(10000, false);

   if (ar.IsCompleted)
   {
      int factor1=0, factor2=0;

      // Obtain the result
      fd.EndInvoke(ref factor1, ref factor2, ar);

      // Output results

      Console.WriteLine("Sequencial : Factors of {0} : {1} {2}",
                    factorizableNum, factor1, factor2);

   }
}

Note that if the ProcessFactorizedNumber is a context-bound class that requires synchronized / thread- affinity context, the callback is dispatched through the context dispatcher infrastructure. In other words, the callback itself may execute asynchronously with respect to its caller for such contexts. That is precisely the semantics of the one-way qualifier on method signatures. It means that any such method call may execute synchronously or asynchronously with respect to caller and the caller cannot make any assumptions about completion of such a call when execution control returns to it.

Also, calling EndInvoke before the asynchronous operation is complete will block the caller. Calling it second time with the same IAsyncResult is undefined.

Summary of Asynchronous Design Pattern

The server splits asynchronous operation into its two logical parts: the part that takes input from the client and kicks off the asynchronous operation and the part that supplies results of the asynchronous operation to the client. In addition to the input needed for the async operation, the first part also takes an AsyncCallback delegate to be called when the asynchronous operation is completed. The first part returns a waitable object that implements IAsyncResult interface used by the client to determine the status of the asynchronous operation. The Server typically also utilizes the waitable object it returned to the client to maintain any state associated with asynchronous operation. The client uses the second part to obtain the results of the asynchronous operation by supplying the waitable object.

Options available to client for initiating asynchronous operations:

  1. Supply the callback delegate when beginning asynchronous
  2. Not supply the callback delegate when beginning asynchronous operation.

Options available to the client for completing asynchronous operations are:

  1. Poll the returned IAsyncResult.IsCompleted property for completion.
  2. Attempt to complete the operation prematurely thereby blocking until the operation completes.
  3. Wait on the IAsyncResult object. The difference between this and previous option is that the client can use timeouts to wake up periodically.
  4. Complete the operation inside the Async callback routine.


Asynchronous Programming Model Implementation Notes

Async calls via delegates

The implementation of the asynchronous methods on the delegate class will utilize the following infrastructure pieces to provide Async calls within the same appdomain and between appdomains (including between processes and machines):

  • IMessage objects inside remoting infrastructure. IMessage objects can be created from method invocation stack and can be converted back to a method invocation stack.
  • Thread pools for efficiently managing concurrency inside the process.
  • Synchronization objects and services for blocking and waking up threads.
  • Context dispatcher infrastructure for efficiently dispatching calls on objects living inside synchronized / thread-affinity contexts.

Delegate BeginInvoke and EndInvoke methods

Compilers should emit delegate classes with Invoke, BeginInvoke and EndInvoke methods using the delegate signature specified by the user. The BeginInvoke and EndInvoke should be decorated as native. Since these methods are marked as native the implementation is automatically provided by the NGWS runtime at class load time. The loader ensures that they are not overridden.

Delegate Parameters

For example if we take the Factorizing delegate.

class FactorizingCallback : delegate
{
   public bool Invoke(
         int factorizableNum,
         ref int primefactor1,
         ref int primefactor2);

   // Supplied by the compiler
   public IAsyncResult BeginInvoke(
         int factorizableNum,
         ref int primefactor1,
         ref int primefactor2,
         AsyncCallback cb,
         Object AsyncObject
         );

   // Supplied by the compiler
   public bool EndInvoke(
         ref int primefactor1,
         ref int primefactor2,
         IAsyncResult ar);
}

BeginInvoke:

  • Include all IN parameters
  • Include all IN/OUT parameters
  • Include all ByRef parameters
  • Take AsyncCallback and AsyncObject as last two parameters. The AsyncObject available via the AsyncObject property on the IAsyncResult interface
  • Return IAsyncResult.

EndInvoke:

  • Include all IN/OUT parameters
  • Include all OUT parameters
  • Include all Byref parameters
  • Take IAsyncResult as last parameter.
  • Return the original return type from the original method signature

Cancel

Cancel is not provided on the IAsyncResult interface. Cancel is not provided on IAsyncResult since in many implementations there can be no guarentee’s on canceling the BeginXXX call. Exposing and implementing Canceling on an Async BeginXXXX call is up to the implementer of the class when exposes BeginXXXX and EndXXXX. They could choose to provide a CancelXXXX method.

Exceptions

BeginXXXX. If the BeginXXX method throws an exception, then the caller can assume that an asynchronous operation was not started and that the callback delegate will not be called. BeginXXX can throw infrastructure exceptions.

EndXXXX. Exceptions are returned from asynchronous operations by throwing them from the EndXXXX method. EndXXX can throw infrastructure and called object exceptions.

Delegates and Contexts

Delegates can also be bound to particular context. When the delegate is call it will route the call to the correct content.

class Delegate

{
    //. . .
    public void BindToContext ();
    //. . .
};
  • Wrap this delegate on an unbound or appdomain object with a new delegate which contains a native stub and return it.
  • When the new delegate is called, it will switch into context in which the new delegate was created and execute the context policies before the real object is called.


Full Async Sample Source

using System;
using System.Threading;
using System.Runtime.Remoting;

// Async delegate
public delegate bool FactorizingCallback(
         int factorizableNum,
         ref int primefactor1,
         ref int primefactor2);

// Class the factorizers the number
public class PrimeFactorizer
{
   public bool Factorize(
                int factorizableNum,
                ref int primefactor1,
                ref int primefactor2)
   {
      primefactor1 = 1;
      primefactor2 = factorizableNum;

      // Factorize using a low tech approach
      for (int i=2;i<factorizableNum;i++)
      {
         if (0 == (factorizableNum % i))
         {
            primefactor1 = i;
            primefactor2 = factorizableNum / i;
            break;
         }
      }

      if (1 == primefactor1 )
         return false;
      else
         return true   ;
   }
}

// Class that receives a callback when the the results are available
public class ProcessFactorizedNumber
{
   private int _ulNumber;

   public ProcessFactorizedNumber(int number)
   {
      _ulNumber = number;
   }

   // Note the qualifier one-way.
   [OneWayAttribute()]
   public void FactorizedResults(IAsyncResult ar)
   {
      int factor1=0, factor2=0;

      // Extract the delegate from the AsyncResult
      FactorizingCallback fd = (FactorizingCallback)ar.AsyncObject;

      // Obtain the result
      fd.EndInvoke(ref factor1, ref factor2, ar);

      // Output results
      Console.WriteLine("On CallBack: Factors of {0} : {1} {2}",
                    _ulNumber, factor1, factor2);
   }
}

// Class that shows variations of using async
public class Simple
{
   // Async Variation 1
   // The ProcessFactorizedNumber.FactorizedResults callback
   // is called when the call completes.
   public void FactorizeNumber1()
   {
      // Client code
      PrimeFactorizer pf = new PrimeFactorizer();
      FactorizingCallback fd = new FactorizingCallback(pf.Factorize);

      // Async Variation 1
      int factorizableNum = 1000589023, temp=0;

      // Create an instance of the class which is going
      // to called when the call completes
      ProcessFactorizedNumber fc = new ProcessFactorizedNumber(factorizableNum);

      // Define the AsyncCallback delegate
      AsyncCallback cb = new AsyncCallback(fc.FactorizedResults);

      // Can stuff any object as the state object
      Object state = new Object();

      // Asynchronously invoke the Factorize method on pf
      IAsyncResult ar = fd.BeginInvoke(
                           factorizableNum,
                           ref temp,
                           ref temp,
                           cb,
                           state);

      //
      // Do some other useful work
      //. . .
   }

   // Async Variation 2
   // Waits for the result
   public void FactorizeNumber2()
   {
      // Client code
      PrimeFactorizer pf = new PrimeFactorizer();
      FactorizingCallback fd = new FactorizingCallback(pf.Factorize);

      // Async Variation 1
      int factorizableNum = 1000589023, temp=0;

      // Create an instance of the class which is going
      // to called when the call completes
      ProcessFactorizedNumber fc = new ProcessFactorizedNumber(factorizableNum);

      // Define the AsyncCallback delegate
      AsyncCallback cb =
      new AsyncCallback(fc.FactorizedResults);

      // Can stuff any object as the state object
      Object state = new Object();

      // Asynchronously invoke the Factorize method on pf
      IAsyncResult ar = fd.BeginInvoke(
                        factorizableNum,
                        ref temp,
                        ref temp,
                        null,
                        null);

      ar.AsyncWaitHandle.WaitOne(10000, false);

      if (ar.IsCompleted)
      {
         int factor1=0, factor2=0;

         // Obtain the result
         fd.EndInvoke(ref factor1, ref factor2, ar);

         // Output results

         Console.WriteLine("Sequencial : Factors of {0} : {1} {2}",
                       factorizableNum, factor1, factor2);

      }
   }

   public static void Main(String[] args)
   {
      Simple simple = new Simple();
      simple.FactorizeNumber1();
      simple.FactorizeNumber2();
   }
}


Questions

What happens if a client does not call End?

If is up to the supplier of the IAsyncResult to cleanup when the IAsyncResult object is finalized.

Can the same instance of a callback object be passed to multiple calls simultaneously? (Assuming of course that is stateless with respect to the calls being made)

Yes.

Does an async server always need to provide a WaitHandle in IAsyncResult or Does the design pattern mandate that a server which provides its own asynchronous processing always provide a wait handle in its implementation of IAsyncResult? Is it legal to return null from IAsyncResult.WaitHandle?

The provider of the asyncresult must supply a wait handle, but you can wait until it is requested and fault it in at that time.


Asynchronous Programming Model Programming Reference

System.IAsyncResult

Interface to an Async result.

interface IAsyncResult
{
   public boolean IsCompleted() { get; }
   public boolean CompletedSynchronously() { get; }
   public WaitHandle AsyncWaitHandle () { get; }
   public Object AsyncObject () { get; }
   public Object AsyncObject () { get; }
}

IsCompleted. The IsCompleted property will be set to “true” after the server has completed processing of the call. It is illegal for the server to use any client supplied resources outside of the agreed upon sharing semantics after it sets the IsCompleted property to “true”. Thus, it is safe for the client to destroy the resources after IsCompleted property returns “true”.

CompletedSynchronously. The CompletedSynchronously property will be set to “true” if the BeginXXXXXX call completed synchronously. If this is detected in the AsyncCallback delegate, it is probable that the thread that called BeginInvoke is the current thread. Most providers of the IAsyncResult interface will not use the capability and will return a default false.

AsyncWaitHandle. The AsyncWaitHandle property returns the WaitHandle that can use to perform a WaitHandle.WaitOne or WaitAny or WaitAll. The object which implements IAsyncResult need not derive from the System.WaitHandle classes directly. The WaitHandle wraps its underlying synchronization primitive and should be signaled after the call is completed. This enables the client to wait for the call to complete instead polling. The Runtime supplies a number of waitable objects that mirror Win32 synchronization primitives e.g. ManualResetEvent, AutoResetEvent and Mutex.

WaitHandle supplies methods that support waiting for such synchronization objects to become signaled with “any” or “all” semantics i.e. WaitHandle.WaitOne, WaitAny and WaitAll. Such methods are context aware to avoid deadlocks. The AsyncWaitHandle can be allocated eagerly or on demand. It is the choice of the IAsyncResult implementer.

Implementors of classes that return IAsyncResult should note the following: The AsyncWaitHandle can be lazily allocated. However once allocated it should be kept alive until the user calls EndXXXX. At that time the object behind AsyncWaitHandle can be recycled.

AsyncObject. The AsyncObject returns the object that provided the IAsyncResult as part of the BeginXXXX method call. When BeginInvoke is called on the delegate and returns IAsyncResult, AsyncObject will contain for delegate object. For mechanisms such as file IO the AsyncObject could contain the file object on which the BeginXXXX method was called.

AsyncObject. The AsyncObject returns the object that was provided as the last parameter as part of the BeginXXXX method call.

System.AsyncCallback

Delegate for methods that complete asynchronously.

public delegate void AsyncCallback(IAsyncResult ar);


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Attributed Custom Marshaling for COM Interop

© 2000 Microsoft Corporation. All rights reserved.


What is a Custom Marshaler

The runtime’s marshaling service makes it possible to expose NGWS objects to COM and vice versa. In order to do this, the marshaler makes extensive use of wrappers called COM Callable Wrappers (CCW’s) and Runtime Callable Wrappers (RCW’s). A CCW is used to expose a managed type to COM and an RCW is used to expose an unmanaged type to the NGWS runtime. In most cases, the standard CCW or RCW that the marshaler generates is adequate for marshaling calls between the managed and unmanaged space. But, under some circumstances a custom wrapper is necessary or more desirable (see section 2). A custom wrapper is a specialized wrapper that allows a developer to control precisely how a managed type is exposed to COM and/or how an unmanaged type is exposed to the NGWS runtime. A custom wrapper is used in place of the standard wrappers when more control is need over the marshaling process.

A custom marshaler is a managed class used to activate custom wrappers as needed. The runtime marshaling service uses the custom marshaler to activate and deactivate wrappers as arguments and fields are marshaled across the Interop boundary. In order to use a custom wrapper in place of the standard wrapper, the parameter or field being marshaled must be decorated with the MarshalAs attribute (see section 3.6). The attribute identifies the custom marshaler used for marshaling that particular parameter. The custom wrappers are only used to marshal parameters or field that have the attribute applied.

Consider the managed interface INew. If exposed to COM through a standard COM callable wrapper (CCW), the interface would have the same methods as the managed interface and would use the marshaling rules built into the runtime’s marshaler. Instead of using the standard wrappers, a custom wrapper could be designed for use with the interface. The custom wrapper may have different marshaling behavior or it may expose the interface to COM in a different way. Suppose a well know COM interface called IOld already existed that provided the same functionality as the INew interface. A customized wrapper could provide an unmanaged implementation of IOld that simply delegated the calls to the managed implementation of the INew interface. The custom wrapper essentially acts as a bridge between the INew and the IOld interfaces.

Custom wrappers can also be built to replace standard runtime callable wrappers (RCW). In that case, the wrapper delegates calls made on the managed interface to another unmanaged interface implementation.

Using a custom marshaler has some significant advantages:

  • It allows the managed objects to customize the implementation of interfaces that are exposed to COM. This may be necessary when the behavior of the standard wrappers is insufficient.
  • It allows managed objects to support common functionality thru two different interfaces; one interface that is exposed to managed users and another interface that is exposed to unmanaged users of the object.
  • It allows unmanaged clients to work with new interfaces while only knowing about the old interfaces.

The wrapper essentially provides a bridge between the functionality of the old and new interfaces. They allow clients built to work with the old interface to continue to work with servers that only implement the new interface. Similarly, they allow clients built to work with the new interface to also work with server that expose the old interface.


Appropriate uses of Custom Marshalers

This section covers some of the common reasons for using custom marshalers.


Handling problem interfaces

While the runtime’s generic marshalers do a good job of handling most types defined within a type library. However, there are certain situations that may require custom marshaling.

Variable sized arrays

Variable sized arrays of occasionally passed through COM interfaces. In order to marshal the array from a managed space to unmanaged space the size of the array must be know. In IDL, the size is designated using the size_is or length_is attribute. For example:

[uuid(…)]
interface foo {
   HRESULT Bar(int cb, [size_is(cb)] char *buffer);
}

In this case, buffer points to an array of characters and the size of the array is defined by the cb argument. By knowing the size (cb) and type (char) of the array the marshaler can safely copy the data.

Unfortunately, the size information is not propagated to the type library by the MIDL compiler. That leaves the runtime’s generic marshaler with no information about how to marshal the array. Consequently there is no way to pass a variable sized array from unmanaged code into managed code using the standard wrappers generated by TlbImp. One alternative is to use the MarshalAs attribute to explicitly supply the array size of to provide a custom marshaler for the interface.

Passing arrays from managed to unmanaged code is not a problem. Since the size of the managed array can always be obtained by checking the Count property of the System.Array class. Managed types being exposed to COM should still pass the array size in a separate parameter so that the size can be obtained on the unmanaged code.

Void pointers

Void pointers present a similar problem. The runtimes generic marshaler has no information telling it how to marshaler the data. Therefore marshaling interfaces with void pointers requires a custom marshaler.

Success results

The generic marshalers automatically convert COM HRESULTs to NGWS runtime exceptions and vice versa. However, some COM methods return success HRESULTs that are not handled by the runtimes marshalers. A custom marshaler can be used to provide associate other semantic with COM success HRESULTs.

Bridging existing functionality

In some cases a custom marshaler is used even though the runtime generic marshaler may have worked just fine. Instead the custom marshaler is used to bridge the “old” technique to a “new” technique of doing something. Consider the IEnumVariant interface. In COM, the interface is used to enumerate over a collection of Variants. It this interface that Visual Basic uses to implement its for…each semantics.

In the new NGWS programming model, the IEnumerator interface provide similar semantics even though the interfaces are not exactly the same. To bridge the 2 programming models the runtime provides a built-in custom marshaler that converts the IEnumVariant interface to the IEnumerator interface and vice versa. As a result, NGWS runtime types that support IEnumVariant now appear to support the IEnumerator interface and NGWS runtime types that support IEnumerator now appear to support IEnumVariant. This makes it possible to use For…each over a managed object that support IEnumerator from Visual Basic 6.0.

Other built-in custom marshalers are provided for bridging ITypeInfo to System.Type and IExpando to IDispatchEx.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Refactoring of an interface

Sometimes a custom marshaler is used to simply refactor an interface.


Override the Generic Marshaler

Another reason for using a custom marshaler may be to replace the behavior of the runtime’s generic marshaler. For example, the generic marshaler always converts COM HRESULTs to NGWS runtime exceptions and vice versa. You may, for one reason or another, not like that approach and would instead prefer to expose the HRESULTs directly to the managed applications. Because the custom marshaler completely replaces the runtime’s generic marshaler for a give parameter of field, any behavior of the generic can be replaced with a custom implementation.


Improved Performance

One last reason for using a custom marshaler is performance. While we have made every effort to optimize the generic marshaler that the runtime provides there is cost associated with having a general-purpose marshaler.


Design Overview

Building a custom marshaler involves the following steps.

  • Defining both the managed and unmanaged interfaces being marshaled. One interfaces is managed and the other is unmanaged.
  • Defining wrappers that are responsible for converting from one interface to the other. If marshaling in both directions is desired, a separate wrapper is used in each direction.
  • Defining a custom marshaler that is responsible for activating and deactivating the appropriate wrapper at the appropriate time.
  • Identifying when and where the custom marshaler is to be used.

This section describes each of these four steps in details

Because custom marshalers need to deal with unmanaged interface definitions and need to be able to make calls through such interfaces, there are limitations on which compilers can be used to build custom marshalers. Currently, Managed C++ is the only language that is suitable for building custom marshalers.


Defining the Marshaling Types

Before you can begin to build a marshaler, the managed an unmanaged interfaces being marshaled must be defined. These interfaces commonly perform the same function but, for one reason or another, are exposed differently in the managed and unmanaged world.

The managed interface is defined in metadata produced by a managed compiler. It looks like any other managed interface.

In New.h
#using <mscorlib.dll>
interface INew {
   void NewMethod();
}

The unmanaged type is defined in IDL and compiled with the MIDL compiler. The interface is defined within a library statement and is assigned an interface ID with the uuid attribute.

In Old.idl
[uuid(9B2BAADA-0705-11D3-A0CD-00C04FA35826)]
Library OldLib {
    [uuid(9B2BAADD-0705-11D3-A0CD-00C04FA35826)]
    interface IOld : IUnknown
       HRESULT OldMethod();
}

The unmanaged interface is compiled with the MIDL compiler. The MIDL compiler produces several output files. If the interface is defined in old.Idl, the output file old_i.c defines a const variable with the IID of the interface:

In Old_i.c
const IID IID_IOld = {0x9B2BAADD,0x0705,0x11D3,{0xA0,0xCD,0x00,0xC0,0x4F,0xA3,0x58,0x26}};

The file Old.h is also produced by MIDL. It contains a C++ style definition of the interface that can be #included into your C++ source code.


Building the Wrappers

Custom wrapper classes are used to bridge the two interfaced defined in the previous step. A custom marshaler typically requires two wrapper classes; one for handling calls from managed to unmanaged and another for handling calls from unmanaged to managed. The wrappers typically cache a reference to one interface and implement the other interface.

Sections 3.3 and 3.4 cover each wrapper.


Custom Runtime Callable Wrappers (CRCWs)

This section describes customized runtime callable wrappers (CRCWs), which handle calls made from managed to unmanaged code.

A CRCW class is typically used to proxy calls made from managed to unmanaged code. The wrapper class is a garbage-collected class that implements a managed interface and caches a reference to the unmanaged interface. As calls are made on the methods of the managed interface, the wrapper handles the call by delegating the call to the cached, unmanaged interface.

In this example, the CRCW class called NewToOldWrapper implements the managed interface INew and delegates the calls to the unmanaged interface IOld.

In Wrappers.h
__gc class NewToOldWrapper : public INew  {
public:
   NewToOldWrapper (int pIOld);
   void NewMethod(void);

private:
   IOld *m_pIOld;
};

In Wrappers.cpp
#using <mscorlib.dll>
#include <New.h>
#include <Old.h>
#include “Wrappers.h”

NewToOldWrapper::NewToOldWrapper (int pIOld) {
   m_pIOld = (IOld *) pIOld;
};

void NewToOldWrapper::NewMethod() {
   HRESULT hr = m_pIRawOld->OldMethod();
   if FAILED(hr)
      ; // throw exception
   return;
};

In the code shown here, the wrapper’s implementation of NewMethod directly calls OldMethod on the IOld interface that the wrapper is referencing. The wrapper could provide an alternative implementation of the NewMethod.


Custom COM Callable Wrappers (CCCWs)

This section discusses customized COM callable wrappers (CCCWs), which handle calls made from unmanaged to managed code.

The CCCW class is typically used to proxy the calls from unmanaged to managed code. The wrapper class is a reference counted class that implements an unmanaged interface and typically caches a reference to a managed interface. As calls are made on the unmanaged interface, the wrapper handles the call by delegating the call cached, managed interface.

For example, consider the OldToNewWrapper shown below. This wrapper implements the unmanaged interface IOld and delegates the calls to the managed interface INew.

In Wrappers.h
class OldToNewWrapper : public IOld {
public:
   OldToNewWrapper (INew *pNew);
   ~OldToNewWrapper();

   ULONG __stdcall AddRef(void);
   ULONG __stdcall Release(void);
   HRESULT __stdcall QueryInterface(REFIID riid, void **ppv);
   HRESULT __stdcall OldMethod(void);

private:
   ULONG m_cRef;
   INew *m_pINew;
};

In Wrappers.cpp
OldToNewWrapper::OldToNewWrapper (INew *pNew) {
   m_pINew = pNew;
   m_cRef = 1;
}

ULONG OldToNewWrapper::AddRef(void) {
   return ++m_cRef;
}

ULONG OldToNewWrapper::Release(void) {
   if (--m_cRef != 0)
         return m_cRef;

   delete this;
   return 0;
}

HRESULT OldToNewWrapper::QueryInterface(REFIID riid, void **ppv) {
   if (riid == IID_IOld)
      *ppv = (IOld *)this;
   else
   {
      *ppv = NULL;
      return E_NOINTERFACE;
   }

   AddRef();
   return S_OK;
}

HRESULT OldToNewWrapper::OldMethod() {
try {
      m_pINew->NewMethod();
catch (Exception *e) {
   return E_FAIL;
}

   return S_OK;
}

In the code shown here, the wrapper’s implementation of OldMethod directly calls NewMethod on the cached INew interface. The wrapper could provide an alterative implementation of the OldMethod.

Note: The code above shows m_pINew as a member of the OldToNewWrapper class. Currently implementations of the MC++ compiler do not allow unmanaged types (like OldToNewWrapper) to contain references to managed types. To get around this issue, the reference to the managed type should be stored in a static member of a global class. For example:

__gc class Globals {
   public:
      static INew *m_pINew;
}


The Custom Marshaler

The custom marshal is a class that activates and deactivates a specific custom wrapper as necessary. The marshaler class must implement the ICustomMarshaler interface defined in the System.Runtime.InteropServices namespace.

The ICustomMarshaler Interface

The custom marshaler implements the ICustomMarshaler interface in order to provide the runtime’s Interop service with the appropriate wrappers. In order to use a custom marshaler, an attribute is associated with a parameter or field being marshaled. The attribute identifies the custom marshaler that knows how to activate the appropriate wrapper. The runtime’s Interop service then examines the attribute and creates the custom marshaler the first time the argument needs to be marshaled. It calls on the custom marshaler’s MarshalNativeToManaged and MarshalManagedToNative methods to activate the correct wrapper to handle the call.

The ICustomMarshaler Inteface
namespace System.Runtime.InteropServices;

public interface ICustomMarshaler {
       Object *MarshalNativeToManaged( int UnmanagedType );
       int MarshalManagedToNative( Object *ManagedType );
       void CleanUpNativeData( int UnmanagedType );
       void CleanUpManagedData( Object *ManagedType );
       static Guid GetNativeTypeGuid();
       static Type GetManagedType();
      static ICustomMarshaler *GetInstance(String *pstrCookie);
}

The MarshalNativeToManaged method returns a custom RCW that can marshal the unmanaged interface passed as an argument. The marshaler should return an instance of the custom RCW for that type. This method is called by the runtime in order to obtain the wrapper the first time a call requiring marshaling is made from managed code into unmanaged code.

The MarshalManagedToNative method returns a custom CCW that can marshal the managed interface passed as an argument. The marshaler should return an instance of the custom CCW for that type. This method is called by the runtime in order to obtain the wrapper the first time a call requiring marshaling is made from unmanaged code into managed code.

The CleanUpNativeData method provides an opportunity for the custom marshaler to perform any necessary cleanup when the marshaler is no longer needed.

The CleanUpManagedData method provides an opportunity for the custom marshaler to perform any necessary cleanup when the marshaler is no longer needed.

In addition, the custom marshaler must also implement three static methods. The GetInstance method is called to retrieve an instance of the custom marshaler. This method can either create a new instance of the custom marshaler every time or it can always use the same one if the custom marshaler is stateless. The method is called by the marshaling service to create the marshaler whenever a parameter of field is marked with the UnmanagedType.CustomMarshaler (see below)

The static methods GetNativeTypeGuid and GetManagedType return information about the managed and unmanaged types that the marshaler knows how to bridge. The managed type information is returned as a System.Type object. The unmanaged type information is returned as a Guid. The Guid that’s returned from GetNativeTypeGuid should be the IID of the COM interface being marshaled.

The cookie passed to GetInstance is explained later.

Sample Custom Marshaler

The runtime uses the custom marshaler to create custom wrappers as they are needed. A separate custom marshaler is typically used for each type being marshaled. For example, the NewOldMarshaler shown below would be used to activate the NewToOldWrapper or the OldToNewWrapper defined above.

In NewOldMarshaler.cpp
#using <mscorlib.dll>
#include “New.h”
#include “Old.h”
#include “Wrappers.h”

__gc class NewOldMarshaler :public ICustomMarshaler
{
private:
   NewOldMarshaler() {};
   static NewOldMarshaler *m_pMarshaler = NULL;

   Object *MarshalNativeToManaged( int pUnmanagedObj)   {
      NewToOldWrapper *crcw = new NewToOldWrapper(pUnmanagedObj);
      return dynamic_cast<Object *>(crcw);
   };

   int MarshalManagedToNative( Object *pManagedObj )    {
      IOld *pio;
      OldToNewWrapper *cccw =
               new OldToNewWrapper(dynamic_cast<INew *>(pManagedObj));
      HRESULT hr = cccw->QueryInterface(IID_IOld, (void **)&pio);
      return (int)(pio);
   };

   void CleanUpNativeData(int i) {
   };

   void CleanUpManagedData(Object *o) {
   };

   static Guid GetNativeTypeGuid() {
      return Guid(IID_IOLD);
   };

   static Type GetManagedType() {
      return Type.GetType(“INew”);
   };

   static ICustomMarshaler *GetInstance(String *pstrCookie)   {
      if (!m_pNewOldMarshaler)
         m_pNewOldMarshaler = new NewOldMarshaler ;
      return m_pNewOldMarshaler;
   };
}

The static method GetInstance is called by the runtime when an instance of the custom marshaler is needed. The implementation shown above ensures that only a single instance of the marshaler is created. As long as the marshaler does not retain state, using this approach is recommended.

The constructor for the marshaler is private to ensure that class is not created anywhere other than thru the GetInstance method.

The MarshalNativeToManaged method is responsible for creating a wrapper that implements the managed interface. The implementation shown above simply constructs an instance of the NewToOldWrapper and returns it to the caller.

The MarshalManagedToNative method creates a wrapper that implements the unmanaged interface. The implementation shown above constructs an instance of the OldToNewWrapper. It then queries the wrapper of its IOld interface and returns the interface to the caller.


Using a Custom Marshaler

Once the marshaler is complete, it can be used as a custom wrapper for a particular type. Consider the managed interface definition IUserData written in C# below:

public interface IUserData {
   public void DoSomeStuff(INew pINew);
}

You would like the IUserData interface to use the NewOldMarshaler in order for unmanaged clients to be able to pass an IOld interface to the DoSomeStuff method method. This implies that the managed description of the DoSomeStuff method would take an INew interface (shown above) while the unmanaged version of DoSomeStuff takes an IOld interface pointer (shown below).

[uuid(9B2BAADA-0705-11D3-A0CD-00C04FA35826)]
Library UserLib {
    [uuid(9B2BABCD-0705-11D3-A0CD-00C04FA35826)]
    interface IUserData : IUnknown
       HRESULT DoSomeStuff(IOld pIOld);
}

When done correctly, the type library generated by exporting the managed definition of IUserData would yield the unmanaged definition shown above rather than the standard definition.

To accomplish this, the INew argument in the managed definition of the DoSomeStuff method is decorated with a custom attribute as shown below. The System.Runtime.InteropServices.MarshalAs attribute is used to indicate that the argument uses a custom marshaler.

using System.Runtime.InteropServices;

public interface IUserData {
void DoSomeStuff(
      [MarshalAs(UnmanagedType.CustomMarshaler, MarshalType=”NewOldMarshaler”]
      INew pINew
   );
}

When used to specify a custom marshaler, the MarshalAs attribute has 2 named arguments:

  • MarshalType (Required) - The fully qualified name of the custom marshaler. The name should include the namespace and class of the custom marshaler.
  • MarshalCookie (Optional) - A cookie passed to the custom marshaler. The cookie can be used to provide additional information to the marshaler. For example, the same marshaler could be used to provide a number of wrappers where the cookie is used to indicate the specific wrapper. The cookie is passed to the GetInstance method of the marshaler.

See the class library documentation for System.Runtime.InteropServices.MarshalAs for more details.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Common tasks performed by custom marshalers

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Marshaling Variable length Arrays

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Marshaling void pointers

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Bridging old to new

© 2000 Microsoft Corporation. All rights reserved.


The System Collection Classes

Overview

This document describes the system collection classes. They are designed to facilitate interoperability between collection definers, provide rich implementations of common collections and support the collection design pattern.

Abstract

The System provides a set of Collection classes that meet three primary goals:

  1. Interfaces that represent a standard definition for common collection to facilitate interoperability between collections. Interfaces are the standards for collection because many objects may want to act like collections.
  2. Reference implementation of the recommended Collection design pattern.
  3. Functionality rich implementations of common collection classes that wrap the collection interfaces we have defined.


Goals and Scenarios

It is impossible to write a set of generic collection classes that meets everyone’s needs. Very frequently a developer will want to create collection classes that are tailored to some domain specific needs. For example, a developer may want a strongly typed, homogenous collection to avoid casting to and from a generic type or there may be special constraints that allow a more efficient sorting algorithm than we can provide generically. We do not attempt to provide every possible collection implementation. What we do provide is a set of interfaces that allow collections to be treated generically and support polymorphism. We provide several implementations that support these interfaces and add additional functionality. Lastly, we provide the reference implementation for the Collection design pattern for the type System.Object.


Standard Definition Interfaces

As we discussed above, collection implementations can vary widely. But there are some common elements of all collections (and particular kinds of collections) that can be logically grouped into interfaces. These interfaces are useful in treating all collections in a common way. For example, language syntax such as VB’s for each statement should work over all collections, regardless of implementation details.


Rich Set of Functionality through Wrapper Classes

Interfaces are needed for developers to treat their own types as collections. For a large set of the runtime’s users, generic implementations are extremely valuable. The runtime will provide wrapper classes to work over top of the interfaces thus allowing the collection implementer to implement the smallest number of methods possible and still allow their users to get a very rich experience when using them. Providing a rich set of functionality in these classes makes it easier for the RAD developer to get the functionality she needs quickly.


Reference Implementations of the Collection Design Pattern

Microsoft is evangelizing a simple design pattern for collections that is applicable across domains. We will provide a generic collection that supports this design pattern and can be used for delegation.

We provide a set of classes that are the reference implementation of that design pattern for the type System.Object. These classes serve as the definitive answer on questions of the “correct” behavior for collection classes.

Note, in a future release, we may provide full support for generics. This design pattern will work nicely with that support.


Explicit Interface Member Implementations (MethodImpls)

For purposes of implementing interfaces, a class or struct may declare explicit interface member implementations. An explicit interface member implementation is a method, property, event, or indexer declaration that references a fully qualified interface member name. For example

interface ICloneable
{
   object Clone();
}
interface IComparable
{
   int CompareTo(object other);
}
class ListEntry: ICloneable, IComparable
{
   object ICloneable.Clone() {...}
   int IComparable.CompareTo(object other) {...}
}

Here, ICloneable.Clone and IComparable.CompareTo are explicit interface member implementations.

It is not possible to access an explicit interface member implementation through its fully qualified name in a method invocation, property access, or indexer access. An explicit interface member implementation can only be accessed through an interface instance, and is in that case referenced simply by its member name.

It is an error for an explicit interface member implementation to include access modifiers, as is it an error to include the abstract, virtual, override, or static modifiers.

Explicit interface member implementations have different accessibility characteristics than other members. Because explicit interface member implementations are never accessible through their fully qualified name in a method invocation or a property access, they are in a sense private. However, since they can be accessed through an interface instance, they are in a sense also public.

Explicit interface member implementations serve two primary purposes:

  • Because explicit interface member implementations are not accessible through class or struct instances, they allow interface implementations to be excluded from the public interface of a class or struct. This is particularly useful when a class or struct implements an internal interface that is of no interest to a consumer of the class or struct.
  • Explicit interface member implementations allow disambiguation of interface members with the same signature. Without explicit interface member implementations it would be impossible for a class or struct to have different implementations of interface members with the same signature and return type, as would it be impossible for a class or struct to have any implementation at all of interface members with the same signature but with different return types.

For an explicit interface member implementation to be valid, the class or struct must name an interface in its base class list that contains a member whose fully qualified name, type, and parameter types exactly match those of the explicit interface member implementation. Thus, in the following class

class Shape: ICloneable
{
   object ICloneable.Clone() {...}
   int IComparable.CompareTo(object other) {...}
}

the declaration of IComparable.CompareTo is invalid because IComparable is not listed in the base class list of Shape and is not a base interface of ICloneable. Likewise, in the declarations

class Shape: ICloneable
{
   object ICloneable.Clone() {...}
}
class Ellipse: Shape
{
   object ICloneable.Clone() {...}
}

the declaration of ICloneable.Clone in Ellipse is in error because ICloneable is not explicitly listed in the base class list of Ellipse.

The fully qualified name of an interface member must reference the interface in which the member was declared. Thus, in the declarations

interface IControl
{
   void Paint();
}
interface ITextBox: IControl
{
   void SetText(string text);
}
class TextBox: ITextBox
{
   void IControl.Paint() {...}
   void ITextBox.SetText(string text) {...}
}

the explicit interface member implementation of Paint must be written as IControl.Paint.

For further information, see http://comrtime/specs/Formats/MiniSpec%20MethodImpls.doc


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Class Hierarchies

Figure 1 Collection Interfaces

Figure 2 Default and Reference Implementations


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Interface Definitions

© 2000 Microsoft Corporation. All rights reserved.


IEnumerable

Exposes the enumerator. The enumerator allows simply ForEach style iteration through a collection.

public interface IEnumerable {
   IEnumerator GetEnumerator();
}


ICollection

ICollection will expose the following functionality:

  • Synchronization. We need to expose synchronization functionality in a first class way. The List and Hashtable classes will provide some synchronization wrappers for people who do not want to implement synchronization themselves. It is not our desire to hide synchronization. Instead, we want the use of synchronization to be a choice developers make explicitly.
  • Interop between Arrays. System.Array will be a collection and collections should transform into basic arrays.
  • We understand that Count may be expensive to implement on some database driven collections. Since this is not the common case, and Count is a useful thing in most cases, we choose to leave count in place. We recommend throwing a NotSupportedException if Count is prohibitively expensive to compute.
  • CopyTo: The array must be a single dimension array, but may have any low-bound. The index must be in the range of array.LowBound to Array.LowBound+Array.Length.
public interface ICollection : IEnumerable {
   void CopyTo(Array array, int index);
   property int Count;
   property Object SyncRoot;
   property boolean IsSynchronized;
   property boolean IsReadOnly;
}


IEnumerator

The enumerator enables very simple ForEach style enumeration. It is not guaranteed that two Enumerators taken from the same collection at the same time will result in the same iteration.

Enumerators iterate over the members of a collection. They represent a logical snapshot of the collection data items. Items added or removed from the underlying collection may have no effect or may put the enumerator in an invalid state.

When an enumerator is started the current position is before the first element in the collection. Therefore, calling Current is invalid and results in an InvalidOperationException being thrown.

MoveNext will return true until it has reached the end of the collection. It will then return false for each successive call. However once MoveNext has returned false, accessing the Current property will throw an InvalidOperationException.

Reset returns the enumerator to the position it started from. Multiple calls to Reset() must result in logically the same state for the enumerator. This may take a new snap-shot from the collection or simply reset to the start of this collection.

public interface IEnumerator {
   bool MoveNext ();
   object Current {get;}
   void Reset ();
}


IDictionaryEnumerator

IDictionaryEnumerator is an extension to enumerator that allows for iterating over a dictionary.

interface IDictionaryEnumerator extends IEnumerator
{
   bool MoveNext ();
   object IEnumerator.Current {get;}
   DictionaryEntry Current {get;}
   DictionaryEntry Entry {get;}
   object Key {get;}
   object Value {get;}
   void Reset ();
}


IList

IList extends from ICollection and adds methods for using a list of items.

public interface IList extends ICollection {

   //From ICollections
   void CopyTo(Array array, int index);
   IEnumerator GetEnumerator();
   int Count {get;};
   Object SyncRoot {get;};
   boolean IsSynchronized {get;};

   //IList
   Object this[int index] {get; set;}
   int Add (Object value);
   void AddRange(ICollection collection); //Note: Not yet implemented
   void Remove (Object value);
   void Clear();
   boolean Contains (Object value);
   int IndexOf (Object value); // Uses Object.Equals method
   void Insert (int index, Object value);
   void RemoveAt(int index);
}


IDictionary

IDictionary extends from ICollection and adds methods for using a key/value pairs.

public interface IDictionary extends ICollection {

  //From ICollections
   void CopyTo(Array array, int index); //Copies the dictionary entries to the array
   IEnumerator ICollection.GetEnumerator();
   //The return value must be castable to IDictionaryEnumerator
   //Used directly, the IEnumerator enumerates over the values.
   property int Count;
   property Object SyncRoot;
   property boolean IsSynchronized;
   property boolean IsReadOnly;

   //IDictionary
   ICollection Keys {get;};
   ICollection Values {get;};
   Object this[Object key] {get;set};

   void CopyTo(DictionaryEntry [] array, int index);//not implemented
   boolean Contains (Object value); //returns true if there is a key that equals value
   void Add (Object key, Object value);
   void Clear ();
   void Remove (Object key);
   IDictionayEnumerator GetEnumerator();
}


DictionaryEntry

This class encapsulates a key/value pair. IDictionary uses it to return the values in the dictionary in CopyTo and GetEnumerator.

public struct DictionaryEntry {
public:
   //Constructors
   DictionaryEntry (Object key, Object value);

   //Fields
   Object Key;
   Object Value;
}
}


CaseInsensitiveComparer

This class is used to form a case insensitive hashtable. It compares two strings as equal ignoring case.

public class CaseInsensitiveComparer: IComparer

   //Constructors
 public CaseInsensitiveComparer ();
 Public Int Compare (Object a, Object b);
 public static CaseInsensitiveComparer Default;

}
}


CaseInsensitiveHashCodeProvider

This class is used to form a case insensitive hashtable. It returns the same hashcode for two strings that differ only in case.

public class CaseInsensitiveHashCodeProvider : IHashCodeProvider

   //Constructors
 public CaseInsensitiveHashCodeProvider ();
 Public int GetHashCode (Object obj);
 public static CaseInsensitiveHashCodeProvider Default;

}
}


Design Patterns

Design patterns are useful to leverage a developer’s experience. If each collection uses the same basic pattern, developers know what to expect and do not have to relearn a different set of collection classes in each library.


Design Guidelines

In addition to following the pattern outlined below, collections should follow these guidelines:

Do name List Collections with a suffix of “Collection”

OrdersCollection, ButtonsCollection

Do implement ICollection on all collection classes.

Do provide a default-indexed property (indexer) named Item that returns a single item in the list.

class OrderCollection : ICollection
{
   public OrderItem this[int index] { get ; set; }
}

Do use strongly typed collections

Do use plural names for properties that are collections

Do use multiple homogeneous collections rather than a single heterogeneous collection.

For instance:

class MyBedroom {

      BooksCollection Books { get ; }
      ToysCollection Toys { get ; }
}

class BooksCollection: ICollection
{
   public default Book Item[String name] { get ; set ; }
}

class ToysCollection: ICollection
{
   public Default Toy Item[String name] { get ; set ; }
}

Rather than a single collection of both Toys and Books.

Do implement IList as a private interface.

Do use collections for subvalues of an object.

Do extend BaseList.

Do not use indexed methods (e.g. methods that take an index as a parameter) as a substitute for a true collection class. Our developers find them confusing.

Consider not creating collections with public constructors.

Collection instances are not created directly by the developer. The developer only gets the instance from the object that contains the collection. The idea is that for Components where you have properties of type "collection", that you would follow use a factory design pattern. General use collection (which pretty much only exist in "System.Collections") would have public constructors, however "WinForms.ControlsCollection" that is used by Control.Controls, would not.

public class FooBarsCollection : ICollection {
     FooBarsCollection() { // non public constructor
     }
}

Do include the owning class name in the name of the collection if it has an owning class.

If a Foo class has a collection of Bar subobjects named Bars, then the type returned from the property is named FooBars:

public class Foo {
    FooBars bars;

    public FooBars Bars {
        get { return bars; }
    }
}

This convention allows collections containing the same type of object to expose different functionality, e.g. a read only collection, a read/write collection, etc.

Do use Indexed properties only on collections that have unambigious key types.

Support more than one lookup key on a collection using overloaded index properties. The type of lookup is unambiguous given the type of the key. For example:

public class FooBarsCollection : ICollection {
    public Bar this[int index] {
        get { }
        set { }
    }

    public Bar this[string index] {
        get { }
        set { }
    }
}

Do follow the pattern that the Add method of a collection adds the value to the end of the collection if the collection is ordered linearly and allows additions.

Do use private interface implementation to enable strongly typed collections.


Index Based Collections

Index based collections are sequenced collections that are accessed via an integer index. The example below shows the design pattern, implementers need only implement the methods that are appropriate to their collection. The List class is the default implementation of this pattern.

class <Item>List : IList, ICollection {

   // ICollection
   void ICollection.CopyTo(Array array, int index); //private impl.
   IEnumerator ICollection.GetEnumerator(); //private implementation
   int Count{get;};
   Object ICollection.SyncRoot {get;}; //private implementation
   boolean ICollection.IsSynchronized {get;}; //private implementation

   // IList
   Object IList.this[int index] {get; set;} //private implementation
   void IList.Add (Object value); //private implementation
   void Clear();
   int  IList.IndexOf (Object value); //private implementation
   void IList.Insert(int index, Object value); //private implementation
   void RemoveAt(int index);
   void Remove(Object value); //private implementation
   boolean Contains (Object value); //private implementation

   // <item> Design Pattern
   boolean Contains (<item> value);
   public <item> this[int index];    // default index (set,get)
   //can’t insert greater than length.
   //ArgumentOutOfRange exception thrown if out of range.
   //ReadOnly collections only

   public void CopyTo(<item>[] array, int index); //ReadWrite only
   public int Add(<item> value); //adds at end  //ReadWrite only
   public void AddRange(<item>[] array);//ReadWrite only
   public void Insert(int index, <item> value); //insert,ReadWrite only
   public int IndexOf (<item> value);
   public <item>[] ToArray ();
   public void Remove (Object value); //ReadWrite only

Implementing IList is optional but highly recommended. If you are implementing a collection around a value type, you should consider using strongly typed collections for efficiency. For Read only collections, the appropriate indicated methods should throw an InvalidOperationException.


Index Based Named Collections

Need description here.

class <Value>IndexedDictionary : ICollection {

   // ICollection
   public void ICollection.CopyTo(Array array, int index);  //Creates an array of DictionaryEntries (private)
   IEnumerator IEnumerator.GetEnumerator(); //Enumerates over DictionaryEntries (private)
   //The return value must be castable to IDictionaryEnumerator
   //Used directly, the IEnumerator enumerates over the values.

   public int Count {get;};
   public Object ICollection.SyncRoot {get;};  // private
   public boolean ICollection.IsSynchronized {get;}; // private

   public void CopyTo(DictionaryEntry[] array, int index);
   public IDictionaryEnumerator GetEnumerator ();

   // Design Pattern
   public virtual void Clear ();
   public virtual bool ContainsKey (<Key> key);
   public virtual bool Contains (<Value>  value);
   public virtual <Value> this[<Key> key] {get;};
   public virtual <Value> Get[int index] {get;};
   public virtual void Remove (<Key>  key);
   public virtual void RemoveAt(int index);
   public virtual int Add (<Key> key, <Value> value);
   public virtual void Insert (int index, <Key> Key, <Value> value);
   public int IndexOf (<item> value);

}


Key Based Collections

Key based collections are non-sequenced collections that are accessed via a key typed as an Object. The key can be any type. Multiple key types are allowed. The example below shows the design pattern, implementers need only implement the methods that are appropriate to their collection. The Dictionary class is the default implementation of this pattern.

class <Value>Dictionary : ICollection, IDictionary{

   // ICollection
   public void ICollection.CopyTo(Array array, int index); //Creates an array of DictionaryEntries (private)
   IEnumerator ICollection.GetEnumerator(); //uses DictionaryEntries    //The return value must be castable to IDictionaryEnumerator
   //Used directly, the IEnumerator enumerates over the values.
   //private

   public int Count {get;};
   public Object ICollection.SyncRoot {get;}; //private implementation
   public boolean ICollection.IsSynchronized {get;}; //private implementation

   // IDictionary
   public ICollection Keys {get;};
   public ICollection Values {get;};
   virtual void IDictionary.Add (Object key, Object value);
//private implementation
   public virtual Object IDictionary.this[Object value];
//private implementation
   public virtual void Clear ();
   virtual void IDictionary.Remove (Object key);
//private implementation
   public void CopyTo(DictionaryEntry[] array, int index);
   public IDictionaryEnumerator GetEnumerator ();

   // Design Pattern
   static  <Value>Dictionary  Synchronized (<Value>Dictionary dictionary);
   public <Value> Item [<Key> key];     // default index (set,get)
                                       //Sets if their, adds otherwise
   public virtual void Add (<Key> key, <Value> value);
   public virtual void Remove (<Key>  key);

}


Key and Index Based Collections

The key and index based collections are sequenced collections that are accessed either by an integer index or via a key. The key can be any type. Multiple key types are allowed. The example below shows the design pattern, implementers need only implement the methods that are appropriate to their collection. The SortedList class is the default implementation of this pattern.

class <Value>IndexedDictionary implements ICollection {

   // ICollection
   public void CopyTo(Array array, int index);  //Creates an array of DictionaryEntries
   IEnumerator IEnumerator.GetEnumerator(); //Enumerates over DictionaryEntries
   //The return value must be castable to IDictionaryEnumerator
   //Used directly, the IEnumerator enumerates over the values.

   public int Count {get;};
   public Object SyncRoot {get;};
   public boolean IsSynchronized {get;};
   public boolean Contains (Object key);

   public void CopyTo(DictionaryEntry[] array, int index);
   public IDictionaryEnumerator GetEnumerator ();

   // Design Pattern
   static <item> IndexedDictionary Synchronized (<item>IndexedDictionary list);
   public property ICollection Keys;
   public property ICollection Values;
   public virtual void Clear ();
   public virtual bool ContainsKey (<Key> key);
   public virtual bool ContainsValue (<Value>  value);
   public virtual <Value> this[<Key> key] {get;};
   public virtual <Value> Get[int index] {get;};
   public virtual void Remove (<Key>  key);
   public virtual void RemoveAt(int index);
   public virtual int Add (<Key> key, <Value> value);
   public virtual void Insert (int index, <Key> Key, <Value> value);
   public int IndexOf (<item> value);

}


Default Implementations

The System provides default implementations of the design pattern described in section 3. The goal of this implementation is to be the definitive source for the “correct” implementation of the design pattern (ObjectList, etc) and provide a useful set of List classes for doing common operations.

Lists

The Collection classes offers a rich set of utility classes (List, ObjectList, etc) and a default implementation of the List design pattern (BaseList).

BaseList (for Beta2)

The BaseList class is the default implementation of the List design pattern.

public class ObjectList : IList, ICollection
{
   public BaseList();
   public BaseList(ICollection collection);

  //Properites
   Object this[int index] {get;set;};
   public int Count {get;};
   public Object SyncRoot {get;};
   public boolean IsSynchronized {get;};

//Methods
   static ObjectList Synchronized (ObjectList list);
   public int Add(Object value); //adds at end
   public void Insert(int position, Object value); //insert
   public void CopyTo(Array array, int index); //from IList
   public void CopyTo(Object[] array, int index);
   public boolean Contains (Object value);
   public void Clear();
   public IEnumerator GetEnumerator();
   public Object[] ToArray ();
   public static List FromArray (Object [] array);
   public void RemoveAt(int position);
   public void Remove(Object value); //shrink the collection
}

ArrayList

public class ArrayList: ICollection, ICloneable
{
   ObjectList();
   ObjectList(int capacity);
   ObjectList(ICollection collection);

   public static readonly List Empty;
public static List Synchronized(List list);
public static List FixedSize(List list);
public static List ReadOnly(List list);
public static List Repeat(object value, int count);

   // ICollection
   void CopyTo(Array array, int index);
   public IEnumerator GetEnumerator();
   public int Count {get;};
   public Object SyncRoot{get;};
   public boolean IsSynchronized{get;};
   public boolean Contains (Object value);

   // IList

   public Object this[int index]; //(read-write)
   public int Add (Object value);
   public void Clear();
   public int IndexOf (Object value); // Uses Object.Equals method
   public void Insert(int index, Object value);
   public void RemoveAt(int index);

   public virtual void AddRange(ICollection c);
   public virtual int BinarySearch(int index, int count, object value, IComparer comparer);
   public virtual int BinarySearch(object value); (not implemented in the current release)
   public virtual int BinarySearch(object value, IComparer comparer); (not implemented in the current release)
   public virtual bool Contains(object value);
   public virtual void CopyTo(Array array);
   public virtual void CopyTo(Array array, int arrayIndex);
   public virtual void CopyTo(int index, Array array, int arrayIndex, int count);
   public virtual IEnumerator GetEnumerator(bool allowRemove);
   public virtual IEnumerator GetEnumerator(int index, int count, bool allowRemove);
   public virtual List GetRange(int index, int count);
   public virtual int IndexOf(object value);
   public virtual int IndexOf(object value, int startIndex);
   public virtual int IndexOf(object value, int startIndex, int endIndex);
   public virtual void Insert(int index, object value);
   public virtual void InsertRange(int index, ICollection c);
   public virtual int LastIndexOf(object value);
   public virtual int LastIndexOf(object value, int startIndex);
   public virtual int LastIndexOf(object value, int startIndex, int endIndex);
   public virtual void Remove(object value);
   public virtual void RemoveRange(int index, int count);
   public virtual void Reverse();
   public virtual void Reverse(int index, int count);
   public virtual void SetRange(int index, ICollection c);
   public virtual void Sort();
   public virtual void Sort(IComparer comparer);
   public virtual void Sort(int index, int count, IComparer comparer);

}

Dictionary

Below are the reference implementations of the Dictionary design pattern.

Dictionary

The Dictionary class is the default implementation of the Dictionary design pattern.

public class Dictionary : IDictionary, ICollection
{
   public Dictionary ();
   public Dictionary(int capacity)
   public Dictionary(int capacity, float loadFactor)
   public Dictionary(IDictionary dictionary)

//Private Implementations
    IEnumerator ICollection.GetEnumerator();

//Properites
   public int Count {get;};
   public Object SyncRoot {get;};
   public boolean IsSynchronized {get;};
   public ICollection Keys;
   public ICollection Values;
   public Object this[Object key] {get; set;}

// Methods
   static Dictionary  Synchronized (Dictionary dictionary);
   public void CopyTo(DictionaryEntry[] array, int index);
   public void CopyTo(Array array, int index); //from ICollection
   public boolean Contains (Object value);
   public IDictionaryEnumerator GetEnumerator();
   public virtual void Add (Object key, Object  value);
   public virtual void Clear ();
   public virtual void Remove (Object  key);
   }

Hashtable

The Hashtable class represents a dictionary of associated keys and values, implemented as a hashtable.

Objects used as keys in a hashtable must implement the GetHashCode and Equals methods (or they can rely on the default implementations inherited from Object if key equality is simply based on referential identity). Furthermore, the GetHashCode and Equals methods of a key object must produce the same results given the same parameters for the entire time the key is present in the hashtable. In practical terms, this means that key objects should be immutable, at least for the time they are used as keys in a hashtable.

When entries are added to a hashtable, they are placed into buckets based on the hashcode of their keys. Subsequent lookups of keys will use the hashcode of the keys to search a particular bucket, thus substantially reducing the number of key comparisons required to find an entry. A hashtable's maximum load factor, which can be specified when the hashtable is instantiated, determines the maximum ratio of hashtable entries to hashtable buckets. Smaller load factors cause faster average lookup times at the cost of increased memory consumption. The default maximum load factor of 1.0 generally provides the best balance between speed and size. As entries are added to a hashtable, the hashtable's actual load factor increases, and when the actual load factor reaches the maximum load factor value, the number of buckets in the hashtable is automatically increased by approximately a factor of two (To be precise, the number of hashtable buckets is increased to the smallest prime number that is larger than twice the current number of hashtable buckets).

  • Issue: Should we use the LKR Hashing algorithm? http://georgere/work/LKRhash/default.htm
  • Issue: For the synchronized version should we lock at the bucket level instead of at the whole table?
class Hashtable : public IDictionary, public ICollection, ICloneable
{
public:

   //Constructors
   Hashtable ();
   Hashtable (IDictionary d, float loadFactor);
   Hashtable (IDictionary d);
   Hashtable (int capacity, float loadFactor);
   Hashtable (int capacity);
   Hashtable (IHashCodeProvider hashCodeProvider, IComparer comparer)

   //Methods
      static IDictionary ReadOnly (IDictionary dictionary);
   static IDictionary Synchronized (IDictionary dictionary);

   // ICollection
   void CopyTo(Array array, int index);
   IEnumerator GetEnumerator();
   //The return value must be castable to IDictionaryEnumerator
   //Used directly, the IEnumerator enumerates over the values.

   property int Count {get;};
   property Object SyncRoot {get;};
   property boolean IsSynchronized {get;};
   boolean Contains (Object value);

   // IDictionary
   property ICollection Keys; {get;}
   property ICollection Values {get;};
   virtual void Add (Object key, Object value);
   virtual void Clear ();
   virtual void Remove (Object key);
   IDictionayEnumerator GetEnumerator();

   //ICloneable
   virtual Object Clone ();

   //Dictionary
   property ICollection Keys {get;};
   property ICollection Values {get;};
   property Object Item [Object key];     // default index (set,get)
   virtual bool ContainsKey (Object key);
   virtual bool ContainsValue (Object value); //This method is real slow
   public bool Equals (Object o);

};

SortedList

This class implements a sorted list of keys and values. Entries in a sorted list are sorted by their keys and are accessible both by key and by index. The keys of a sorted list can be ordered either according to a specific IComparer implementation given when the sorted list is instantiated, or according to the IComparable implementation provided by the keys themselves. In either case, a sorted list does not allow entries with duplicate keys.

A sorted list internally maintains two arrays that store the keys and values of the entries. The capacity of a sorted list is the allocated length of these internal arrays. As elements are added to a sorted list, the capacity of the sorted list is automatically increased as required by reallocating the internal arrays. The capacity is never automatically decreased, but users can call either TrimToSize or Capacity explicitly.

The GetKeyList and GetValueList methods of a sorted list provides access to the keys and values of the sorted list in the form of List implementations. The List objects returned by these methods are aliases for the underlying sorted list, so modifications made to those lists are directly reflected in the sorted list, and vice versa.

The SortedList class provides a convenient way to create a sorted copy of another dictionary, such as a Hashtable. For example:

Hashtable h = new Hashtable();
h.Set(...);
h.Set(...);
...
SortedList s = new SortedList(h);

The last line above creates a sorted list that contains a copy of the keys and values stored in the hashtable. In this particular example, the keys will be ordered according to the IComparable interface, which they all must implement. To impose a different ordering, SortedList also has a constructor that allows a specific IComparer implementation to be specified.

class SortedDictionary : Dictionary {
public:

   //Constructors
   SortedDictionary ();
   SortedDictionary (IDictionary d, IComparer comparer);
   SortedDictionary (IDictionary d);
   SortedDictionary (IComparer comparer, int capacity);
   SortedDictionary (IComparer comparer);

   //Methods

   static IDictionary ReadOnly (IDictionary dictionary);
   static IDictionary Synchronized (IDictionary dictionary);

   // ICollection
   void CopyTo(Array array, int index);
   IEnumerator ICollection.GetEnumerator();
   //The return value must be castable to IDictionaryEnumerator
   //Used directly, the IEnumerator enumerates over the values.

   property int Count {get;};
   property Object SyncRoot {get;};
   property boolean IsSynchronized {get;};
   boolean Contains (Object key);

   // IDictionary
   property ICollection Keys {get;};
   property ICollection Values {get;};
   virtual void Add (Object key, Object value);
   virtual void Clear ();
   virtual void Remove (Object key);
   IDictionayEnumerator GetEnumerator();

   //Dictionary
   property ICollection Keys {get;};
   property ICollection Values {get;};
   property Object Item [Object key];     // default index (set,get)

   //ICloneable
   virtual Object Clone ();

   virtual bool ContainsKey (Object key);
   virtual bool ContainsValue (Object value); //This method is realslow
   sealed IEnumerator GetEnumerator ();
   virtual IList GetValueList ();
   virtual IList GetKeyList ();
   virtual Object GetKey (int index);
   virtual Object GetByIndex (int index);
   virtual int IndexOfKey (Object key);
   virtual int IndexOfValue (Object value);
   virtual void RemoveRange (int index, int count);
   virtual void RemoveAt (int index);
   virtual void SetByIndex (int index, Object value);
   virtual void TrimToSize ();


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Related Classes

© 2000 Microsoft Corporation. All rights reserved.


System.Array

This is the base class for arrays, both one-dimensional and multi-dimensional. The indices of an array are 0-based. The length and rank (number of dimensions) of an array are 1-based. Note that jagged arrays (Type[][]...[]) are considered to be arrays of arrays, and therefore Rank returns 1 on a jagged array.

class Array : ICloneable,  ICollection
{
public:
   //Fields
   const static Array Empty;

   //Properites
   property int Count;
   property Object SyncRoot;
   property boolean IsSynchronized;
   property boolean IsReadOnly;
   property int Rank;
   property int Length;

   //Methods
   virtual Object BinarySearch(Object value, IComparer comparer)
   virtual Object Clone ();
   virtual void CopyTo(Array array, int index);
   virtual bool Equals (Object obj); //inherited from Object
   virtual IEnumerator GetEnumerator();

   virtual int GetLength (int rank);
   virtual int GetLowerBound (int dimension);
   virtual int GetUpperBound (int dimension);
   virtual object GetValue (int index);
   virtual object GetValue (int index1, int index2);
   virtual object GetValue (int index1, int index2, int index3);
   virtual object GetValue (int indices[]);
   virtual object GetValue (int index1, …);

   virtual void Sort(IComparer comparer);
   virtual void SetValue (object value, int index);
   virtual void SetValue (object value, int index1, int index2);
   virtual void SetValue (object value, int index1, int index2, int index3);
   virtual void SetValue (object value, int indices[]);
   virtual void SetValue (object value, int index1, …);

   static [sealed] IList AsList (Array array);
   static int BinarySearch (Array array, int index, int count, Object value);
   static int BinarySearch (Array array, int index, int count, Object value, IComparer comparer);
   static [sealed] int BinarySearch (Array array, Object value);
   static int BinarySearch (Array array, Object value, IComparer comparer);
   static void Clear (Array array, int index, int length);
   static void Copy (Array sourceArray, int sourceIndex, Array destinationArray, int destinationIndex, int length);
   static void Copy (Array sourceArray, Array destinationArray, int length);
   static [sealed] Array CreateInstance (Type elementType, int length);
   static [sealed] Array CreateInstance (Type elementType, int length1,
 int length2);
   static [sealed] Array CreateInstance (Type elementType, int length1,
 int length2, int length3);
   static [sealed] Array CreateInstance (Type elementType, int lengths[]);

   static [sealed] Array CreateInstance (Type elementType, int lengths[], int lowerBounds[]);
   static int IndexOf (Array array, Object value);
   static int IndexOf (Array array, Object value, int startIndex);
   static int IndexOf (Array array, Object value, int startIndex, int endIndex);

   static int LastIndexOf (Array array, Object value);
   static int LastIndexOf (Array array, Object value, int startIndex);

   static int LastIndexOf (Array array, Object value, int startIndex,
 int endIndex);
   static void Reverse (Array array);
   static void Reverse (Array array, int index, int count);
   static void Sort (Array array);
   static void Sort (Array array, int index, int count);
   static void Sort (Array array, int index, int count, IComparer comparer);

   static void Sort (Array array, IComparer comparer);
   static void Sort (Array keys, Array items);
   static void Sort (Array keys, Array items, int index, int count);

   static void Sort (Array keys, Array items, int index, int count,
 IComparer comparer);
   static void Sort (Array keys, Array items, IComparer comparer);

};


StringTable

class StringTable : public ICollection
{
public:
   //Constructors
   StringTable ();
   StringTable (int capacity);
   StringTable (int capacity, float loadFactor);
   //Properties
   property int Count {get;};
   property ICollection Keys {get;};
   property Object SyncRoot {get;};

   //Methods
   static StringTable Synchronized (StringTable stringTable);
   virtual int Add (String key);
   virtual void Clear ();
   virtual bool ContainsKey (String key);
   virtual void CopyTo(Array array, int index);
   virtual void CopyTo (String [] array, int arrayIndex);
   virtual IEnumerator GetEnumerator ();
   virtual String [] ToArray ();
   virtual StringTable FromArray (String [] array);
};


Stack

public class Stack : ICollection {

   public Stack ();
   public Stack (int initialCapacity)
   public Stack (ICollection collection);

   property int Count;
   property Object SyncRoot;
   property boolean IsSynchronized;
   property boolean IsReadOnly;

   static Stack Synchronized(Stack stack)
   virtual Object Peek ();
   virtual Object Pop ();
   virtual void Clear ();
   virtual void Push (Object value);
   virtual Object[] ToArray();
   virtual Stack FromArray (Object [] array)

   void virtual CopyTo(Array array, int index);
   virtual IEnumerator GetEnumerator();

}


Queue

public class Queue: ICollection {

   public Queue ();
   public Queue (int initialCapacity)
   public Queue (ICollection collection);
   property int Count;
   property Object SyncRoot;
   property boolean IsSynchronized;
   property boolean IsReadOnly;

   virtual Object[] ToArray();
   virtual Queue FromArray (Object [] array)
   virtual Object Peek ();
   virtual Object Peek (int count);
   virtual Object Dequeue();
   virtual Object Clear ();
   virtual void Enqueue (Object value);
   static Queue Synchronized(Queue queue)

   void virtual CopyTo(Array array, int index);
   virtual IEnumerator GetEnumerator();

}


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

BinarySearchTree

TBD


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

IHashCodeProvider

interface IHashCodeProvider
{
   int GetHashCode (Object o);

};


BitArray

The BitArray class manages a compact array of bit values. Individual bit values are represented as booleans, where true indicates that the bit is on (1) and false indicates the bit is off (0). The size of a BitArray is controlled by the client; indexing past the end of the BitArray throws an ArgumentException.

The intended use for BitArray is:

if (bitflag->Get(bitindex)) { ... }

public sealed class BitArray
{
   //Constructors
 public BitArray (BitArray bits);
 public BitArray (byte[] bytes);
 public BitArray (int length);
 public BitArray (int length, bool defaultValue);
 public BitArray (int[] values);

//Methods
 public virtual BitArray And(BitArray value);
public virtual bool Get(int index);
public virtual int GetLength();
public virtual BitArray Not();
public virtual BitArray Or(BitArray value);
 public virtual void Set(int index, bool value);
 public virtual void SetAll(bool value);
 public virtual void SetLength(int length);
public virtual BitArray Xor(BitArray value);
}


Comparers

IComparer

The IComparer interface supports a method that compares two objects. It is used in conjunction with the Sort and BinarySearch methods on the Array and List classes

public interface IComparer
{
   //Methods
 int Compare(object x, object y);
}

Comparer

Comparer is a default implementation of the IComparer interface that compares two Objects that implement the ICompareable interface

public class Comparer: IComparer
{
   //Fields
 public static readonly Comparer Default;
   //Methods
 public virtual int Compare(object a, object b);
}


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Com Classic Interop Issues

TBD.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Security Issues

none.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

NGWS Runtime Resources Specification

© 2000 Microsoft Corporation. All rights reserved.


Overview of NGWS Runtime Resources

Nearly every production quality application needs to use resources. A resource is any non-executable data that is logically deployed with an application. The data can take any number of forms such as as strings, images, persisted objects, etc.

An application often needs to be customized for different cultures. A culture is a set of preferences based on a user’s language, sub-language, and cultural conventions. In the NGWS framework, the Base Class Library describes a culture using the CultureInfo class. A culture is used to customize operations such as formatting of dates and numbers, sorting of strings, etc.

An application also often need to be customized for different countries or regions. A region is a set of standards for a particular country or region of the world. In the NGWS framework, the Base Class Library describes a region using the RegionInfo class. A region is used to customize operations such as formatting the currency symbol.

Localization of an application is the process of sharing the application’s executable code with the application’s resources customized for specific cultures. Although a culture and a region together constitute a locale, localization is not concerned with customizing an application to specific regions. The NGWS runtime does not support localization of component metadata.

This specification describes resources at a low-level as a collection of named bits in an assembly. An assembly is the unit of deployment, versioning, and name resolution in the NGWS runtime. The goal at this level is for the resources to be generic, extensible, and consistent with content generated by other tools. Resources at this level are stored and retrieved using metadata APIs and low-level Base Class Library methods of the Assembly class. These APIs are primarily of interest to developers of compilers, assemblers, and other tools.

The Base Class Library also provides a high-level abstraction (see System.Resources Overview) as a set of managed classes. The managed classes impose policies and ensure consistency in how resources are stored and retrieved. The high-level APIs are primarily of interest to developers of applications.

The scope of this specification is limited to describing NGWS runtime resources. Unmanaged code uses resources in legacy Win32 format. Win32 provides a rich set of APIs to access legacy resources. Trusted managed code can use the PInvoke mechanism to call the Win32 APIs.

The following sections detail resources in the NGWS runtime. Section Packaging and Deployment of Resources describes how resources are packaged and deployed. The two resource models are described in the next two sections. Section Low-Level Resource Model describes the low-level abstraction of resources.


Packaging and Deployment of Resources

The NGWS runtime uses a hub and spoke model for packaging and deploying resources. The hub is the main assembly that contains the non-localizable executable code and the resources for a single culture called the neutral culture. The neutral culture is the fallback culture for the application. Each spoke connects to a satellite assembly that contains the resources for a single culture. Satellite assemblies do not contain code.

The advantages of this model are:

  • Resources for new cultures can be added incrementally after an application is deployed.
  • An application needs to load only those satellite assemblies that contain the resources needed for a particular run.

The disadvantages of this model are:

  • Multiple SKU’s need to be managed.
  • The testing cost is larger because several configurations need to be tested.

The NGWS frameworks provide APIs to manage resources deployed using the above model. While most Microsoft applications built for the NGWS platform are expected to use the hub and spoke model with one culture per assembly, the NGWS runtime does not require applications to use this packaging model. The following are some examples of packaging models an application can use:

  • Package the non-localizable executable code and resources for all cultures in a single assembly.

    The advantages of this model are:

    • A single assembly needs to be deployed.
    • A single SKU needs to be managed.
    • The testing cost is lower than the satellite assembly model.

    The disadvantages of this model are:

    • All cultures must be ready to ship at the same time.
    • To add resources for a new culture, the single assembly must be redeployed.
    • An application needs to load the resources for all locales regardless of whether they are needed or not.
  • Package the non-localizable executable code and resources for the neutral culture in the main assembly. Package resources for related cultures in separate satellite assemblies. For example, U.S. English and British English resources could be packaged in an English satellite assembly and Japanese, Korean, and Chinese resources could be packaged in an Asian satellite assembly.

An application that does not use the NGWS packaging and deployment model for resources will have to supply a custom resource manager to access resources. The Base Class Library provides APIs that can be used to build such a resource manager.


Low-Level Resource Model

The NGWS runtime provides the low-level infrastructure for reading and writing resources. The Base Class Library provides a higher level abstraction built on top of the low-level infrastructure. In the low-level resource model, a resource consists of a name and associated data.

This section should be read in conjunction with the Assembly Manifest Specification.


Resource Name

Resources are identified by a logical name. The manifest contains a resource table that describes the mapping from the logical name to a physical location.

The resources for an assembly may be in one of the following locations:

  • In separate resource files in the same assembly. Each resource file can contain one or more resources.
  • Embedded in a PE in the same assembly
  • In another assembly

If a set of resources is packaged in a resource file, the resource table will contain a single entry for the set that describes the mapping from the set’s logical name to the physical localtion of the resource file.


Resource Data

The resource data can be of any kind. The resource data is not directly used or validated by the deployment subsystem or the loader. The NGWS runtime provides a default resource data format. Files that contain resource data in this format typically have the .resource file type. Resources can also be used for data in other formats such as .GIF, .WAV, etc.

The resource data is available to a client as a managed stream. The following figure shows the resource model.

Figure. Resource Model

Resources in PE Files

All resource data in a runtime PE file is kept in a contiguous block pointed to by the “Resources” entry in the COR Header. The first DWORD of each individual resource carries the length of the resource. When referring to resources in PE files, the assembly manifest contains the offset of the beginning of the resource within the resource data. The following figure shows the layout of resources in PE files:

Figure. Resources in PE Files


Resource Visibility

A resource may be visible outside the assembly or its visibility may be restricted to the assembly in which it is contained. As with types, a resource’s visibility scope is described in the assembly manifest.


Localization

The primary goal of the NGWS runtime resource model is to facilitate the development of localizable applications. The resource model allows a resource to be retrieved based on its name and culture.

Localization Requirements

The localization design must satisfy the following requirements:

  • Must allow a single worldwide binary to be developed.
  • Must allow the culture to be “switched” on the fly, i.e., the runtime must allow resources to be loaded from multiple locales while an assembly is executing.
  • The design must allow a wide variety of packaging models to be used. One application may use a single culture per assembly model while another may choose to package the resources for all cultures and the code in a single assembly.
  • The design should not require resources such as images and sound files to be repackaged before they are included in an assembly.

Localization and Assemblies

Cultures and Assembly Identity

Assemblies have culture information as part of their identity. As a result, a culture affects the process of binding to an assembly. The culture of an assembly is included in the assembly’s manifest and can be one of the following values:

  • Culture Independent. A culture independent value specifies that the resources packaged in the assembly are not specific to any particular culture.
  • A Specific Culture. The culture value specifies the culture for the resources packaged in the assembly. Examples of cultures are fr for French, de for German, etc.

The culture independent value should be specified for an assembly that packages resources for different cultures. Since the resource table does not associate a culture with a resource, some other means should be used to infer the culture for a resource. One possible scheme is to embed information about the culture name in the resource name. For example, resource names for resources of the French culture could use the suffix “.fr”.

Cultures and Assembly Binding

Culture is part of the assembly identity and plays a part in the binding process. The runtime must always provide culture information as part of each bind request. The assembly cache manager never assumes or derives any culture information based on the current thread or environment.

A client retrieves resources for a specific culture by first retrieving the assembly for that culture. For the neutral culture, this is the main assembly of the application. For other cultures, the client retrieves the satellite assembly by passing the culture to Assembly.GetSatelliteAssembly(CultureInfo). Once the assembly is retrieved, a client can retrieve resources from the assembly by calling one of the Assembly.GetResource methods. The runtime looksup the specified resource in the resource table of the assembly’s manifest and returns the resource data as a managed stream.

The rules followed to bind to a satellite assembly are as follows:

  1. Satellite assemblies may be either simply named or strongly named subject to the constraint that if the main assembly is strongly named, all its satellite assemblies must also be strongly named.
  2. As with all assembly binds, the assembly cache manager will not honor version numbers for binds to simply named assemblies but will honor version numbers for binds to strongly named assemblies.
  3. As with all assembly binds, any appropriate policy will be applied to all binds to satellite assemblies.
  4. When binding to a strongly named satellite assembly, a resource manager should take the version number of the main assembly and request that version of a satellite assembly. The request should ignore the revision and build numbers of the version. To support the scenario where a QFE update is made to the main assembly, i.e., a change in revision and build numbers only, but the version of the satellite assembly remains unchanged, the assembly cache manager will allow binding to a revision+build that is older than the one requested.

If the main assembly’s simple name is “MyProject”, the runtime assumes that the satellite assembly’s simple name is “MyProject.resources”.

Naming Conventions

The NGWS runtime assumes that satellite assembies will have the file type “.mcl”. Resource names are assumed to be of the form:

typeNameSpace + Type.Delimiter + baseName + “.” + cultureName [+ “-” + subCultureName] + “.resources”

The culture specification is dropped when the resource is in a main assembly. Loose resource files in formats such as .GIF, .WAV, etc. follow a similar naming convention with the difference that they use the appropriate file type.

For example, a German resource with base name “MyResource” in namespace “MyNamespace” that is packaged in a satellite assembly would be named “MyNamespace.MyResource.de.resources”. The same resource packaged in a main assembly would be named “MyNamespace.MyResource.resources”. A .GIF localized for the German culture that is packaged in the satellite assembly would be named “MyNamespace.MyGif.de.resources”. A French (Belgium) resource would be named “MyNamespace.MyResource.fr-be.resources”.

The following example illustrates how resources and assemblies can be named in a project:

Project system:
Project1
Form1.cs
   Form1.resx
   Form1.de.resx
Intermediary files:
obj\form1.msil
obj\project1.form1.resources
obj\project1.form1.de.resources
Assembly outputs:
Main assembly:
simple name = project1
file name = project1.mcl
Manifest:
   name = project1
   culture = neutral
Resource table:
   project1.form1.resources
Satellite assembly:
simple name = project1.resources
file name = de\project1.resources.mcl
Manifest:
   name = project1.resources
   culture = de
Resource table:
   project1.form1.de.resources


Metadata API Overview

The metadata API provide methods to create and retrieve resources. The IMetaDataAssemblyEmit::DefineManifestResource method creates a ManifestResource data structure for a resource. The IMetaDataAssemblyImport::GetManifestResourceProps returns all the properties of a given resource. Detailed descriptions of these APIs can be found in Assembly Manifest Specification.


Assembly Class Overview

The Assembly class provides the following methods for retrieving resources:

public virtual Stream GetManifestResourceStream(Type type, String name);

This method returns a managed stream for the resource data given the logical name of the resource qualified by the namespace of the type. For example, if the logical name of the resource is MyResource and the fully qualified name of the type is MyNameSpace.Foo, GetResource will return the resource data for the resource named MyNameSpace.MyResource.

public virtual Stream GetManifestResourceStream(String name);

This method returns a managed stream for the resource data given the logical name of the resource.

public Assembly GetSatelliteAssembly(CultureInfo culture);

This method returns an assembly’s satellite assembly for the given culture.

public String[] GetManifestResourceNames();

This method returns the logical resource names of the resources defined in an assembly.


NGWS Runtime Resource Section

Resources embedded in a PE file are stored in a special NGWS runtime RC section. The NGWS runtime uses the NGWS runtime RC section instead of the Win32 RC section for the following reasons:

  • NGWS runtime resources are managed serializable objects.
  • NGWS runtime resources have a well defined format that allows an application to package a set of fine grain resources in the form of name/value pairs in a single coarse grain resources.
  • When NGWS runtime resources are indexed using an offset from the beginning of the resources section in the PE file. The runtime needs to stream the resources to the correct section of the PE file when writing the resources.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Using COM+ 1.0 Services from Managed Code Whitepaper

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Executive Summary

© 2000 Microsoft Corporation. All rights reserved.


COM+ 1.0 Services, NGWS Frameworks and Runtime, and VS7

Today customers use tools such as VB6 to build components. They install them in the Windows 2000 Component Services (aka COM+ 1.0 Services) or NT4 MTS to get services. Moving forward they are going to build the components in Visual Studio 7 and then install them in Windows 2000 Component Services to get services such as transactions, object pooling and activity semantics.

Using Visual Studio 7 and the NGWS frameworks and runtime provides a number of advantages, namely better tools and a much better runtime. The NGWS runtime managed languages are a lot better.

Using this new technology it will be possible to eliminate a lot of the legacy perf problems that VB used to have (e.g. apartment threading, heavy use of TLS) that enable them to take advantage of new Windows 2000 Components Services features like object pooling to build more scalable, higher performance applications.


Transactions

Developers targeting the NGWS runtime on Windows 2000 that want to use Component Services such as Transactions have a number of choices that are similar to what is provide today with Windows 2000 Component Services. The NGWS runtime provides Manual and Automatic Distributed transactions. Here is brief summary of the support that is detailed in this paper:

Manual Transactions

Today, developers can control Manual Transactions through BeginTransaction, CommitTransaction and AbortTransaction via the applicable Resource API, be that ADO, OLEDB, ODBC, MSMQ. They get explicit control over transaction. There is no co-ordination between data stores and no automatic enlistment in transactions across components. Transactions do not flow from user object to user object. This said, it is still a very valuable model and many customers want and need it. This programming model is also fully supported in the NGWS frameworks and runtime by:

  • ADO+ (ADO Interop Provider and SQL Server Provider – native TDS)
  • Messaging MSMQ Class Libraries (allowing MSMQ native transactions).

Automatic Distributed Transactions

One of major advantages of the Automatic Distributed Transaction programming model (Windows 2000 Component Services) is that data stores (the Resource Managers) automatically enlisted in transactions. When the user is happy they just call SetComplete or and if unhappy call SetAbort. The transactions can span multiple databases. This automatic use of transactions is very popular and powerful

Today, Windows 2000 Component Services (and MTS on NT4) provides hosting of components. Some of the service configuration (such as transactions) for the Components can be specified on the component (for classic COM components in the TLB or in the NGWS Components using custom attributes). For example the transaction configuration could be Not Supported, Supported, Requires or RequiresNew.

The Component service configuration can be set through the Component Services MMC snapin Admin tool or via the related Admin APIs.

Whenever a request is made to instanciate one of these objects (via CoCreateInstance, the request is received by the COM (be that ComponentServices or MTS) it looks up the configuration for the component and depending on the settings (e.g. RequiresNew Transaction) creates a context, initiates a transaction with the DTC (Distributed Transaction Coordinator), and dispatches calls to this object. If the object accesses databases (or other resources managers that use transactions e.g. MSMQ), the transaction will be flowed to the database that will enlist in the transaction with DTC. The database drivers (ADO, ODBC, OLEDB) lookup the transaction in the Component Services Context. If the object tries to create new objects and these objects have transaction requirements, the transaction is flowed to the new context and these new objects. This programming model is also fully supported in the NGWS by:

  • Windows 2000 Component Services (aka COM+ 1.0 Services). The NGWS component is hosted in Component Services and the programming model is MTS style with SetComplete / SetAbort
  • WebServices webmethods can be marked as TransactionRequired, TransactionRequiresNew etc
  • ASP+ pages can be marked as TransactionRequired, TransactionRequiresNew etc
  • ADO+ providers automatically enlist in Component Services transactions (ADO Interop Provider and SQL Server Provider – native TDS) initiated in ComponentServices.
  • Messaging MSMQ Class Libraries automatically enlist in transactions initiated in ComponentServices.
  • Existing Resource Managers (3rd parties, end users) that use the regular Windows 2000 Component Services APIs continue to work.

Model

In this model transactions work with both NGWS Managed components and regular COM+ 1.0 components. Customers can build NGWS managed components that use transactions. Customer can also flow transactions between the managed and regular COM+ 1.0 components:

  • Transactions can flow from COM+ 1.0 components to NGWS Managed components.
  • Transactions can flow from NGWS Managed components to COM+ 1.0 components.

This allows customers to upgrade parts of their COM+ 1.0 application without upgrading everything i.e. leave some components in VB6 and move some components to VB7.

They can also write new NGWS components and these will participate in automatic distributed transactions. They can just mark their managed class with a custom attribute i.e. [TransactionAttribute(RequiresNew)] and be done. When it is time to create a new instance of the object a transaction is created. When a database operation was started the transaction would be flowed to the database, which would enlist in the transaction with DTC. NGWS components and classic COM components support the same model.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Data

© 2000 Microsoft Corporation. All rights reserved.


ODBC, OLEDB, ADO

Today existing ODBC, OLEDB, ADO and the related drivers and providers support manual and automatic transactions. The resource managers for ODBC and OLEDB check if they are executing in an automatic Component Services transaction and if so, enlist the database in the transaction automatically.


ADO+

ADO+ Managed Provider. The ADO+ Managed Provider uses native OLEDB through COM Interop to enable data access.

The OLEDB provider supports both manual transactions and automatic trasnsactions. For automatic transactions the OLEDB provider automatically enlists in the transaction and obtains the transaction details from the Component Services Context.

public class System.Data.ADO.ADOConnection
{
  //. . .
  public int BeginTransaction();
  public void CommitTransaction();
  public void RollbackTransaction()
  //. . .
}

ADO+ SQL Server Managed Provider (TDS). This adapter talks direct TDS to SQL Server. It does not use OLEDB, ADO or ODBC. It is all managed code.

The SQL Managed Adapter uses Microsoft.ComServices ContextUtil.GetObjectObject InTransaction to determine if it should enlist the SQL Server associated with its Connection in the transaction. If so, it obtains the Transaction from the COM+ Services Context and enlists the SQL Server.

public class System.Data.SQL.SQLConnection
{
  public int BeginTransaction();
  public int BeginTransaction(String transactionName);

  public void CommitTransaction();
  public void RollbackTransaction();
  public void RollbackTransaction(String transactionName);
  public void SaveTransaction(String savePointName);
  int TransactionLevel {get;}
}


ASP+ and Transactions

Scenario: An ASP+ Page written in a NGWS runtime managed language (VB7) and updates multiple databases. The managed page is hosted in an ASP+ Worker process.

ASP+ supports the following Transaction directives for use on pages. The level of transaction support for the page is indicated with a Transaction directive, as shown in the following example:

<@% Transaction=”Required” %>

The default transaction state is none, which indicates that the transaction context will be ignored by the ASP+ framework. There is no explicit directive for none (e.g. Transaction=”None”) since the absence of the directive indicates this.

Transaction directive

Directive | Explanation
Required | The page requires a transaction. It will run in the context of an existing transaction, if one exists. If not, it will start one.
Requires_New | The page requires a transaction and a new transaction will be started for each request.
Supported | The page will run in the context of an existing transaction, if one exists. If not, it will run without a transaction.
Not_Supported | This value indicates that the page does not run within the scope of transactions. When a request is processed, its object context is created without a transaction, regardless of whether there is a transaction active.


Explicitly Completing Transactions

A transaction can be explicitly committed or aborted using Microsoft.ComServices. Page developers can explicitly call the SetComplete or SetAbort members to commit or abort an ongoing transaction.

// try to do something crucial to transaction completing
if( !DoSomeWork() )
{
  ContextUtil.SetAbort();
}


WebServices and Transactions

Scenario: An ASP+ WebMethod written in a NGWS runtime language (VB7) updates multiple databases. The managed page is hosted in an ASP+ Worker process.

ASP+ provides built-in support for creating and exposing web services – using a programming abstraction that is consistent and familiar to both ASP+ Web Forms developers and existing VB users. The resulting model is both scalable and extensible – and embraces open Internet standards (HTTP, XML, SOAP, SDL) so that it can be accessed and consumed from any client or Internet enabled device.

Like ASP+ Web Forms, ASP+ WebServices optionally enable developers to execute their code within the context of a MTS Transaction (for more information on the ASP+ Transaction Story, please see topics on ASP+ Transactions). This will ensure that all interactions with Resource managers – SQL Servers, MSMQ Servers, Oracle Servers, SNA Servers, etc – maintain the ACID (atomic, consistent, isolated, durable) properties required to run robust distributed applications.

WebService developers indicate that they wish to run in the context of a MTS transaction using a custom enum property – Transaction – on the WebMethod metadata attribute of a webmethod. This will ensure that a new MTS Transaction is created whenever the webmethod is activated from the client.

For example, the below web service exposes a single webmethod – “DeleteDatabase” – that performs a database operation that is scoped within the context of a MTS Transaction:

<%@ WebService Language=”C#” %>

using System;
using System.Data;
using System.Data.SQL;
using System.Web.Services;

public class TransactionStuff : WebService
{
  [WebMethod(Transaction=Transaction.Required)]
  public void DeleteDatabase()
  {
    SQLConnection sqlConn = new SQLConnection(
                          "FooBar", "sa", "", "northwind");
    SQLDataSetCommand sqlAdapter1 = new SQLDataSetCommand(
                         "delete orders", sqlConn);
    SqlConn.Execute();
  }
}

Important: Note that the transaction will only be created if the activating webmethod (ie: the method called from the client) has the transaction metadata. It will not be created if the webmethod on which the webservice is first created is not marked with the appropriate transaction metadata (even if a method that has the transaction metadata is subsequently invoked from it).


Messaging with MSMQ and Transactions

Scenario: Managed objects written in a NGWS runtime language (VB7) can send and receives messages from a MSMQ Queue.

MSMQ is a technology that allows users to take advantage of Message Queues. MSMQ Message Queues allow for: Create/Delete message queues; Send/Receive messages; Manage message queues.

Transactions are an important part of enterprise systems. MSMQ supports full DTC transactions for sending and receiving messages. MSMQ also supports internal MSMQ transactions. These transactions have limited abilities, but are optimized for performance in specific scenarios.


Internal Transactions

Internal transactions are simpler thant DTC transactions. They are called internal because they are handled entirely inside the MSMQ code. They don’t rely on any support from MS DTC or any other transactional mechanism.

Internal transactions have a very simple programming model. Assuming, I want to receive, in at most two seconds, either two messages or not to receive any messages at all, I can write the following code:

try
{
   mq.BeginTransaction();
messageA = mq.Receive(1000);
messageB = mq.Receive(1000);
mq.CommitTransaction();
}
catch(Exception e)
{
mq.RollbackTransaction();
}

This is pretty simple. The problem is that internal MSMQ transactions cannot include database operations. So, for example if I would like to receive a message from a message queue and put the message in a database – all of these as one transaction – I would have to use external transactions.

namespace System.Messaging
{
  public class MessageQueue
  {
    //. . .
    public void BeginTransaction();
    public void CommitTransaction();
    public void RollbackTransaction();
    //. . .
  }
}
Method | Explanation
BeginTransaction | Creates a new Microsoft Message Queue Server (MSMQ) internal transaction.
CommitTransaction | Commits a pending internal transaction.
RollbackTransaction | Rolls back the pending internal transaction.


External Transactions

External transactions are supported through Windows 2000 Component Services. This transactional mechanism allows you to intermix MSMQ and database operations in one single transaction. The MSMQ Resource Manager automatically enlists in Component Services Transactions.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Component Services and Transactions

© 2000 Microsoft Corporation. All rights reserved.


Component Services Scenarios

Bank Sample Components

The bank sample has four components:

Bank. Uses MoveMoney to transfer money between two accounts.

MoveMoney. Transfers money between two accounts. Subtracts from one, then adds to another. Hosted in Windows 2000 Component Services and requires a transaction.

Account. Posts a change in money to a specified account using ADO+. This can be positive or negative. Hosted in Windows 2000 Component Services and requires a transaction.

Database. Tables that hold bank account details including balances in a database.

Upgrade Scenarios

One scenario of steps to consider:

  1. Start. All the class COM components are written in VB6.
  2. Bank is upgraded to a NGWS runtime language such as VB7 or C#.
  3. MoveMoney is upgraded to a NGWS runtime language.
  4. Account is upgraded to a NGWS runtime language.

Another scenario of steps to consider:

  1. Start. All the class COM components are written in VB6.
  2. MoveMoney is upgraded to a NGWS runtime language.

Calling between Classic COM and NGWS components

Classic COM component calling a NGWS Component

  • A CCW (Com Callable Wrapper) provides a COM Proxy that makes the NGWS component look to classic COM as a regular classic COM object. The CCW is used to make the transition between unmanaged code and managed code.
  • The NGWS SDK tools TLBEXP is used to generate the TLB from the NGWS Components MetaData.
  • Classic COM supports interfaces with methods and properties. Classic COM does not support fields, statics or type hierarchies, so these elements on NGWS components are not accessible from classic COM.
  • The default interface can be thought of as the class interface.
  • For the classic COM component to communicate with the NGWS component, the NGWS component needs to registered with classic COM.
  • For the NGWS Component to be hosted in Windows 2000 Component Services it needs to be configured in the Windows 2000 Component Services catalog.

NGWS Component calling a Classic COM component

  • A RCW (Runtime Callable Wrapper) provides a Managed “Proxy” that looks like a NGWS component to other runtime code. The RCW is used to make the transition between managed code and unmanaged code when the NGWS component calls the classic COM Component.
  • Classic COM supports interfaces with methods and properties. Classic COM does not support fields, statics or type hierarchies.
  • Just like today, the classic COM component is registered with classic COM.
  • For the classic COM Components to be hosted in Windows 2000 Component Services it needs to be configured in the Windows 2000 Component Services catalog.
  • For the NGWS Component to be hosted in Windows 2000 Component Services it needs to be registered in the Windows 2000 Component Services catalog.

NGWS Component calling NGWS Component

  • NGWS Components support a high fidelity object model: class hierarchies, interfaces, methods, properties, fields and statics.
  • Today classic COM components have the restriction that only interfaces elements (methods and properties) are supported. For PDC Tech Preview, NGWS Components hosted in Windows 2000 Components Services have the same restriction i.e. only interface elements (methods, properties) are callable. The NG Component may still have fields, class hierarchies etc, but they are not accessible from other NGWS Components. In Product Release, the full fidelity (class hierarchies, interfaces, methods, properties, fields, etc) of the object model will be available. This is detailed in the paper below.
  • For the NGWS Component to be hosted in Windows 2000 Component Services it needs to be registered in the Windows 2000 Component Services catalog.

Statics

The behavior of static methods (and static fields) is pretty much the same in PDC Tech Preview and Product Release. Unmanaged clients can never access managed statics.

Managed clients can access managed statics directly (note that statics are scoped per AppDomain), subject to accessibility controls like private/public, of course. Such access never goes through any kind of remoting boundary, including Windows 2000 Component Services Contexts (aka COM+ 1.0 contexts). Indeed, the access is always to a local type. So the access bypasses any semantics associated with the context.

Historically, MTS and COM+ 1.0 programmers are aware of the fact that global functions are part of their implementation, and they are unrelated to the context boundaries. They are also aware of the fact that they can accidentally leak references across boundaries using globals. In the managed world, most of those restrictions remain for statics.

However, things will be slightly brighter in this regard, after the planned enhancements for ComEmulate is done for the product release (ComEmulate is a custom attribute used to mark types (classes) that enable COM+ 1.0 Services on manage code classes.

At that point, references will go through a transition or not, based on whether the thread is already in the correct context. It won't be possible to leak a direct reference via a static, because in the managed world the clients will *never* have a truly direct reference to an instance of a ComEmulate type.


Microsoft.ComServices

Overview

The purpose of the components and interfaces contained in Microsoft.ComServices is to provide access to COM+1.0 services from a managed environment. All methods raise the standard NGWS runtime exceptions for error handling.

This version of the specification is targeted for PDC of the NGWS frameworks and runtime and hence W2K platforms only.

The goal of Microsoft.ComServices for PDC is to enable objects in the managed world to make use of COM+ services and to participate in services with both managed and unmanaged objects.

The first scenario is for two managed objects to participate in a transaction initiated from a managed client. In this case, both managed classes must be implemented as COM components including wrapper classes using the COMEmulate attribute. Further transactional attributes are added to the classes. Then the classes are registered in the COM+ catalog using the tool described in this document.

A further scenario is using one managed object and one COM+ object within one transaction. This can be achieved in a similar way by implementing the managed object as a component and registering the managed component.

Sample code for the first scenario is shown below and the process required to deploy and run the components.

// -----------------------------------------------------------------
// Component.cs
// -----------------------------------------------------------------
using System;
using System.Runtime.InteropServices;
using Microsoft.ComServices;

[guid("CE2A4888-82BD-40ad-9BBA-8D129747886F")]
[InterfaceTypeAttribute(ComInterfaceType.InterfaceIsIDispatch)]
public interface IAccount
{
   void Debit(int amount);
}

[guid("557B96C4-F7E6-4d48-BBC3-10981E0CA664")]
[ComEmulate("Component.Account")]
[TransactionAttribute(TransactionOption.Required)]
public class EmulAccount
{
}

[guid("871D8D5C-ECC6-40a5-BD82-D5E3B1740095")]
internal class Account : IAccount
{
  public void Debit(int amount)
  {
     //… do some Db work
     if (ok) ContextUtil.SetComplete();
     else ContextUtil.SetAbort();
  }
}
// -----------------------------------------------------------------
// Client.cs
// -----------------------------------------------------------------
using System;
using Component;

class client
{
  public static int Main()
  {
    IAccount accountX = (IAccount)new EmulAccount();
    accountX.Debit(100);
    return 0;
  }
}
// -----------------------------------------------------------------
// Compile and run procedure
// -----------------------------------------------------------------
sn -k TestKey.snk
csc /debug /a.keyfile:Testkey.snk /t:library /r:Microsoft.ComServices.dll /out:Component.dll Component.cs
Create a new COM+ application in the COM+ catalog or use an existing name)
RegSvcs Component.dll SomeApp
al /I:Component.dll
csc /I:Component.dll client.cs
start client.exe

Note: The ASP intrinsic objects can be obtained within a NGWS object using the named properties available from the NGWS object context when the object was created from ASP. With ASP+ and the NGWS frameworks and runtime, a new set of instrinsic objects are used which is not accessed from a context but rather methods in a namespace, much like the ContextUtil class. The case of legacy COM+ objects created from ASP+ is viable and therefore the instrinsic objects available to legacy COM objects from the COM+ object context may be required so that legacy COM+ object can interact with ASP+. This scenario does not fall into the category of this spec. However, the scenario of managed objects created from ASP would imply that a wrapper would be required to access ASP instrinsics from a managed object. This would typically be a method in the ContextUtil class facilitating the access to these instrinsics. This appears to be an unusual scenario which will not be covered by this spec at this time.

Deployment and Configuration

Interface IRegistrationHelper

A registration component and interface facilitates the registration of a managed assembly either from a managed or an unmanaged environment. The component is accessed through an interface (IRegistrationHelper) or the class (RegistrationHelper). The interface and class is for internal use only, however, clients can use the command line tool that essentially wraps this private class.

InstallAssembly
Syntax | void InstallAssembly(String assemblyName, String applicationName, String tlbName);
Description | Leverages the functionality of regAsm, tlbExp, tlbreg and then configures the NGWS application based on attributes in the assembly.
RegisterAssembly
Syntax | void RegisterAssembly(String assemblyName, String applicationName);
Description | Leverages the functionality of regAsm, tlbExp and tlbreg.
ConfigureAssembly
Syntax | void ConfigureAssembly(String assemblyName, String applicationName);
Description | Configures the NGWS application based on attributes in the assembly.
UninstallAssembly
Syntax | void UninstallAssembly(String assemblyName, String applicationName);
Description | Leverages the functionality of regAsm, tlbExp, tlbreg and then unconfigures the NGWS application based on attributes in the assembly.

Regsvcs.exe

The tool uses the InstallAssembly method on the IRegistrationHelper interface. The tlb file name default is the same as the dll name unless explicitly named in the third and optional parameter. Command line help is obtained from Regsvcs /? or Regsvcs /h

regsvcs assemblyName.dll applicationName [tlbName.tlb]

The regsvcs tool goes through the following steps:

  1. Loads the assembly
  2. Registers the assembly
  3. Generates a type library
  4. Registers the type library
  5. Installs the type library into the requested application
  6. Configures the components

Possible failures during each corresponding step could be:

  1. If the assembly loading fails, the error message "Failed to load assembly assemblyName.dll" is displayed. It should be followed by an explanation.
  2. If type registration fails, a list of the exception messages that were generated is displayed.
  3. If type library generation fails, a list of exceptions should also be listed. For both steps 2. and 3. the most likely problems will be TypeLoadExceptions, an incorrectly specified ComEmulate attribute on one of the types can be the cause, or referenced assemblies can't be found.
  4. An exception is displayed. This step uses the oleaut LoadTypeLibrary call for registration.
  5. If this step fails, the most likely cause is that the application cannot be found. If the error "One of the objects could not be found" is displayed, first make sure that the type library listed exists and if it does, make sure that the application exists. The most likely problem is that the target application does not exist.
  6. The most likely error in this step could be a mismatched set of properties on one of the components. Most likely it is a SynchronizationAttribute being specified that conflicts with either a default (such as JustInTimeActivation, which is enabled by default), or with a specified transaction. If any transaction setting other than TransactionOption.Ignored is specified, Synchronization must be either Requires or RequiresNew.

Attributes

The following attributes will be supported in ComServices for PDC.

public enum TransactionMode
{
    Ignored = 0,
    None = 1,
    Supported = 1,
    Required = 2,
    RequiresNew = 3
}
[AttributeUsage(AttributeTargets.Class)]
public class TransactionAttribute : Attribute
{
  public TransactionAttribute();
  public TransactionAttribute(TransactionMode policy);
}

public enum SynchronizationMode
{
    Ignored = 0,
    None = 1,
    Supported = 1,
    Required = 2,
    RequiresNew = 3
}
[AttributeUsage(AttributeTargets.Class)]
public class SynchronizationAttribute : Attribute
{
  public SynchronizationAttribute();
  public SynchronizationAttribute(SynchronizationMode policy);
}

[AttributeUsage(AttributeTargets.Class)]
public class JustInTimeActivationAttribute : Attribute
{
  public JustInTimeActivationAttribute();
  public JustInTimeActivationAttribute(bool mode);
}

[AttributeUsage(AttributeTargets.Class)]
public class MustRunInClientContextAttribute : Attribute
{
  public MustRunInClientContextAttribute();
  public MustRunInClientContextAttribute(bool mode);
}

Shared properties

Microsoft.ComServices

Constants

Enum PropertyLockMode
{
    LockSetGet = 0;
    LockMethod = 1;
}
Enum PropertyReleaseMode
{
    Standard = 0;
    Process = 1;
}
enum TransactionVote
{
    int Commit = 0
    int Abort = Commit + 1
}

Microsoft.ComServices.SharedPropertyGroupManager

This is the standard SharedPropertyGroupManager component found in COM+ services. New instances are created programmatically.

Methods

SharedPropertyGroupManager
Syntax | SharedPropertyGroupManager ()
Description | Creates an instance of a PropertyGroupManager component.
CreatePropertyGroup
Syntax | SharedPropertyGroup CreatePropertyGroup (String name, PropertyLockMode lockMode, PropertyReleaseMode releaseMode, bool alreadyExists)
Description | Creates a PropertyGroup component corresponding to the COM+1.0 equivalent.
Parameters | String name: See COM+ documentation. PropertyLockMode lockMode: See COM+ documentation. PropertyReleaseMode releaseMode: See COM+ documentation. bool alreadyExists: See COM+ documentation.
Return values | A SharedPropertyGroup object.
Group
Syntax | SharedPropertyGroup Group(String name)
Description | Accesses the SharedPropertyGroup object in the collection associated with name.
Parameters | String name: The name of the shared property group to access.
Return values | A SharedPropertyGroup object indexed by name.
GetEnumerator
Syntax | IEnumerator GetEnumerator()
Description | Retrieves the enumerator for accessesing the SharedPropertyGroupManager collection.
Return values | An IEnumerator interface

Microsoft.ComServices.SharedPropertyGroup

This is the standard SharedPropertyGroup component found in COM+ services. Instances of this component are not directly created but obtained from a call to SharedPropertyGroupManager.CreatePropertyGroup.

Methods

CreateProperty
Syntax | SharedProperty CreateProperty (String name, bool alreadyExists)
Description | Creates a SharedProperty component corresponding to the COM+1.0 equivalent.
Parameters | String name: See COM+ documentation. bool alreadyExists: See COM+ documentation.
Return values | A SharedProperty object for the newly created or existing property.
CreatePropertyByPosition
Syntax | SharedProperty CreatePropertyByPosition (int index, bool alreadyExists)
Description | Creates a SharedProperty component corresponding to the COM+1.0 equivalent.
Parameters | Int index: See COM+ documentation. bool alreadyExists: See COM+ documentation.
Return values | A SharedProperty object for the newly created property.
PropertyByPosition
Syntax | SharedProperty PropertyByPosition (int index)
Description | See COM+ documentation.
Property
Syntax | SharedProperty Property (String name)
Description | See COM+ documentation.

Microsoft.ComServices.SharedProperty

This is the standard SharedProperty component found in COM+ services.

Properties

Value
Syntax | Object Value {get; set;}
Description | See COM+ documentation

Shared property usage

SharedPropertyGroupManager spgm = new SharedPropertyGroupManager();
SharedPropertyGroup groupEarl = spgm.CreatePropertyGroup(“Earl”, PropertyLockMode.LockSetGet, PropertyReleaseMode.Standard, alreadyExisits);

spgm.Group(“earl”).Item(“salary”).Value = 100000000;
groupEarl.PropertyByPosition(1).Value = 100000000;
for each SharedPropertyGroup group in spgm
   group.Property(“salary”).Value = 1000
next group

Contexts

Microsoft.ComServices.ContextUtil

This component provides access to COM+ context objects. New instances are not created programmatically.

Methods

SetComplete
Syntax | static void SetComplete ()
Description | Calls SetCompete on IObjectContext for the calling thread.
SetAbort
Syntax | static void SetAbort ()
Description | Calls SetAbort on IObjectContext for the calling thread.
EnableCommit
Syntax | static void EnableCommit()
Description | Uses the COM+ call on IObjectContext for the calling thread.
DisableCommit
Syntax | static void DisableCommit()
Description | Uses the COM+ call on IObjectContext for the calling thread.
GetNamedProperty
Syntax | static Object GetNamedProperty(String name)
Description | Accesses additional context properties specific to ASP, Application, Session, Request, Response and Server objects.

Properties

DeactivateOnReturn
Syntax | static void DeactivateOnReturn {get; set}
Description | Calls SetDeactivateOnReturn and GetDeactivateOnReturn on IContextState in COM+
MyTransactionVote
Syntax | static void MyTransactionVote {get; set;}
Description | Calls SetMyTransactionVote and GetMyTransactionVote on IContextState in COM+
ObjectContext
Syntax | static IObjectContext ObjectContext {get;}
Description | Retrieves the IObjectContext interface pointer for the current context
Transaction
Syntax | static Object Transaction {get;}
Description | Calls GetTransaction on IObjectContextInfo in COM+.
Return values | The returned object supports the ITransaction interface detailed below
IsInTransaction
Syntax | static bool IsInTransaction {get;}
Description | Calls IsInTransaction on IObjectContextInfo in COM+
IsTransactionDoomed
Syntax | static bool IsTransactionDoomed {get}
Description | Provides information about the current transaction’s status
TransactionId
Syntax | static GUID TransactionId {get;}
Description | Calls GetTransactionId on IObjectContextInfo in COM+
ContextId
Syntax | static GUID ContextId {get;};
Description | Calls GetContextId on IObjectContextInfo in COM+
ActivityId
Syntax | static GUID ActivityId {get;}
Description | Calls GetActivityId on IObjectContextActivity in COM+

Microsoft.ComServices.IObjectContext

This is the same interface defined in COM+

Inteface IObjectContext
{
    Object CreateInstance(Guid rclsid, Guid riid);
    void  SetComplete();
    void  SetAbort();
    void  EnableCommit();
    void  DisableCommit();
    bool  IsInTransaction();
    bool  IsSecurityEnabled();
    bool  IsCallerInRole(String role);
}

Microsoft.ComServices.ITransaction

This is the same interface defined in the DTC interfaces. This interface is implemented by the object returned in ContextUtil.Transaction.

Interface ITransaction
{
    void  Commit(int retaining,
                 int grfTC,
                 int grfRM);
    void  Abort(BOID reason,
                int retaining,
                int async);
    void  GetTransactionInfo(out XACTTRANSINFO  info);
}

Microsoft.ComServices.BOID

This component serves as the same structure defined in the DTC interfaces.

class BOID extends System.ValueType
{
    int[] rgb;
}

Microsoft.ComServices.XACTTRANSINFO

This component serves as the same structure defined in the DTC interfaces.

class XACTTRANSINFO extends System.ValueType
{
    BOID unitOfWork;
    int isolationLevel;
    int isolationFlags;
    int grfTCSupported;
    int grfRMSupported;
    int grfTCSupportedRetaining;
    int grfRMSupportedRetaining;
}

Contexts usage

ContextUtil.SetAbort();
ContextUtil.DeactivateOnReturn = true;
Object object = ContextUtil.GetNamedProperty(“Session”)
Isession session = (Isession)object;
session.Timeout = 1000;
session[“someVariableName”] = 4;

Security

Microsoft.ComServices.SecurityCallContext

This component provides access to COM+ security call context and is similar but not identical to the SecurityCallContext object in Visual Basic. New instances are not created programmatically but obtained through the CurrentCall method.

Methods

IsCallerInRole
Syntax | bool IsCallerInRole(String role)
Description | Calls IsCallerInRole on ISecurityCallContext.
Parameters | String role: See COM+ documentation.
Return values | See COM+ documentation.
IsUserInRole
Syntax | bool IsUserInRole(String user, String role)
Description | Calls IsUserInRole on ISecurityCallContext.
Parameters | Object user: See COM+ documentation. String role: See COM+ documentation.
Return values | See COM+ documentation.

Properties

CurrentCall
Syntax | static SecurityCallContext CurrentCall {get;}
Description | Returns a reference to a SecurityCallContext object associated with the current call.
SecurityEnabled
Syntax | bool SecurityEnabled {get;}
Description | Calls IsSecurityEnabled on ISecurityCallContext.
DirectCaller
Syntax | SecurityIdentity DirectCaller {get;}
Description | Retrieves the DirectCaller item from the ISecurityCallContext collection in COM+ and returns the item as a SecurityIdentity object.
OriginalCaller
Syntax | SecurityIdentity OriginalCaller {get;}
Description | Retrieves the OriginalCaller item from the ISecurityCallContext collection in COM+ and returns the item as a SecurityIdentity object.
Callers
Syntax | SecurityCallers Callers {get;}
Description | Retrieves the Callers item from the ISecurityCallContext collection in COM+ and returns the item as a SecurityCallers object.
NumCallers
Syntax | int NumCallers {get;}
Description | Retrieves the NumCallers item from the ISecurityCallContext collection in COM+
MinAuthenticationLevel
Syntax | int MinAuthenticationLevel {get;}
Description | Retrieves the MinAuthenticationLevel item from the ISecurityCallContext collection in COM+

Microsoft.ComServices.SecurityIdentity

This component accesses ISecurityIdentityColl in COM+ and provides the items in the ISecurityIdentityColl collection as properties. New instances are not created programmatically but obtained from components SecurityCallContext and SecurityCallers.

Properties

SID
Syntax | Object SID {get;}
Description | Retrieves the SID item from ISecurityIdentityColl in COM+
AccountName
Syntax | String AccountName {get;}
Description | Retrieves the AccountName item from ISecurityIdentityColl in COM+
AuthenticationService
Syntax | Int AuthenticationService {get;}
Description | Retrieves the AuthenticationService item from ISecurityIdentityColl in COM+
ImpersonationLevel
Syntax | Int ImpersonationLevel {get;}
Description | Retrieves the ImpersonationLevel item from ISecurityIdentityColl in COM+
AuthenticationLevel
Syntax | Int AuthenticationLevel {get;}
Description | Retrieves the AuthenticationLevel item from ISecurityIdentityColl in COM+

Microsoft.ComServices.SecurityCallers

This component accesses ISecurityCallersColl in COM+ and provides the items in the ISecurityCallersColl collection as indexed properties. New instances are not created programmatically but obtained from the SecurityCallContext component. The component also implements the IEnumerable interface.

Methods

GetEnumerator
Syntax | IEnumerator GetEnumerator()
Description | Retrieves the enumerator for the ISecurityCallersColl collection

Properties

Count
Syntax | Int Count {get;}
Description | The number of indexed SecurityIdentity objects
this
Syntax | SecurityIdentity this[int index]
Description | An indexed property used to retrieve a specified item in the security callers collection.

Security usage

SecurityCallContext call = SecurityCallContext.CurrentCall;
foreach (SecurityIdentity id in call.Callers)
{
    Console.WriteLine("Caller: " + id.AccountName);
}

SecurityCallers callers = SecurityCallContext.CurrentCall.Callers;
For (int i = 0; i < callers.Count; i++)
{
    Console.WriteLine("Caller: " + callers[i].AccountName);
}

Object construction

Microsoft.ComServices.IObjectControl

This is the same interface defined in COM+

Interface IObjectControl
{
    void Activate();
    void Deactivate();
    bool CanBePooled();
}

Microsoft.ComServices.IObjectContruct

This is the same interface defined in COM+

Interface IObjectConstruct
{
    void  Construct(Object constructorObject);
}

Microsoft.ComServices.IObjectContructString

This is the same interface defined in COM+

Interface IObjectConstructString
{
    String ConstructString { get; }
}

Object pooling

Object pooling is supported on legacy COM+ components called from a managed environment and on registered managed components with the ComEmulate attribute. Simply set the object pooling properties in the COM+ catalog by script, code or the component services explorer.

Services not yet included

  • QC
  • DTC/TIP transactions
  • Events
  • CRM
  • Resource dispenser
  • COM Admin


Hosting classes in Windows 2000 Component Services

Hosting NGWS Components in PDC Tech Preview

In order to mark a managed type in such a way that it can get Component Services, a managed type can be marked with a new custom attribute ComEmulate. When users mark NGWS Components classes as being ComEmulate, they can leverage Windows 2000 Component Services, including transactions, activities, JIT activation, object pooling, and so forth.

Managed clients talk to all such instances via Interop. When the marked NGWS managed class is instantiated from managed code (newobj, Activator.CreateInstance, . . .), the EE will activate the object via CoCreateInstance.

For PDC TechPreview, when the CCW is re-introduced to managed code, we never unwrap it. (Normally a CCW over a managed object in the same appdomain / subprocess is unwrapped to produce a direct managed reference). Instead, we produce an RCW over the CCW. Then all calls between a managed client and one of these managed servers go through COM Interop. This allows the Windows 2000 Component Services Context engine to have full control.

Key Points

  • NGWS ComEmulate classes created via interop under the covers i.e. CoCreateInstance
  • Managed Interfaces with public methods and properties are exposed to managed code just as though they were classic COM components. (PDC Tech Preview restriction, restriction removed in Product Release)
  • NGWS ComEmulate Classes accessed via COM Interop, even from managed code - have methods, properties - no fields. (PDC Tech preview restriction, restriction removed in Product Release)
  • NGWS ComEmulate Classes Statics are never accessed via COM Interop. They are still available from managed code to managed code, even from managed code. (PDC Tech Preview restriction, restriction removed in Product Release). However, since statics have no relation to the instance method or instance state, they execute independent of the context i.e. are always accessed directly and locally. The programmer must follow Windows 2000 Component Services rules for accessing global variables and passing interfaces into and out of globals. If in a ComEmulate class and you place yourself in a global, it can only be accessed from things in that context.
  • NGWS ComEmulate Classes & Threading Model.
    • Managed objects are currently registered as Both, they behave as though they have aggregated the Free Threaded Marshaler.
    • NGWS ComEmulate classes behave as though they have not aggregated the Free Threaded Marshaler
  • NGWS ComEmulate classes are created in the current appdomain of the executing thread.

Today, MTS and Windows 2000 Component Services applications need to be registered and configured. ComEmulate enables NGWS Components to utilizes Windows 2000 Component Services. Just as today, with class COM components there are restrictions on deployment and configuration.

Hosting NGWS Components in Product Release

In Product Release some of the restrictions will disappear. In particular, the type would be a full managed type with inheritance, fields, etc. It's as natural as any other managed type that inherits from MarshalByRefObject.

All the operational nuisances related to the current ComEmulate types (either author a 2nd type or use special switches to TLBIMP the managed type) would also disappear. The operational nuisances that would remain are the same ones that exist today for classic COM types. Specifically, you must register these things into the Component Services catalog, thereby breaking xcopy deployment. And you retain many of the other Component Services restrictions, like threading limitations.

However, it's worth pointing out that some things are better than classic COM. For example, you don't have to worry about leaking references between contexts by storing them in statics. The runtime would make it impossible to leak, because the instance remains fully wrapped at all times. Also, we can perhaps make memory management simpler than with Component Services, particularly if we use the trick of making the cycle through COM Interop visible to the GC, in one direction.

We can also address the performance hit of pumping calls through COM Interop, and the lack of fidelity caused by marshaling arguments through COM Interop. Specifically, once we do the DCR work, we can efficiently short-circuit the case where the thread is already in the correct unmanaged context. And we think we may be able to pass the call through COM Interop without marshaling any arguments -- if the client and server are in the same managed process. This is a trick we've used in the past.

The lack of xcopy deployment is a serious problem, despite it already being a problem with existing COM+ 1.0 apps. It has been discussed about making xcopy deployment work for the case where the client is managed code. When JITting the "new MyComponent()", or when loading the class MyComponent, we would notice that the component is marked as ComEmulate. So we could register it with the catalog at that time, assuming that it has all the necessary custom attributes for TransactionAttribute(Requires), etc.

Unmanaged clients don't get xcopy deployment, of course, so it would be nice if Tools could assist with the deployment of these components so that they are automatically configured upon installation. The DCR work for ComEmulate types is the first step, but it would be great to have Tools to help out -- for instance by deploying users' apps and by providing wizards.

Finally, the final managed transaction story will be not significantly different than the COM+ 1.0 story based on ComEmulate. The fact that Component Services group is doing the work, and the fact that it will be built on top of the Runtime Transparent Proxies etc, means we can pretty much predict what the final managed story is going to look like. It's going to look a lot like where we'll be after we implement that DCR.

Steps to Host a managed class in Windows 2000 Component Services

Step1 – write the managed class

The elements in gray are needed for PDC Tech Preview

The elements in yellow are needed for Product Release

BankClassLibrary.cs

PDC Tech Preview | Product Release
In PDC Tech Preview two classes are needed. The class with ComEmulate emulates the internal class. | In Product Release only one class will be needed.
namespace BankClassLibrary

{

namespace BankClassLibrary

{

[guid(“1237e50f-6085-49f7-8293-3febb8580e78”)]

[InterfaceTypeAttribute(ComInterfaceType.InterfaceIsIDispatch)]

public interface IMoveMoney
{

/*

lngPrimeAccount -   "From" Account
lngSecondAccount -  "To" Account
lngAmount -         Amount of transaction
lngTranType -       Transaction Type
1 = Withdrawal,
2 = Deposit,
3 = Transfer)
Returns:  String -        Account Balance
*/

  public String Perform (
                  int lngPrimeAccount,
                  int lngSecondAccount,
                  int lngAmount,
                       int tranType);

}

public interface IMoveMoney
{

/*

lngPrimeAccount -   "From" Account
lngSecondAccount -  "To" Account
lngAmount -         Amount of transaction
lngTranType -       Transaction Type
1 = Withdrawal,
2 = Deposit,
3 = Transfer)
Returns:  String -        Account Balance
*/

  public String Perform (
                  int lngPrimeAccount,
                  int lngSecondAccount,
                  int lngAmount,
                       int tranType);

}

[TransactionAttribute(TransactionOption.Required)]

[Guid(“b963fbbd-d1b1-4a2f-8501-89252dcb116f”)]
[ComEmulate(“BankClassLibrary.MoveMoneyInternal\0”)]
public class MoveMoney
{

}

internal class MoveMoneyInternal : IMoveMoney
{
  public String Perform (
         int lngPrimeAccount,
         int lngSecondAccount,
         int lngAmount,
           int tranType)
  {
    // Use IAccount to move money
    // between two accounts
  }

}

[TransactionAttribute(TransactionOption.Required)]
[ComEmulate()]
public class MoveMoney : IMoveMoney
{
  public String Perform (
         int lngPrimeAccount,
         int lngSecondAccount,
         int lngAmount,
           int tranType)
  {
    // Use Account to move money
    // between two accounts
  }

}

[Guid(“41ee971a-93f9-4111-89ee-4e0f4e0fca0a”)]

[InterfaceTypeAttribute(ComInterfaceType.InterfaceIsIDispatch)]

public interface IAccount
{
   public String Post (
              int lngAccountNo,
              int lngAmount);

}

public interface IAccount
{
   public String Post (
              int lngAccountNo,
              int lngAmount);

}

[TransactionAttribute(TransactionOption.Required)]

[Guid(“41ee971a-93f9-4111-89ee-4e0f4e0fca0a”)]
[ComEmulate(“BankClassLibrary.AccountInternal\0”)]
public class Account
{
}

internal class AccountInternal
{
  public String Post(
          int lngAccountNo,
          int lngAmount)
  {
    // Use ADO+ to post money into
    // the account in the database.
  }

}

[TransactionAttribute(TransactionOption.Required)]
[ComEmulate()]
public class Account
{
  public String Post(
          int lngAccountNo,
          int lngAmount)
  {
    // Use ADO+ to post money into
    // the account in the database.
  }

}

[Guid(“0bc94b5e-311d-4106-aa5f-70ace030ebc4”)]

[ComEmulate(“BankClassLibrary.TransferInternal\0”)]
public class Transfer
{
}

internal class TransferInternal : IObjectControl
{
  public String DoSomeWork(
          int i,
          int j)
  {
    // do some interesting work
  }

  void Activate()
  {
     //. . .
  }

  void Deactivate()
  {
     //. . .
  }

  bool CanBePooled()
  {
     //. . .
  }
}

[ComEmulate()]

public class Transfer
{
}

internal class TransferInternal : IObjectControl
{
  public String DoSomeWork(
          int i,
          int j)
  {
    // do some interesting work
  }

  void Activate()
  {
     //. . .
  }

  void Deactivate()
  {
     //. . .
  }

  bool CanBePooled()
  {
     //. . .
  }
}

Step2 – Write the managed Bank App

The App code does not change between PDC Tech Preview and Product Release.

MyBankApp.cs

using System;
using BankClassLibrary;

public MyApp
{
  public static Main(String[] args)
  {
    MoveMoney mm = new Foo();
    IMoveMoney imm = (IMoveMoney)mm;

    //Transfer $1000 from account 12345 to account 20566
    String balance = imm.Perform (
                       12345,
                  20456,
                  1000,
                       3);
  }
}

Step3 – Deployment and Configuration - Host the Managed classes in Component Services

A Managed Assembly that needs to be part of a COM+ Library or Server Application  needs have a strong name and placed in the Assembly cache.

Steps to add a managed assembly to a COM+ Server Application.

Description
Command Line

Generate a key for the Assembly

sn -k TestKey.snk

Compile the Assembly passing in the key

Csc /debug /a.keyfile:Testkey.snk /t:library /r:Microsoft.ComServices.dll /out: BankClassLibrary.dll BankClassLibrary.cs

Register the Assembly in a NGWS Application

RegSvcs BankClassLibrary.dll SomeApp

Place the Assembly in the Assembly Cache

al /I:BankClassLibrary.dll

Optionally configured using Component Services Admin Tools

For the product release the following capabilities will be enabled.

Product Release

The NGWS runtime will Dynamically Generate TLB on managed “new” or dropping into Component Services Admin Tools

TheNGWS runtime will Dynamic Register on managed “new” or dropping into Component Services Admin Tools.

Optionally configured using Component Services Admin Tools

Managed types that have managed context may be deployed and configured via NGWS configuration files. They require no registration and take full advantage of the xcopy deployment features of the NGWS, if all clients are managed.

Note: For PDC Tech Preview, Managed managed types that are marked as ComEmulate must be configured and registered just as in Component Services today.

Key Points

PDC Tech Preview
Product Release

  • Classes which are ComEmulate will be registered in the COM+1.0 Catalog.
    •

  • Classes which are ComEmulate may be registered in the COM+1.0 Catalog.
    •

  • DllRegisterServer with “ComEmulate” clases is not supported, use RegAsm to register Assemblies (RegAsm is the new name for ComReg) or via the APIs that RegAsm uses to register managed classes.
    •

  • We can do some work to make this easier.
    •

  • Classes that are ComEmulate can optionally be configured though the Windows Component Services Admin Tool.
    •

  • Classes that are ComEmulate can optionally be configured though the Windows Component Services Admin Tool.
    •

  • Exposed simple Managed Custom Attribute to allow user to mark their classes to need certain services e.g. TransactionsRequired (similar to marking Transactions in an IDL file today).
    •

  • Exposed simple Managed Custom Attribute to allow user to mark their classes to need certain services e.g. TransactionAttribute(Required) similar to marking Transactions in an IDL file today.
    •

  • The Component Services could be made aware more aware of NGWS Components, by looking for additional custom attribute on the managed classes. This will help users significantly.
    •

  • Programmatically manipulate the COM+ 1.0 Catalog via the COM Admin Interfaces to set Transactions etc.
    •

  • Programmatically manipulate the COM+ 1.0 Catalog via the COM Admin Interfaces to set Transactions etc.
    •

  • XCOPY deployment only works if a managed client activates the managed component.  Under those circumstances, we have enough information to register the component when we JIT the callsite.  But if an unmanaged client does a CoCreate, there is no reasonable pathway from the CLSID to the managed component.  So XCOPY deployment doesn't work in that case.  It is hoped that VS Tools might spit out an installer for components marked with ComEmulate, that properly puts them into the catalog.
    •

Step4 – Run the managed application

Execute MyBankApp.exe

Security

The NGWS runtime supports role-based security mechanism similar to those provided in the earlier MTS and COM+1.0 services (see “Role-based Security in NGWS”).  The NGWS runtime mechanisms were designed as an integral part of the managed application programming model and are specifically geared toward the needs of Internet-scale applications.  These are summarized below.

We recognize that one may wish to write managed code extending an existing application using COM+1.0 security services.  To accommodate this need, we provide mechanisms to integrate managed code with COM+1.0 security services.  The NGWS and COM+1.0 role-based security mechanisms are independent and only one should be used within a single application.

NGWS Role-based security

NGWS role-based security services are integrated into managed code runtime environment.  The model provides a great deal of flexibility while still being easy to use.  The mechanisms are platform independent and do not assume any specific authentication or OS account infrastructure.

Key features include:

  • NGWS role-based security mechanisms are automatically available to all managed code applications.  There is no required application registration or administration.
    •

  • NGWS roles and identities may be application defined or alternately mapped to existing OS accounts and groups.  Application can choose which approach best meets their needs.
    •

  • The runtime will automatically create a Principal object, which encapsulates role and identity information, for each AppDomain execution thread.  By default this will represent an unauthenticated user with no roles.  Alternately, policy may be set to have this reflect the OS identity the application code is running on behalf of with group membership mapped to roles.
    •

  • Application code may handle Principal object creation if the default behavior isn’t appropriate.  This operation is limited to trustworthy code granted the SecurityPermission( ControlPrincipal) permission by the local Code Access Security Policy.
    •

  • Role or identity checks may be incorporated into managed code using either declarative or imperative mechanisms.
    •

COM+ 1.0 Security Interoperability

Support is provided for managed code developers who need to integrate with COM+1.0 security.  The primary reason using this mechanism is to extend existing COM+1.0 applications using managed code.  This functionality is only available on Windows2000 and relies on the NT token associated with executing code as the basis for identity.

To simplify use of COM+1.0 security, we provide a managed code wrapper so a managed application can easily access the COM+1.0 ISecurityContext.  Applications using this mechanism will need to:

  • Mark any types accessing this functionality with the ComEmulate custom attribute
    •

  • Register these types with the COM+1.0 services registration database
    •

Developers should be cognizant of the following issues:

  • Managed types marked as ComEmulate are indistinguishable from standard COM objects, hence any managed callers are required to have permission to call Unmanaged code.
    •

  • COM+1.0 security relies on NT accounts and process/thread impersonation.  If the managed code provides authentication services it will need to obtain an NT security token and do an impersonation prior to calling any COM objects.
    •

  • Any calls to managed types not marked as COMEmulate are opaque to COM+1.0 security services.  That is, they are treated no differently than calls from COM objects to Win32 APIs in unmanaged DLLs.
    •

One can also use the earlier MTS security functionality available on NT 4 with the Option Pack installed. We do not, however, provide managed code objects to assist in using MTS security and using applications will need to be aware of the additional limitations imposed relative to COM+1.0.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Visual Studio

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Visual Studio Compilers

VB

C#

Managed C++

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Visual Studio Tools

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Visual Studio IDE

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Visual Studio Enterprise Frameworks

© 2000 Microsoft Corporation. All rights reserved.


Configuration File Syntax

The application configuration file is an XML file that contains a handful of different directives that can control the assembly binding behavior for the application. This section lists and describes each of the directives that apply to versioning, and provides an example configuration file.


Terminology Used

The terminology used in this document is consistent with the terminology used in the XML file that describes the configuration file syntax.

A collection defines the beginning of a set of rows, and is delineated by start and end tags, such as: <tag> and </tag>.

A row is an entry inside a collection, which contain specific settings in the form of name-value pairs.

A property is a name-value pair inside a row that contains the setting information.


Collections Supported

The outer-most collection that defines the configuration file is the Configuration collection. Each of the sub-collections and rows described below appears between the start and end tags of the Configuration collection.


AppDomain Row

Under the Configuration collection, an AppDomain row may exist which contains information that can help the Assembly Resolver when it probes different locations for assemblies. Specifically, the AppDomain row contains properties that can be used to specify a private binpath that is appended to the binpath provided by the application context during a bind operation.

Example:

<AppDomain PrivatePath="[semicolon delimited private binpath]"/>


BindingMode Collection

The BindingMode collection contains a single row that describes the “binding mode” for the application. This row, named AppBindingMode has a single property called Mode that can be set to one of two values: “safe” and “normal”.

Example:

<BindingMode>
   <AppBindingMode Mode="[safe|normal]"/>
</BindingMode>

If a value for the Mode property is not “safe” or “normal”, the binding mode is assumed to be “normal”.


BindingPolicy Collection

The BindingPolicy collection contains BindingRedir rows that redirect one version of an assembly to another. This version redirection, or policy only applies to shared assemblies (ie. assemblies which contain a public key). The BindingRedir row contains a number of properties, which are listed below:

Property Name | Supported Values | Description
Name | Any string | Name of the assembly
Originator | String form of originator (8-byte hash of public key) | Originator of the assembly
Version | String form of version: w.x.y.z, or “*” for all versions. | Version which should be redirected.
VersionNew | String form of version: w.x.y.z | Version which the assembly should be redirected to.
UseLatestBuildRevision | “yes” or “no” | Indicates whether or not this assembly should be subject to automatic QFE updates from the global cache.

Example:

<BindingPolicy>
   <BindingRedir Name="[name]" Originator="[orig]" Version="[ver|*]"
                 VersionNew="[ver]" UseLatestBuildRevision="[yes|no]"/>
</BindingPolicy>


Assemblies Collection

The Assemblies collection contains one or more CodeBaseHint rows, which can tell the Assembly Resolver where to find a particular assembly. These codebase “hints” are always attempted before the standard probing URLs are attempted. The CodeBaseHint row contains a number of properties which describe the assembly, as well as the location where the assembly can be found:

Property Name | Supported Values | Description
Name | Any string | Name of the assembly
Originator | String form of originator (8-byte hash of public key) | Originator of the assembly
Version | String form of version: w.x.y.z | Version of assembly
CodeBase | String: valid URL | Codebase URL where the assembly can be found

Example:

<Assemblies>
   <CodeBaseHint Name="[name]" Originator="[orig]" Version="[ver]"
                 CodeBase="[URL]"/>
</Assemblies>


Configuration File Sample

<?xml version ="1.0"?>

<Configuration>

   <AppDomain
       PrivatePath="bin;bin2;foo"
    />

   <BindingMode>
      <AppBindingMode Mode="normal"/>
   </BindingMode>

   <BindingPolicy>
      <BindingRedir Name="g_SglAsm_SglMod1.dll"
                    Originator="8e47bf1a5ed0ec84"
                    Version="*" VersionNew="3.3.3.3"
                    UseLatestBuildRevision="no"/>
   </BindingPolicy>

   <Assemblies>
      <CodeBaseHint Name="g_SglAsm_SglMod1.dll"
                    Originator="8e47bf1a5ed0ec84"
                    Version="3.3.3.3"
                    CodeBase="http://codebasehint/foo.dll"/>
   </Assemblies>

</Configuration>


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Configuring the NGWS Runtime in the PDC Tech Preview

© 2000 Microsoft Corporation. All rights reserved.


NGWS Runtime Configuration

The primary goal of this document is to describe how the NGWS runtime and applications that target the Runtime are configured in the Tech Preview. The development of the NGWS configuration system is ongoing and only portions are available in this Tech Preview. As such, a secondary goal of this document is to provide an introduction to the full configuration system that will be implemented in future milestones.

The purpose of the NGWS configuration system is to allow administrators and developers to configure those aspects of an application that are either specific to the deployment environment, or can logically be changed after the application was built.

A typical example of configuration data is security policy. When building an assembly, a developer specifies the set of permissions that are required for the code to run. However, the decision about whether to grant the assembly the requested permissions is up to the administrator and is made independent of when the assembly was built.

In the Tech Preview an administrator will be able to configure a) security policy, at both the machine and user level; b) application behavior; and c) runtime behavior. In addition, a developer or support engineer will also be able to configure various runtime settings in order to debug problems.

When the runtime is installed, there are defaults for all of these settings so that an administrator does not have to take any specific action to configure the system to run applications. The following sections provide the details on how to configure the various aspects of the Runtime and applications that target the Runtime. There are three appendices that provide sample files that demonstrate how to configure various aspects of the NGWS runtime.


Goals of NGWS Configuration

The goals and requirements of the configuration system from the Runtime's perspective include:

Storage of rich data types. The settings used to configure the NGWS runtime do not all have the same size and shape. Some of the data is static and can readily be represented as a relational schema. However, some of the data is stored as serialized objects. In addition, configuration data in NGWS is user extensible, and therefore its contents are not natively known to the NGWS frameworks and runtime. The configuration system must support the storage of these "flavors" of data.

Flexibility. NGWS developers must be able to add configuration data to the system without going through a central authority. Additionally, end user customers must be able to leverage the configuration systems for customer-specific settings as well (i.e the DSN for my app).

Xcopy deployment. The Runtime makes xcopy deployment feasible through self-describing applications. The configuration system should not hamper this goal.

Hierarchical Configuration. It is often useful for administrators to specify configuration information at various levels of a hierarchy. Example levels include machine, user, application and assembly. The configuration system must support the ability to specify configuration at multiple, custom defined levels, and must provide for custom merge rules to be supplied.

Secure storage of secrets. Portions of the configuration data managed by the Runtime are secrets that must be secured against unwarranted access and tampering. Examples of secrets include passwords and private keys. The configuration must provide for secure storage of these types of data.

Configuration data security. In addition to storing secrets, the configuration system must protect all configuration data from against unwarranted access and tampering.

Integration with Admin Tools. The interfaces defined by the configuration system must support both read and write access and be efficiently usable by Admin tools (for example, they must be provided in managed code).

Varying Application Scenarios. The NGWS frameworks and runtime team has identified three primary application scenarios for the Tech Preview. These are an ASP+ application, a client executable launched from the shell, and code that is downloaded and executed in a web browser. The configuration system and corresponding admin tools must support all three scenarios. In addition, NGWS is both a client and a server platform. The configuration system must not emphasize one at the expense of the other.


Security Policy

Security policy allows the administrator to restrict the permissions granted to code running on the machine. The policy system looks at evidence about all managed code – where it comes from, name or publisher identity, and so forth – as the basis of a trust decision about the code, that results in a set of permissions the system is willing to grant the code. Security policy may be configured at the machine and user levels in the Tech Preview.

The details of how to configure security policy are given in Runtime Code Access Security Administration.


Application Behavior

By default, all information that the runtime needs to locate, load, and execute an application is available either implicitly (e.g., the location where you started the application establishes the default search path) or with the manifest for each assembly that's loaded as a part of that application For example, by default, each assembly binds to the versions of its dependencies it was compiled with as specified in its manifest.

However, there are several application settings that may be configured either on an application-by-application basis, or at the machine level, for all applications launched on that machine. There are rules for how settings at each of these levels are merged to obtain a single set of configuration settings for a given application. The rules for each setting are given in the description of the settings that follow.

The configuration settings available in the Tech Preview are:

  • Application specific search path - the path that the runtime uses to locate assemblies that are referenced during the execution of an application. The application specific search path is relative to the application path.
  • Version policy -- for a given application, it is possible to specify as policy what version of a particular assembly should be used (or that the application should run in "safe mode", using exactly those versions that it was compiled with)
  • Codebase - explicit information about where to find a referenced assembly.
  • NGWS Runtime Remoting – the information necessary for the remoting infrastructure to properly expose and consume remote types and assemblies.

Excluding NGWS runtime Remoting, these configuration settings are located in one of two places:

  • The machine wide configuration file located at %windir%\admin.cfg where %windir% is the Windows directory.
  • The application specific configuration file located in the same directory as the application that is executing. The filename is the same name as the application with a .cfg extension instead of an .exe extension. For example, the configuration file for xyz.exe is xyz.cfg.

NGWS runtime remoting currently uses its own configuration file (detailed in the NGWS runtime remoting section). In the future, NGWS runtime Remoting configuration information will be included in the configuration files mentioned above.


Application Specific Search Path

A common deployment paradigm used in NGWS is to simply xcopy all the assemblies that make up an application into a particular directory structure and run the app. In this scenario, the NGWS runtime needs to know where to look to resolve references to private assemblies. Private assemblies are those assemblies that typically have simple names and are deployed in the directory structure of the application in which they are used. We do have a default search path that is sufficient for most cases, but this can be customized through configuration if desired. The application specific search path is typically used to find private, simply named assemblies. However, shared named assemblies can also be found on the application specific search path.

To change the application specific search path, <AppDomain> tags are added to the configuration file. Each <AppDomain> tag has a PrivatePath property that contains semicolon-delimited paths relative to the application path.

For more information about the <AppDomain> tag or to see examples, please look at Configuration File Syntax.


Version Policy

Sometimes, it is useful or necessary to tell an application to use a specific version of an assembly. These policies are established by entries in the configuration files.

A policy can state that:

“References to assembly XZY, version 1.0 should be bound to version 1.2”

When the runtime needs an assembly, these policies are reviewed to determine if the runtime should use the originally targeted version or a different version of the assembly.

For detailed information about configuring version policy, please look at Versioning in NGWS.


Codebases

A configuration can supply Codebases, providing locations for particular assemblies. If a codebase is specified for an assembly, the NGWS runtime will get the assembly directly from that codebase and not search for it on disk.

Unlike version policy, codebases are specified using only the application specific configuration files and do not included in the machine wide configuration file.

To specify a codebase for an assembly, <CodeBaseHint> tags are added to the <Assemblies> collection in the application configuration file. Each <CodeBaseHint> tag has several properties that describe that the assembly name, version, originator, and the URL where the assembly can be found.

For more information about the <CodeBaseHint> tag or to see examples, please look at Configuration File Syntax.


NGWS Runtime Remoting

NGWS runtime Remoting allows an application to be configured so that the assemblies used by the application can be part of the local application or part of a remote application. If an assembly is configured to be part of the remote application, the types in that assembly will be instantiated in the remote application and a “proxy” will be returned to the local application.

The NGWS runtime remoting configuration is used to specify where a type or assembly lives, what kind of communication is used to talk to the remote type, and the lifetime of the remote type.

For detailed information about configuring NGWS runtime remoting, please look at Remoting Configuration.


Runtime Behavior

The preceding sections have concentrated on how to configure various aspects of applications that target the NGWS runtime. There are also a number of configuration settings that apply to the Runtime itself. These settings are primarily used by developers and support engineers that build tools for the NGWS runtime or diagnose problems.

These include:

  • Developer Services, including debugging and profiling
  • JIT Settings

Both of these currently have their own methods for storing their settings. Eventually, both Developer Services and JIT settings will be stored in the application and machine configuration files mentioned in the Application Behavior section.


Developer Services

The developer services settings allow developers to turn on and off debuggers and profilers, enable edit and continue, and configure various optimization settings that affect generated code. These settings are described in detail in Enabling Profiling and Debugging.


JIT Settings

Various JIT settings can be changed using the JitMan.exe tool included in the SDK. These settings are:

  • Use the EconoJit and set the percentage of time spent jitting code vs. running code
  • Limit the size of the Code Cache and set the maximum size of the Code Cache in bytes
  • Optimize jitting for size instead of execution speed
  • Enable Concurrent GC so that threads running user code are not used for garbage collection

For documentation on the JitMan.exe tool, look at Jit Manager Utility.


Assembly Cache

Each machine on which the NGWS runtime is installed has a machine wide code cache called the assembly cache. The assembly cache serves three primary purposes:

  • It is a store for code downloaded from the internet or other http or fileservers. Code downloaded on behalf of a particular application is stored in the "private" portion of the cache to prevent it from being seen and used by others.
  • It is a repository for components that are shared among many applications on the machine, or for those applications wishing to restrict a user's ability to see and accidentally delete files. Those assemblies explicitly installed into the cache are stored in the "global" portion of the cache. These assemblies are visible to all applications on the machine.
  • Its holds native code versions of NGWS runtime assemblies that have been pre-jitted for optimal performance

The SDK includes a shell extension called the Assembly Cache Viewer. The shell extension exposes the assemblies stored in the global cache through the Windows Explorer. Through this extension, assemblies can be added to or deleted from the cache. The extension also allows you to view an individual assembly’s properties.

More documentation on the Assembly Cache Viewer is available in Assembly Cache Viewer.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Mini-Spec on Custom Attributes

© 2000 Microsoft Corporation. All rights reserved.


Overview of Custom Attributes:

Custom attributes add user-defined annotations to the metadata. Custom attributes allow an instance of a type to be stored with any element of the metadata. This mechanism can be used to store application specific information at compile time and access it either at runtime or when another tool reads the metadata. While any user-defined type can be used as an attribute it is expected that most attributes will be instances of types whose parent is System.Attribute. The NGWS runtime predefines some attribute types and uses them to control runtime behavior. Some languages predefine attribute types to represent language features not directly represented in the VOS. Users or other tools are welcome to define and use additional attribute types.


Emitting the Right Metadata

This is covered in the metadata specifications. "http://comrtime/specs/Formats/MiniSpec%20Metadata%20Extensibility.doc"


System.Attribute

public class Attribute
{

protected Attribute ();

public static Attribute   GetCustomAttribute (MemberInfo    element, Type attributeType);
public static Attribute   GetCustomAttribute (ParameterInfo element, Type attributeType);
public static Attribute   GetCustomAttribute (Module        element, Type attributeType);
public static Attribute   GetCustomAttribute (Assembly      element, Type attributeType);
public static Attribute[]   GetCustomAttributes (MemberInfo    element, Type attributeType);
public static Attribute[]   GetCustomAttributes (ParameterInfo element, Type attributeType);
public static Attribute[]   GetCustomAttributes (Module        element, Type attributeType);
public static Attribute[]   GetCustomAttributes (Assembly      element, Type attributeType);
public static Attribute[] GetCustomAttributes(MemberInfo    element);
public static Attribute[] GetCustomAttributes(ParameterInfo element);
public static Attribute[] GetCustomAttributes(Module        element);
public static Attribute[] GetCustomAttributes(Assembly      element);
public static bool IsDefined (MemberInfo    element, Type attributeType);
public static bool IsDefined (ParameterInfo element, Type attributeType);
public static bool IsDefined (Module        element, Type attributeType);
public static bool IsDefined (Assembly      element, Type attributeType);
}

All Attributes inherit (directly or indirectly) from System.Attribute. This common base class gives all Attributes a central point in the hierarchy from which they can inherit. Compilers and other development tools use this information to identify which types are Custom Attributes

Just inheriting from System.Attribute gives an attribute certain default behaviors: Allowed on any element, Inheritable, duplicate instances are not allowed.

GetCustomAttriubte

Returns an instance of customAttributeType or null if non-exists on the given attributedElement. If customAttributeType is a base class or interface this method will return any implementation of subclass of that type. Throws an AmbiguousMatchException if there is more than one attribute of type customAttributeType defined on this member. Ignores any attributes that don’t inherit from System.Attribute.

GetCustomAttriubtes

Returns an instance of each custom attribute defined on the given attributedElement or an empty array if non-exists. If customAttributeType is a base class or interface this method will return any implementation of subclass of that type. Returns an empty array if non-custom attributes exists on this the given element. Ignores any attributes that don’t inherit from System.Attribute. If a type is specified, the array returned must be an array of that type.

IsDefined

Returns true if this attribute is defined on the given element (including inherited members)


System.AttributeUsageAttribute

[AttributeUsageAttribute (AttributeTargets.Attribute)]
public sealed class AttributeUsageAttribute : Attribute
{
   public AttributeUsageAttribute (AttributeTargets validOn);
public AttributeUsageAttribute (AttributeTargets validOn, bool inherited, bool allowMultiple); //Obsolete
public bool AllowMultiple { virtual get; virtual set;}
public bool Inherited { virtual get; virtual set; }
public AttributeTargets ValidOn { virtual get; }
}

This attribute allows some customization of how the attribute it is attached to is treated.

ValidOn

This property indicates on which elements the attribute may be attached. It is legal to OR multiple targets together to get the right combination. For example the following would indicate that the attribute is allowed to be applied to any Type or any Methods.

[AttributeUsage (AttributeTargets.Type | AttributeTargets.Method)];
public class FooAttribute : Attribute
{
}

If this attribute were applied to Property, the compiler would give an error.

[FooAttribute]  //OK
public class CheckingAccount
{
   [FooAttribute] //OK
   public void TransferFunds (String name) {…};

   [FooAttribute] //Error! Balance is a property
   public Decimal Balance {get {…}};
}

See System. AttributeTargets for more information.
The default is AttributeTargets.All.

Inherited

This property indicates if the attribute is to be inherited by subclasses or not. Compilers provide no special support for this functionality; it is the job of the attribute consumers (Reflection, etc) to respect this information. If allowMultiple is true, the attributes accumulate on the derived member; if allowMultiple is false, these attributes will override (or replace) in inheritance.

For example the BarAttribute is inherited but the BazAttribute is not.

//Defaults to Inherited=true
public class BarAttribute : Attribute
{
}

[AttributeUsageAttribute (Inherited = false)];
public class BazAttribute : Attribute
{
}

The BarAttribute is inherited but the BazAttribute is not.

public class Account
{
   public void TransferFunds (String name) {…};

   [BarAttribute]
   [BazAttribute]
   public Decimal Balance {get {…}};
}

public class CheckingAccount : Account
{
    //Reflection will show Balance as having the BarAttribute
//but not the BazAttribute.
    public Decimal Balance {get {…}};
}

Notice, only Attributes on Types, Methods, Properties, Events, Parameters are inherited.

See Reflection Support for more information.
The default is Inherited = true.

AllowMultiple

This property indicates if more than one instance of the attribute is allowed to exist on the same element. In heritance, if this value is true, the attributes accumulate on the derived member; if the value is false, these attributes will override (or replace) in inheritance.

//Defaults to AllowMultiple=false
public class BarAttribute : Attribute
{
}

[AttributeUsageAttribute (AllowMultiple = true)];
public class AuthorAttribute : Attribute
{
    public AuthorAttribute (String name) {…};
}

The BarAttribute is not allowed to be duplicated but the BazAttribute is.

public class Account
{
   [BarAttribute]
   [BarAttribute] //Error: duplicates are not allowed.
   public void TransferFunds (String name) {…};

   [AuthorAttribute (“Bob Jones”)]
   [AuthorAttribute (“Mary Jane”)]
   public Decimal Balance {get {…}};
}

See Reflection Support for more information.
The default is AllowMultiple = false.


System. AttributeTargets

enum AttributeTargets
{
    Assembly     = 0x0001;
    Module       = 0x0002;
    Class        = 0x0004;
    Enum         = 0x0008;
    Struct    = 0x0010;
    Constructor  = 0x0080;
    Method       = 0x0100;
    Property     = 0x0200;
    Field        = 0x0400;
    Event        = 0x0800;
    Interface    = 0x0020;
    Parameter    = 0x1000;
    Delegate     = 0x1001;

    ReturnType = 0x1003;

    All          = Assembly | Module | Class | ValueType |
                   Enum | Constructor | Method | Property |
                   Field | Event | Interface | Parameter |
                   Delegate | MulticastDelegate |
                     ReturnType;
}

This enum indicates which elements an attribute is allowed to be applied to. See ValidOn for more dertails.


Reflection Support

Reflection supports returning custom attributes defined directly on a member (non-inherited) thought the ICustomAttributeProvider interface. This interface is defined on MemberInfo, ParamaterInfo, Module and Assembly.

interface ICustomAttributeProvider
{
public object[] GetCustomAttributes(Type attributeType);
public object[] GetCustomAttributes();
public bool IsDefined (Type attributeType);
}

GetCustomAttriubtes (Type)

Returns an array of instances of customAttributeType or null if non-exists on this member. If customAttributeType is a base class or interface this method will return any implementation of that type. Throws an AmbiguousMatchException if there is more than one attribute of type customAttributeType defined on this member. Does NOT do any inheritance walk. The array returned must be an array of the type specified.

GetCustomAttriubtes

Returns an instance of each custom attribute defined on the given on this member or an empty array if non-exists. If customAttributeType is a base class or interface this method will return any implementation of that type. Does NOT do any inheritance walk.

IsDefined

Returns true if one or more instance of attributeType is defined on this member.


Reflection Emit Support

A custom attribute class is built in the same way any other class is built, using DefineType off ModuleBuilder.

Reflection Emit supports defining custom attributes directly on a member thought the ICustomAttributeBuilder interface. This interface is defined on MethodBuilder, TypeBuilder, ConstructorBuilder, ParamaterInfo, ModuleBuilder and Assembly.

interface ICustomAttributeBuilder
{
public void DefineCustomAttribute(ConstructorInfo attributeType, byte [] attributeData);
}

public class CustomAttributeHelper
{
    public static byte [] GetAttributeData (ConstructorInfo attributeType);
    public static byte [] GetAttributeData (ConstructorInfo attributeType, Variant [] paramaterValues);
    public static byte [] GetAttributeData (ConstructorInfo attributeType, Variant [] parameterValues, String [] members, Variant [] memeberValue);
    public static byte [] GetAttributeData (Object attributeInstance);
}

DefineCustomAttriubte

Defines a custom attribute on this member represented by attributeType with the data from attributeData. See the metadata spec "http://comrtime/specs/Formats/MiniSpec%20Metadata%20Extensibility.doc" for details on how to lay out the attributeData or use the methods on CustomAttributeHelper to build the byte array.

GetAttributeData

Returns the byte [] needed for DefineCustomAttribute() given the constructor from the attribute type, the values for that constructor and the names and values for the properties or fields of the custom attribute.


Data Marshaling Specification

This document highlights the marshaling services provided by the NGWS runtime. These services are responsible for marshaling data across the managed and unmanaged code boundary. Both COM Interop and PInvoke use this service.


The Need for Marshaling

As method calls are made between managed and unmanaged code, the data associated with the call often requires marshaling. The data to be marshaled includes the method arguments as well as the method return value. Marshaling is required whenever the caller and the callee have different representations for the data being passed between them.

To better understand why the marshaling service is needed, consider the following method definition written in C#:

[DllImport("UTILITY.DLL", EntryPoint="PrintMessage")]
public static extern void PrintMessage(String msg);

This method uses the Runtime’s Platform Invocation Service (PInvoke) to call the unmanaged method PrintMessage from managed code. The DllImport attribute provides the name of the unmanaged DLL containing the implementation of PrintMessage as well as the entry point within that DLL. The method has a single string parameter and could be called like this:

System.String msg = "Hello World";
PrintMessage(msg);

The type System.String is a managed type that stores the characters of the string as a buffer of Unicode characters. The String class dynamically manages the size of the buffer internally so the programmer using the class doesn’t have to. When an instance of the String type is declared, as in the example above, the variable msg refers to an instance of the String class not to the character buffer stored within the class.

Now consider the unmanaged implementation of the PrintMessage routine. The definition may look like this if it were written in C++:

void PrintMessage(LPSTR msg);

It expects a pointer to an ANSI character buffer to be passed to it, not a reference to a String class. If fact, it doesn’t even know what the String Type is because it was built with and compiler that isn’t even NGWS aware. The string argument being passed therefore needs to be converted from a System.String object to a buffer of ANSI characters. It’s the runtime’s marshaling service that’s responsible for performing this conversion.

The marshaling service has built-in rules that govern the conversions that take place for each type. For the String type for example, the marshaler will automatically copy the contents of String classes internal Unicode character buffer into a new buffer containing ANSI characters. Other types have other marshaling characteristics. In spite of the built-in rules, it is often necessary to be more explicit about the exact marshaling behavior that is needed. For example, consider a different implementation of PrintMessage that expects a buffer of Unicode characters to be passed instead of an ASNI character buffer.

In that case the managed method definition could be attributed with the MarshalAs attribute explicitly control the msg argument.

[DllImport("UTILITY.DLL", EntryPoint="PrintMessage")]
public static extern void PrintMessage(
   [MarshalAs(UnmanagedType.LPWSTR)]String msg);

The MarshalAs attribute tells the marshaler the format of the data that is expected on the unmanaged side of the boundary. The marshaler then ensures that the data is copied and transformed to the correct format in the unmanaged space. In this case the data would be copied to a buffer containing Unicode characters.

The runtime’s marshaling service is responsible for performing the conversions needed to represent the data accurately on both sides of the managed and unmanaged boundary.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

General Marshaling Behavior

© 2000 Microsoft Corporation. All rights reserved.


P/Invoke vs COM Interop

The runtime’s marshaler is used by both PInvoke and COM Interop. Both services provide interoperability with unmanaged code and therefore both services rely heavily on the marshaler. The two services are similar, the main difference being that PInvoke is used to access static DLL entry points while COM Interop is used to interact with COM interfaces. At the lowest levels, both service use the same marshaler. For details on these two services, see the COM Interoperability Specification and the Platform Invocation Services User Guide.

In general, the marshaler provides the same features to both COM Interop and PInvoke. However, there are some data types that are only supported by PInvoke or COM Interop. Those types are identified throughout his document.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Element Types

© 2000 Microsoft Corporation. All rights reserved.


Isomorphic Types

Most data types have common a representation in both the managed and unmanaged worlds. These types are considered Isomorphic because they do not require conversion when marshaled between the managed and unmanaged world.

Table 1 shows the complete list of isomorphic element types supported by the runtime along with the respective language types used to represent those element types in VB, MC++ and C#.

Isomorphic Types | VB | MC++ | C#
ELEMENT_TYPE_PTR | N/A | void | N/A
ELEMENT_TYPE_CHAR | char | char | char
ELEMENT_TYPE_I1 | N/A | char | sbyte
ELEMENT_TYPE_UI1 | Byte | unsigned char | byte
ELEMENT_TYPE_I2 | integer | short | short
ELEMENT_TYPE_UI2 | N/A | unsigned short | ushort
ELEMENT_TYPE_I4 | long | int, long | int
ELEMENT_TYPE_UI4 | N/A | unsigned int, unsigned long | uint
ELEMENT_TYPE_I8 | long8 | __int64 | long
ELEMENT_TYPE_U8 | N/A | unsigned __int64 | ulong
ELEMENT_TYPE_R4 | single | float | Float
ELEMENT_TYPE_R8 | double | double | Double
ELEMENT_TYPE_R | N/A | N/A | N/A
ELEMENT_TYPE_I | N/A | N/A | N/A
ELEMENT_TYPE_U | N/A | N/A | N/A
ELEMENT_TYPE_FNPTR | N/A | N/A | N/A

Table 1. Isomorphic element types and their respective language representation

Some complex types are also isomorphic. For example, single dimensional arrays of isomorphic types are also isomorphic and formatted value types that contain only isomorphic types are also isomorphic.


Non-Isomorphic Types

Non-Isomorphic have different or sometimes ambiguous representations in the managed and unmanaged worlds. These types are considered Non-Isomorphic because they may require conversion when marshaled between managed and unmanaged code. For example, managed strings are non-isomorphic types because they can have several different unmanaged representations, some of which may require conversion. Table 3 shows the complete list of non-isomorphic element types supported by the runtime along with the respective language types used to represent those element types in VB, MC++ and C#. Section 5 of this document describes how to use the MarshalAs attribute to control the marshaling behavior for non-isomorphic types.

Isomorphic Types | VB | MC++ | C#
ELEMENT_TYPE_BOOLEAN | Boolean | Bool | Bool
ELEMENT_TYPE_CHAR |  |  |
ELEMENT_TYPE_STRING | String | String | String
ELEMENT_TYPE_OBJECT |  |  |
ELEMENT_TYPE_CLASS | Class | Class | Class
ELEMENT_TYPE_VALUETYPE | UDT | [value] class | struct
ELEMENT_TYPE_ARRAY |  |  |
ELEMENT_TYPE_SZARRAY |  |  |

Table 3. Non-Isomorphic element types and their respective language representation


By Value vs By Reference Types

In general, when marshaling method arguments, value types are marshaled to unmanaged code as a value on the stack whereas reference types are marshaled as pointers on the stack. Classes (except Object), Interfaces, Strings, and Arrays are all considered reference types and are therefore marshaled as pointers. For example,

Method arguments can also be passed by reference. For marshaling purposes, passing arguments by-reference simply adds a level of indirection to the unmanaged type. For example, a managed integer is marshaled as 4-byte value on the stack, while a by-reference integer is marshaled as a pointer to an integer on the stack. When a reference type is passed by reference, another level of indirection is added to the unmanaged signature. For example,

The following code shows some examples of how marshaling is effected by the use of ByRef parameters:

Mgd Signature: void Method(long i);
Unmgd Signature: void Method(long i);
Mgd Signature: void Method(ref long i);
Unmgd Signature: void Method(long *i);
Mgd Signature: void Method(Object o);
Unmgd Signature: void Method(VARIANT o);
Mgd Signature: void Method(ref Object v);
Unmgd Signature: void Method(VARIANT *v);
Mgd Signature: void Method(IList il );
Unmgd Signature: void Method(IList *il);
Mgd Signature: void Method(ByRefIList il );
Unmgd Signature: void Method(IList **il);
Mgd Signature: void Method(String s);
Unmgd Signature: void Method(char *s);
Mgd Signature: void Method(ByRefString s);
Unmgd Signature: void Method(char **s);
Mgd Signature: void Method([MarshalAs(UnmanagedType.Bstr)] String s);
Unmgd Signature: void Method(BStr s);
Mgd Signature: void Method(ByRef [MarshalAs(UnmanagedType.Bstr)] String s);
Unmgd Signature: void Method(BStr *s);


In/Out Marshaling

Every parameter can optionally have associated with it a setting for it’s IN and OUT attributes. These settings describe the direction that data is intended to flow between the caller and the callee but they have no bearing on the level of indirection used by the parameters. The runtime marshaling infrastructure uses the IN/OUT attributes to decide how each parameter should be marshaled between managed and unmanaged code.

  • IN parameters are initialized by the caller and changes by the callee are not reflected in the caller’s view.
  • OUT parameters are considered not to be initialized by the caller and changes made by the callee are reflected in the caller’s view.
  • IN/OUT parameters are initialized by the caller and changes made by the callee are reflected in the callers view.

In the absence of explicit IN/OUT settings, the runtime assumes that all by-reference parameters are passed IN/OUT and that all other non by-reference parameters are IN. The only exception is the StringBuilder class, which is also always assumed to be IN/OUT.

Other points concerning marhaling direction:

  • Value types and base data types are, by default, In (in fact marking them as Out or In/Out is an error).
  • NGWS classes are, by default, In. As described above classes can be marshaled as COM interfaces or as pointers to structures. When classes are marshaled as a COM Interface Out and In/Out are inapplicable and therefore illegal. When classes are marshaled as a pointer to a structure they can be marked as Out and In/Out. There are two interesting cases of marshaling as a pointer to a structure (see the section above on Formatted Types for an explanation):
    • Formatted, Isomorphic – Regardless of the setting of the In and Out attributes, a pointer to the actual class contents is passed and therefore it is as if the parameter had been marked In/Out.
    • Formatted, Non-isomorphic – When a non-isomorphic class is marshaled as a pointer to a structure the callee receives a pointer to a copy of the structure. If the In attribute is set, this copy is initialized (marshaling as necessary) with the instances state. If the Out attribute is set the state is copied (again marshaling as necessary) back in to the instance on return. If both In/Out is set, both copies occur.

There are a few classes which are exceptions to the above rules.

  • String – Strings can only be marshaled in. When Strings are marshaled as LPWStrs (or LPTStrS on a Unicode platform) they are marshalled as Formatted, Isomorphic classes but should not be modified by the callee as strings are intended to be immutable in the Runtime.
  • StringBuilder – StringBuilders default to In/Out rather than In. This reflects their common usage in marshaling.
  • Function Pointer – Bi-directional Need to cover lifetime issues
  • By-ref parameters default are, by default In/Out, but can be marked as In or Out as well.


Data Copying

Whether or not data is copied during the marshaling process depends on the type of the data and it’s IN/OUT attributes.

  • Formatted, Isomorphic classes - Regardless of the setting of the In and OUT attributes, a pointer to the actual class contents is passed and therefore it is as if the parameter had been marked IN/OUT.
  • Formatted, Non-isomorphic – When a non-isomorphic class is marshaled as a pointer to a structure the callee receives a pointer to a copy of the structure. If the IN attribute is set, this copy is initialized (marshaling as necessary) with the instances state. If the OUT attribute is set, the state is copied (again marshaling as necessary) back in to the instance on return. If both IN/OUT is set, both copies occur.
  • String – Strings can only be marshaled IN. When Strings are marshaled as LPWStrs (or LPTStrS on a Unicode platform) they are marshalled as Formatted, Isomorphic classes but should not be modified by the callee as strings are intended to be immutable in the Runtime.


Pinning Data

Dennis, can you update the spec accordingly. I've just checked in the new functionality to GCHandle so that arrays of isomorphic valuetypes (with layout information) and isomorphic classes with layout information can also be pinned.


Defining Explicit Marshaling Behavior

In order to accurately marshal data between the managed and unmanaged space, the marshaler must know the expected managed and unmanaged data representations of the data being passed. For isomorphic types the representation is always the same and the managed signature always describes that representation. For example, a 4-byte integer is always marshaled to a 4-byte integer.

For non-isomorphic types, the marshaler knows the managed representation but not the unmanaged representation. For example, in the PrintMessage method shown in section 1, the marshaler can tell that the managed representation of the msg argument is a string but does not know what representation the PrintMessage function is actually expecting. Somehow the marshaler needs to know that the msg argument needs to be converted from a String to an ANSI character buffer.

The marshaler always knows the managed data representation from method signature or the field type. The unmanaged data representation can either be supplied explicitly or it can be inferred from the manage representation. For example, a string is converted to a BSTR when marshaled from managed to unmanaged code unless the marshaler is explicitly told to marshal the string to some other type like a LPSTR. You can explicitly supply the unmanaged data representation using the MarshalAsAttribute.


Using the MarshalAsAttribute

The System.Runtime.InteropServices.MarshalAsAttribute can be applied to a field, parameter or return value within the source of the type definition. For example:

public static extern void PrintMessage(
   [MarshalAs(UnmanagedType.LPWSTR)]String msg);

Or for a field within a class:

class MsgText {
   [MarshalAs(UnmanagedType.LPWSTR)] String msg;
}

And for a return value:

public static extern
[MarshalAs(UnmanagedType.LPWSTR)] String GetMessage();

The System.Runtime.InteropServices.MarshalAsAttribute uses the System.Runtime.InteropServices.UnmanagedType enum (abbreviated UT throughout this document) to indicate the desired format of the unmanaged type. The above examples all indicate that the msg data should be marshaled as a buffer of Unicode characters (LPWSTR).


Supplying Additional Marshaling Information

Sometimes the marshaler requires move than just the managed and unmanaged data format. The MarshalAs attribute also allows you specify this additional information. For example, in order to marshal an array the marshaling service must also know the element type, rank, size and bounds of the array.


Marshaling Data Types

This section describes how the runtime marshaling service handles each of the non-isomorphic types supported by the runtime. The tables that follow show unmanaged representation that the marshaling service supports for each data type. The table has a column of parameters and one for fields because the marshaling behavior is commonly different depending on how the type is being used. Each column has one format shown in bold letters. The format shown in bold is the default format for that usage of that particular type. If the defaults are different for PInvoke and COM Interop, the words P/Invoke or COM are shown in bold next to their respective defaults.


Boolean Type

The marshalling of most base data types is generally straightforward with the exception of ELEMENT_TYPE_BOOL and ELEMENT_TYPE_CHAR data types. The Boolean type can be marshaled to one of two possible representations. Typically Boolean types are marshaled to a 2-byte integer value where the value 1 represents TRUE and 0 represents FALSE. This representation is identified by UnmanageType.Bool. The other representation is commonly used by Visual Basic. UnmanageType.VariantBool is a 4-byte integer value where the value –1 represents TRUE and 0 represents FALSE.

The marshaling options for ELEMENT_TYPE_BOOL are as follows:

Unmanaged Type | Description of unmanaged format
UnmanagedType.Bool (P/Invoke) | a 2-byte integer value where the value 1 represents TRUE and 0 represents FALSE.
UnmanagedType.VariantBool (COM) | 4-byte integer value where the value –1 represents TRUE and 0 represents FALSE

Table 5 - Marshaling Options for booleans


String Types

Both String and StringBuilder have similar marshaling behavior. Strings can be can be marshaled as COM style BTSR’s which have a prepended length or as a null terminated array of characters. The characters within the string can be marshaled as Unicode, ANSI or in a platform dependent manner such that the characters are Unicode on Windows NT and ANSI on Window 9x.

The marshaling options for ELEMENT_TYPE_STRING, ELEMENT_TYPE_CLASS <String> and ELEMENT_TYPE_CLASS <StringBuilder> are as follows:

Unmanaged Type | Description of unmanaged format
UnmanagedType.BStr (COM) | A COM style BSTR with a prepended length and Unicode characters. Not supported for StringBuilder.
UnmanagedType.ANSIBStr | A COM style BSTR with a prepended length and ANSI characters (rarely used). Not supported for StringBuilder.
UnmanagedType.TBStr | A COM style BSTR with a prepended length and platform dependent characters format (rarely used)
UnmanagedType.LPStr (P/Invoke) | A pointer to a null terminated array of ANSI characters
UnmanagedType.LPWStr | A pointer to a null terminated array of Unicode characters
UnmanagedType.LPTStr PInvoke ONLY | A pointer to a null terminated array of platform characters
UnmanagedType.ByValTStr |

Table 6 - Marshaling options for strings


Object Types

Parameters and fields that are explicitly typed as System.Object can be exposed to the unmanaged world as either a Variant or as an interface. The default behavior is to marshal Objects to COM Variants. These rules only apply to the type System.Object and do not apply to strongly types object that derive from System.Object.

The marshaling options for ELEMENT_TYPE_OBJECT are as follows:

Unmanaged Type | Description of unmanaged format
UnmanagedType.Variant | A COM style Variant.
UnmanagedType.Interface | An IDispatch interface

Importing and Exporting

Consider the following managed interface definition:

interface MarshalObject {
   Void SetVariant(Object o);
   Void SetVariantRef(Ref Object o);
   Object GetVariant();

   Void SetIDispatch ([MarshalAs(UnmanagedType.Interface)]Object o);
   Void SetIDispatchRef([MarshalAs(UnmanagedType.Interface)]Ref Object o);
   [MarshalAs(UnmanagedType.Interface)] Object GetIDispatch();
}

The MarshalObject interface defined above would be exported to a type library as shown below:

interface MarshalObject {
   HResult SetVariant([in] Variant o);
   HResult SetVariantRef([in out] Variant *o);
   HResult GetVariant([out retval] Variant *o)

   HResult SetIDispatch([in] IDispatch *o);
   HResult SetIDispatchRef([in out] IDispatch **o);
   HResult GetIDispatch([out retval] IDispatch **o)
}

Likewise, the following formatted value type

class ObjectHolder : public System.Value {
   Object o1;
   [MarshalAs(UnmanagedType.Interface)]Object o2;
}

Would be exported to a type library as shown below:

struct ObjectHolder {
   Variant o1;
   IDispatch *o2;
}

Marshaling Object to Interface

When an object is exposed to COM as an interface, the interface that exposed is the class interface for the managed type Object (the _Object interface). This interface is typed as an IDispatch (UnmanagedType.IDispatch) or an IUnknown (UnmanagedType.IUnknown) in the resulting type library. COM clients can dynamically invoke the members of the managed class or any members implemented by it subclasses through the _Object interface. The client can also call QueryInterface to obtain any other interface explicitly implemented by the managed type.

Marshaling Object to Variant

When an object is marshaled to a variant, the internal variant type is determined at runtime based on the following rules:

  • If the object reference is NULL, the object is marshaled to a variant of type VT_EMPTY.
  • If the object is an instance of any type list in the table below, the resulting Variant type is determined by the rules build into the marshaler and shown in the table below.
  • Other objects that need to explicitly control the marshaling behavior can implement the IConvertible interface. In that case the variant type is determined by the type code returned from IConvertible.GetTypeCode.
  • Otherwise the object is marshaled as a variant of type VT_UNKNOWN

Marshaling System types to Variant

The system value types shown in the table below are automatically converted to the variant types shown in the table when marshaled as an object. These conversions only apply when the signature of the method being called is of type System.Object.

Type of object instance | COM Variant Type used for marshaling
Null | VT_EMPTY
System.DBNull | VT_NULL
System.ErrorWrapper | VT_ERROR
System.Missing | VT_ERROR with E_PARAMNOTFOUND
System.Exception | VT_ERROR
System.IDispatchWrapper | VT_DISPATCH
System.IUnknownWrapper | VT_UNKNOWN
System.CurrencyWrapper | VT_CY
System.Boolean | VT_BOOL
System.Sbyte | VT_I1
System.Byte | VT_UI1
System.Int16 | VT_I2
System.Uint16 | VT_UI2
System.Int32 | VT_I4
System.Uint32 | VT_UI4
System.Int64 | VT_I8
System.Uint64 | VT_UI8
System.Single | VT_R4
System.Double | VT_R8
System.Decimal | VT_DECIMAL
System.Currency | VT_CY
System.DateTime | VT_DATE
System.String | VT_BSTR
System.Int | VT_INT
System.Uint | VT_UINT
System.Array | VT_ARRAY

For example, using the MarshalObject interface defined above, the code below demonstrates how to pass various types of variants to a COM server.

   MarshalObject mo = new MarshalObject();
mo.SetVariant(null)            // marshal as variant of type VT_EMPTY
mo.SetVariant(System.DBNull)    // marshal as variant of type VT_NULL
mo.SetVariant((int)27)          // marshal as variant of type VT_I2
mo.SetVariant((long)27)          // marshal as variant of type VT_I4
mo.SetVariant((single)27.0)   // marshal as variant of type VT_R4
mo.SetVariant((double)27.0)   // marshal as variant of type VT_R8

COM types that do not have corresponding managed types can be marshaled using wrappers classes such as ErrorWrapper, IDispatchWrapper, IUnknownWrapper, and CurrencyWrapper. The following code demonstrates how to use wrappers to pass various types of variants to a COM server.

// pass ifoo as a Variant of type VT_UNKNOWN interface
mo.SetVariant(IUnknownWrapper(ifoo));

// pass ifoo as a Variant of type VT_DISPATCH interface
cs.SetVariant(IDispatchWrapper(ifoo));

// pass ifoo as a Variant of type VT_ERROR interface
cs.SetVariant(IErrorWrapper(0x80054002));

// pass ifoo as a Variant of type VT_CURRENCY interface
cs.SetVariant(ICurrencyWrapper(System.Currency(5.25)));

The wrapper classes are defined in the System.Runtime.InteropSevices namespace.

Marshaling IConvertible interface to Variant

Types other than those listed above can control how they are marshaled by implementing the IConvertible interface.

If the object implements the IConvertible interface, the COM variant type is determined at runtime by the TypeCode returned from IConvertible.GetTypeCode. The table below shows the TypeCodes along with the corresponding COM variant type for each type code.

TypeCode | COM variant type
TypeCode.Empty | VT_EMPTY
TypeCode.Object | VT_UNKNOWN
TypeCode.DBNull | VT_NULL
TypeCode.Boolean | VT_BOOL
TypeCode.Char | VT_UI2
TypeCode.SByte | VT_I1
TypeCode.Byte | VT_UI1
TypeCode.Int16 | VT_I2
TypeCode.UInt16 | VT_UI2
TypeCode.Int32 | VT_I4
TypeCode.UInt32 | VT_UI4
TypeCode.Int64 | VT_I8
TypeCode.UInt64 | VT_UI8
TypeCode.Single | VT_R4
TypeCode.Double | VT_R8
TypeCode.Decimal | VT_DECIMAL
TypeCode.DateTime | VT_DATE
TypeCode.String | VT_BSTR
Not Supported | VT_INT
Not Supported | VT_UINT
Not Supported | VT_ARRAY
Not Supported | VT_RECORD
Not Supported | VT_CY
Not Supported | VT_VARIANT

Table 7 – Type Code to Variant type mapping

The value of the COM variant is determined by calling the IConvertible.ToXXX (Where ToXXX is the conversion routine that corresponds to the type was returned from IConvertible.GetTypeCode). For example, an object that returns TypeCode.Double from IConvertible.GetTypeCode will be marshaled as a COM Variant of type VT_R8. The value of the variant (stored in the dblVal field of the COM variant) is obtained by calling Object.ToDouble().

Marshaling Variant to Object

When marshaling a variant to an object, the type, and sometimes the value, of the Variant being marshaled determine the type of object produced. The table below identifies each variant type and the corresponding object type that the marshaler creates when a variant is passed from COM to the NGWS runtime.

COM VT Type | Marshal to this type of Object
VT_EMPTY | Null object reference
VT_NULL | System.DBNull
VT_DISPATCH | System.__ComObject or Null if (pdispVal == null)
VT_UNKNOWN | System.__ComObject or Null if (punkVal == null)
VT_ERROR | System.Uint32
VT_BOOL | System.Boolean
VT_I1 | System.Sbyte
VT_UI1 | System.Byte
VT_I2 | System.Int16
VT_UI2 | System.Uint16
VT_I4 | System.Int32
VT_UI4 | System.Uint32
VT_I8 | System.Int64
VT_UI8 | System.Uint64
VT_R4 | System.Single
VT_R8 | System.Double
VT_DECIMAL | System.Decimal
VT_DATE | System.DateTime
VT_BSTR | System.String
VT_INT | System.Int32
VT_UINT | System.Uint32
VT_ARRAY | VT_* | System.Array
VT_CY | System.Currency (This Will change to System.Decimal post PDC)
VT_RECORD | Not Supported
VT_VARIANT | Not Supported

Table 8 – Variant to Type mapping

Round-trip Marshaling

It should be noted that Variants passed from COM to the NGWS runtime and then back out to COM may not retain the same variant type for the duration of the call. Consider what happens when a variant of type VT_DISPATCH is passed from COM to the NGWS runtime. During marshaling, the Variant is converted to a System.Object. If the object is then passed back out to COM, the Object is marshaled back to a variant of type VT_UNKNOWN. There is no guarantee that the variant produce when a object is marshaled from the NGWS runtime to COM will be the same type as the variant used to produce the object initially.

Marshaling ByRef Variants

While Variants themselves can be passed by value or by reference the VT_BYREF flag can also be used with any variant type to indicate that the contents of the variant are being passed by reference as opposed to by value.

The difference between marshaling variants by reference and marshaling a variant with the BY_REF flag set can be confusing. The following diagrams show the difference between passing a variant by reference and passing a variant whose BYREF flag is set to indicate that the variant contains a reference.

When passing objects from the NGWS runtime to COM, the objects contents are copied into a new variant created by the marshaler according to the rules defined in section 5.3.3. Changes made to the variant on the unmanaged side are not back propagated to the original object on return from the call. Similarly, when passing variants from COM to the NGWS runtime, the variants contents are copied to a newly created object according to the rules defined in section 5.3.4. Changes made to the object on the managed side are not back propagated to the original variant on return from the call.

In order to get back propagation, the parameters must be passed by reference. For example, in C# the ref keyword is used to pass parameters by reference. In COM, reference parameters are passed through a pointer such as a variant *.

When passing an object to COM by reference, the marshaler creates a new variant and the contents of the object reference are copied into the variant before the call is made. The variant is passed to the unmanaged function where the user is free to change the contents of the variant. Upon return from the call, any changes made to the variant on the unmanaged side are back propagated to the original object. If the type of the variant differs from the type of the variant passed in to the call, then the changes are back propagate to an object of a different type. . In other words, the type of the object passed into the call may differ from the type of the object returned from the call.

When passing a variant to the NGWS runtime by reference, the marshaler creates a new object and the contents of the variant are copied into the object before the call is made. A reference to the object is passed to the managed function where the user is free to change the object. Upon return from the call, any changes made to the referenced object are back propagated to the original variant. If the type of the object differs from the type of the object passed in to the call then the type of the original variant is changed and the value is back propagate into the variant. In other words, the type of the variant passed into the call may differ from the type of the variant returned from the call.

A variant being passed to the NGWS runtime by value can also have the VT_BYREF flag set to indicate that the variant contains a reference as opposed to a value. In this case, the variant is still marshaled to an object because the variant is being passed by value. The marshaler will automatically de-reference the contents of the variant and copy it into a newly created object before making the call. The object is then passed into the managed function however, upon return from the call, the object is NOT back propagated into the original variant. Any changes made to the managed object are lost. There is no way of changing the value of a variant passed by value, even if the variant has the BYREF flag set.

A variant being passed to the NGWS runtime by reference can also have the VT_BYREF flag set to indicate that the variant contains another reference. In this case the variant is marshaled to a ref object because the variant is being passed by reference. The marshaler will automatically de-reference the contents of the variant and copy it into a newly created object before making the call. Upon return from the call, the value of the object is back propagated to the reference within the original variant only if the object is the same type as the object passed in. In other words, back propagation will never change the type of a variant with the VT_BYREF flag set. If the type if the object is changed during the call an InvalidCastException will occur on return from the call.

From | To | Chgs Back Propagated
Variant v | Object o | Never
Object o | Variant v | Never
Variant *pv | Ref Object o | Always
Ref object o | Variant *pv | Always
Variant v(VT_BYREF|VT_*) | Object o | Never
Variant *v(VT_BYREF|VT_*) | Ref Object o | Only if type hasn’t changed

Value Types and Other Formatted Classes

A formatted type is a type that contains information that explicitly controls the layout of it members. The member layout information is provided with the StructLayout attribute. The layout can be LayoutKind.Sequential or LayoutKind.Explicit. LayoutKind.Sequential layout indicates that the members of the typed are to be laid out in memory exactly as they appear in the type definition. LayoutKind.Explicit is used to indicate that the members are laid out according to the FieldOffset attribute supplied with each filed. For example, the Point and Rect types defined below provide member layout information using the StructLayout attribute.

using System.Runtime.InteropServices;

[StructLayout(LayoutKind.Sequential]
public struct Point {
       public int x;
      public int y;
}

[StructLayout(LayoutKind.Explicit]
public struct Rect {
       [FieldOffset(0)] public int left;
      [FieldOffset(4)] public int top;
      [FieldOffset(8)] public int right;
      [FieldOffset(8)] public int bottom;
}

When marshaled to unmanaged code these formatted types are marshaled as C-style structures. This provides a handy way of calling unmanaged API’s that have structure arguments. For example, the Point and Rect structures can be passed to the Win32 API PtInRect:

BOOL PtInRect(const RECT *lprc, POINT pt);

using the P/Invoke definition shown below:

class Win32API {
   [DllImport(“User32.dll”)]
   public static extern Bool PtInRect(ref Rect r, Point p);

}

The managed Rect must be passed by reference because the unmanaged API is expecting a pointer to a RECT to be passed to the function. The managed point is passed by value because the unmanaged API expects the POINT to be passed on the stack. This subtlety is very important and is pointed out in section 2.3. References are passed to unmanaged code as pointers. Values are passed to unmanaged code on the stack.

Reference types (classes) can also be marshaled to unmanaged code as a C-style structure provided they have fixed member layout. The member layout information for a class is also provided with the StructLayout attribute. The main difference between value types with fixed layout and a reference types with fixed layout is in how they are passed as method arguments. Value types are passed by value (on the stack) and consequently any changes made to the members of the type by the callee are not seen by the caller. Reference types are passed by reference (a reference to the type is passed on the stack) and consequently any changes made to the members of the type by the callee are seen by the caller. The same rules apply when reference types are passed to unmanaged code. For example, the SystemTime class has sequential member layout and could be passed to the Win32 API GetSystemTime.

[StructLayout(LayoutKind.Sequential)]

public class SystemTime {
   public ushort wYear;
   public ushort wMonth;
   public ushort wDayOfWeek;
   public ushort wDay;
   public ushort wHour;
   public ushort wMinute;
   public ushort wSecond;
   public ushort wMilliseconds;
}

GetSystemTime is defined as:

void GetSystemTime(SYSTEMTIME* SystemTime);

The P/Invoke definition would be:

class Win32API {
   [DllImport(“User32.dll”)]
   public static extern void GetSystemTime(SystemTime st);

}

Notice that the SystemTime argument is not typed as a ref argument because SystemTime is a class as opposed to a value type. Unlike value types, classes are always passed by reference.

Unmanaged Type | Description of unmanaged format
UnmanagedType.Struct | A C style structure

Table 9 – Class Types

When a formatted type is marshaled as a structure, only the fields within the type are accessible. If the type has methods, properties or events, there are inaccessible from unmanaged code.

For example, the following code shows the Point class that has a method called SetXY. Because the type has sequential layout, in can be passed to unmanaged code and marshaled as a structure. however, the SetXY member is not callable from unmanaged code (even through the object is passed by reference).

[StructLayout(LayoutKind.Sequential)]
public class Point {
   int x, y;
   public void SetXY(int x, int y){
      this.x = x;
      this.y = y;
   }
};

Formatted types can also be passed to COM Interop method calls. In fact, when exported to a type library value types are automatically converted to structures. For example, the Point value type becomes a typedef with the name Point. All references to the Point value type elsewhere in the type library are replaced with the Point Typedef.

typedef struct tagPoint {
   int x;
   int y;
} Point;

interface _Graphics {
   …
   HResult SetPoint ([in] Point p)
   HResult SetPointRef ([in out] Point *p)
   HResult GetPoint ([out retval] Point *p)
}

.

The same rules used to marshal values and references to PInvoke calls are used when marshaling through COM interfaces. When an instance of the Point value type is passed from the NGWS runtime to COM, the Point is passed by value. If the Point value type is passed by reference, a pointer to a Point is passed on the stack.

The marshaler does not support higher levels of indirection (Point **) in either direction.

System Value Types

The System namespace has several value types that represent the boxed form of the runtime primitive types. For example, the value type System.Int32 represents the boxed form of the ELEMENT_TYPE_I4. Instead of marshaling these types as structures (as other formatted types would be), they are marshaled exactly the same as the primitive types they box. System.Int32 therefore is marshaled as ELEMENT_TYPE_I4 instead of a structure containing a single member of type long. The following table contains a list of the value types in the system namespace that are boxed representations of primitive types.

System Value Type | Marshaled as Element Type
System.Boolean | ELEMENT_TYPE_BOOLEAN
System.Sbyte | ELEMENT_TYPE_I1
System.Byte | ELEMENT_TYPE_UI1
System.Char | ELEMENT_TYPE_CHAR
System.Int16 | ELEMENT_TYPE_I2
System.Uint16 | ELEMENT_TYPE_U2
System.Int32 | ELEMENT_TYPE_I4
System.Uint32 | ELEMENT_TYPE_U4
System.Int64 | ELEMENT_TYPE_I8
System.Uint64 | ELEMENT_TYPE_U8
System.Single | ELEMENT_TYPE_R4
System.Double | ELEMENT_TYPE_R8
System.String | ELEMENT_TYPE_STRING
System.Int | ELEMENT_TYPE_I
System.Uint | ELEMENT_TYPE_U

Some other value types in the System namespace are also handled a bit different. Because the unmanaged world already has well established formats for these types, the marshaler has special rules for how these types are marshaled. The following table lists the special value types in the System namespace along with the unmanaged type that the marshaler that the marshaling service will marshal these types as.

System Value Type Marshaled as IDL Type
System.DateTime
typedef double DATE;
System.Decimal
typedef struct tagDEC {
    USHORT    wReserved;
    BYTE      scale;
    BYTE      sign;
    ULONG     Hi32;
    ULONGLONG Lo64;
} DECIMAL;
System.Currency
typedef struct tagCY {
        ulong Lo;
        long  Hi;
} CY;
System.Guid
typedef struct tagGUID {
    DWORD Data1;
    WORD  Data2;
    WORD  Data3;
    BYTE  Data4[ 8 ];
} GUID;


Reference Types

Reference types are either interfaces or classes. Some classes contain explicit layout information with the StructLayoutAttribute. be either formatted or unformatted. Interfaces and classes are both marshaled through a proxy interface generated by the marshaler. Formatted class can also be marshaled as structures if marked with UnmanagedType.Struct.

Unmanaged Type | Description of unmanaged format
UnmanagedType.Interface | A COM Interface.
UnmanagedType.Struct | A C style structure

Table 10 – Class Types

Because the layout of formatted types is fixed, the marshaler is able to safely and predictably move data between the managed and unmanaged representation of these types. The marshal does not support marshaling of unformatted types as structures. Doing so would cause problems if the layout of the type changed over time. Instead, unformatted type can only be accessed via and interface.

Passing an unformatted value type through the marshaler will cause an exception.

Marshaling Interfaces

Managed classes and interface are always marshaled through a proxy created by the runtime marshaling service. Every interface has both a managed and an unmanaged representation. The runtime SDK provides tools for generating one representation form another representation. The proxy is then responsible for marshaling calls from on representation to the other. This includes any necessary conversion of data that is passed along with the call.

While the managed and unmanaged representations of the interface share the same identity (based on GUID), their layout and method signatures differ quite a bit. The unmanaged definition of the interface is usually described in a type library while the managed definition of the interface is described in NGWS runtime metadata. See the Interop Spec for details on how the two interfaces differ.

Interfaces are marshaled between managed and unmanaged code as interface. Every interface has both a managed and an unmanaged representation. All references to an interface from managed code refer to the managed representation while references from unmanaged code refer to the unmanaged representation:

For example, consider the following managed code with the IPoint interface:

interface IPoint {
   void SetXY(int x, int y);
}

class Graphics {
   void SetPoint (IPoint p);
   void SetPointRef (ref IPoint p);
   IPoint GetPoint ();
}

When exported to a type library, the IPoint interface is converted to a COM Interface. All references to the Point interface are maintained as they were in managed code.

interface IPoint : IUnknown {
   HRESULT SetXY(int x, int y);
}

interface _Graphics {
   …
   HRESULT SetPoint ([in] IPoint *p);
   HRESULT SetPointRef ([in out] IPoint **p);
   HRESULT GetPoint ([out retval] IPoint **p);
}

public coclass Graphics {
   [default] interface _Graphics;
}

Marshaling Classes

Classes can only be marshaled by COM Interop and are always marshaled as interfaces. In most cases the actual interface used to marshal the class is what is known as the “class interface” (see the Interop Spec for details on overriding the class interface with an interface of your choice).

Every managed class has an implicit class interface that contains all the members of the class. The class interface can only be used from COM; managed client never see or know about the class interface and there is never a need to explicitly define a class interface in managed code. The class interface contains all the methods and properties of the class and carries the same name as the class with a prepended underscore. The fields of the class are also available through getters and setters on the class interface.

Clients can late bind against the class interface because it always derives from IDispatch. Like any other interface, the class interface is exposed via a proxy. The proxy is responsible for marshaling the calls between the class interface and the actual members of the managed class.

Importing and Exporting

When an assembly is exported to a type library, a class interface is produced for each accessible class with the assembly. A coclass is also created for the class and the class interface is marked as the default interface for the coclass. All parameter and field references to the class are replaced with references to the class interface for that class. For example:

Managed Types

class Foo {…}
class Bar {
   Void SetFoo(Foo f);
   Void SetFooRef(ref Foo f);
   Foo GetFoo();
}

Generated Type Library

[uuid(…), …]
interface _Foo : IDispatch {…}

[uuid(…), …]
coclass Foo {
   [default] interface _Foo;
}
[uuid(…), …]
interface _Bar {
   HRESULT SetFoo([in] _Foo *f);
   HRESULT SetFooRef([in, out] _Foo **f);
   HRESULT GetFoo([out, retval] _Foo **f)
}

[uuid(…), …]
coclass Bar {
   [default] interface _Bar;
}

When a type library is imported to produce an assembly, the coclasses within the type library are converted to managed classes that have all the member of the coclass’es default interface. For example, the managed class Foo below has the same method m() as the IFoo interface. All references to the coclass that appear as parameters or fields are converted to references to the managed class (as in the SetFoo() method below).

Type Library

[uuid(…), …]
interface IFoo {
   void m{};
}
[uuid(…), …]
coclass Foo {
   [default] interface IFoo;
}

[uuid(…), …]
interface IBar {
   HRESULT SetFoo([in] Foo *f);
   HRESULT SetFoo([in, out] Foo **f);
   HRESULT GetFoo([out, retval] Foo **f)
   HRESULT SetIFoo([in] IFoo *f);
   HRESULT SetIFoo([in, out] IFoo **f);
   HRESULT GetIFoo([out, retval] IFoo **);
}

Generated Assembly

interface IFoo {
   HRESULT m()
}

class Foo {
   HRESULT m()
}

interface IBar {
   void SetFoo(Foo f);
   void SetFooRef(ref Foo f);
   Foo GetFoo();
   void SetIFoo(IFoo f);
   void SetIFooRef(ref IFoo f);
   IFoo GetIFoo();
}

Passing Classes to the NGWS Runtime

When a managed class is passed to COM, the Interop marshaler automatically wraps the class with a COM proxy and passes the class interface produced by the proxy on to the COM method call. The proxy then delegates all calls on the class interface back to the managed object. The proxy also exposes other interfaces that are not explicitly implemented by the class. Interfaces like IUnknown and IDispatch are automatically implemented by the proxy on behalf of the.

Passing Classes to the NGWS Runtime

Coclasses are not typically used as method arguments in COM. Instead, a classes default interface is usually passed in place of the coclass itself.

When a interface is passed into managed code, the Interop marshaler is responsible for wrapping the interface with the proper wrapper and passing that wrapper on to managed method. Determining which wrapper to use can be difficult.

Every instance of a COM object has a single unique wrapper, regardless of how many interfaces the object implements. In other words, a single COM object that implements 5 interfaces has only one wrapper. The same wrapper exposes all 5 interfaces. If 2 instances of the COM object are created then 2 instances of the wrapper are created.

In order for the wrapper to maintain the same type throughout its lifetime, it’s important that the Interop marshaler determine which is the correct wrapper the first time an interface exposed by the object is passed through marshaler. That means that the marshaler must try to determine the identity of the object by look at one of the interfaces the object implements. Consider the example below where the marshaler needs to determine that the Foo wrapper should be used to wrap the I3 interface that was passed into managed code.

When the interface I3 is first passed through the marshaler, the marshaler first checks to see if the interface is coming from a known object. This would occur in two possible situations.

  1. The interface I3 is being implemented by another managed object that happened to be passed out to COM elsewhere. The marshaler can readily identify interfaces exposed by managed objects and is able to match the interface up with the managed object that provides the implementation. The actual managed object is then passed to the method and no wrapper is needed.
  2. The interface I3 is being implemented by an object that has already been wrapped. To determine if this is the case, the marshaler queries the object for its IUnknown interface and compare the returned interface to the interfaces of other objects that are already wrapped. It the interface is the same as that of another wrapper, the objects are determined to have the same identity and the existing wrapper is passed to the method.

If interface is not from a know object, the marshaler then needs to determine how a new wrapper should be created:

  1. The marshaler will first query the object for the IProvideClassInfo2 interface. If provided, the marshaler will use the CLSID returned from IProvideClassInfo2.GetGUID() to identify the coclass providing the interface. With the CLSID, the marshaler can potentially locate the wrapper class in one of two possible ways
    1. If the assembly containing the metadata for the class has already been loaded, the marshaler will use the CLSID to locate the metadata for the class and create the proper wrapper. This approach assumes that the metadata for the class was generated with TlbImp prior to the interface being passed in.
    2. If the assembly containing the metadata has not been loaded, the marshaler will check the registry to see if metadata for the CLSID has been registered. In order to register the metadata for the class, TlbImp would have to be run prior to the interface being passed in.
  2. The marshaler will then query the interface for the IProvideClassInfo interface. If provided, the marshaler will use the ITypeInfo returned from IProvideClassInfo.GetClassinfo() to determine the CLSID of the class exposing the interface. With the CLSID the marshaler will use to process described above to locate the metadata for the wrapper.
  3. Finally, if all attempts to identify the class fail, the marshaler will wrap the interface with a generic wrapper class called System.__ComObject.


Array Types

Managed Arrays

Managed arrays come in many flavors. The class System.Array is the base class of all array types. The System.Array class has properties for determining the rank, length, lower and upper bounds of an array as well as methods for accessing, sorting, searching, copying and creating arrays.

There are also 2 specialized array types:

ELEMENT_TYPE_ARRAY is a typed array with any rank and any lower bound. Both the element type and the rank must be specified with this type of array.

ELEMENT_TYPE_SZARRAY is a typed array with rank 1 and 0 lower bounds. The element type must be specified with this type of array. The rank and bounds are fixed.

Both array types are dynamic and therefore do not have a corresponding static type defined in the base class library. It’s convenient to think of each combination of element type, rank as a distinct type of array. Therefore, a 1-dimensional array of integers is of a different type than a 1-dimensional array of doubles. Similarly a 2-dimensional array of longs is different than a 1-dimensional array of longs. The bounds of the array are not considered when doing type comparisons.

Any instance of a managed array must be of a specific element type, rank and low bound. The type System.Array cannot be instantiated because element type, rank and bounds are not specified. However, parameters are commonly typed as System.Array in order to support polymorphism.

Managed Array Type | ElementType | Rank | Low Bound | Signature Notation
ET_CLASS <System.Array> | Unknown | Unknown | Unknown | System.Array
ET_SZARRAY <Type> | Specified by Type | 1 | 0 | Type[n]
ET_ARRAY <Type><Rank>[<Bounds>…] | Specified by Type | Specified by Rank | Optionally specified by Bounds | Type[n,m]

Table 11 – Overview of managed array types

Unmanaged Arrays

Unmanaged arrays are either COM style SafeArrays or C-style arrays with fixed or variable length. SafeArrays are self-describing arrays that carry the type, rank and bounds of the associated array data. C-style arrays are single dimensional, typed arrays with fixed lower bound of 0.

The marshaling service has limited support for both types of arrays.

Passing Arrays to the NGWS Runtime

Both C-style arrays and SafeArrays can be passed into the NGWS runtime from unmanaged code.

Unmanaged Type | Imported as…
SafeArray(Type) | ET_SZARRAY <ConvertedType> Rank = 1, low bound = 0. Size known only if provided in managed signature. SafeArrays that are not Rank =1 or low bound = 0 are not marshalable.
Type [] | ET_SZARRAY <ConvertedType> Rank = 1, low bound = 0. Size known only if provided in managed signature.

SafeArrays

When a SafeArray is imported from a type library to a NGWS runtime assembly, the array is converted to a single dimensional array of a known type (i.e. int[]). The same type conversion rules that are applied to parameters also apply to array elements. So, for example, a SafeArray of BSTRs becomes a managed array of Strings and a SafeArray of Variants becomes a managed array of Objects. The SafeArray element type is captured with SafeArraySubType named argument passed to the MarshalAs attribute applied to the parameter.

Because the rank and bounds of the SafeArray cannot be determined from the type library, the rank is assumed to 1 and the lower bound is assumed to be 0. The rank and bounds must be defined in the managed signature produced by TlbImp. If the actual rank or low bounds passed to the method at runtime differ, a SafeArrayTypeMismatchException is thrown. Multi dimensional or not 0 bound SafeArrays therefore cannot be marshaled from COM to the NGWS runtime.

For example

HRESULT Foo1([in] SAFEARRAY( int ) ar);
HRESULT Foo2([in] SAFEARRAY( DATE ) ar);
HRESULT Foo3([in, out] SAFEARRAY( BSTR ) *ar);

Is converted to:

void Foo1([MarshalAs(UnmanagedType.SafeArray, SafeArraySubType=VT_I4)] int[] ar) ;
void Foo2([MarshalAs(UnmanagedType.SafeArray, SafeArraySubType=VT_DATE)] DateTime[] ar);
void Foo3([MarshalAs(UnmanagedType.SafeArray, SafeArraySubType=VT_BSTR)] ref String[] ar);

It should be noted that if the type of the actual array passed at runtime does not match the type indicated in the type library the marshaling may fail.

C-style arrays

When a C-style array is imported from a type library to a NGWS runtime assembly, the array is converted to ELEMENT_TYPE_SZARRAY.

The array element type is determined from the type library and preserved during the import. The same conversion rules that are applied to parameters also apply to array elements. So, for example, an array of LPSTRs becomes an array of Strings. The array element type is captured with ArraySubType named argument passed to the MarshalAs attribute and applied to the parameter.

The array rank is assumed to always be 1. If the rank is greater than 1, the array is marshaled as a single dimension array in column major order. The lowers bound is always 0.

Type libraries can contain arrays of fixed or variable length. Only fixed length arrays can be imported from type libraries with TlbImp. Type libraries containing variable length arrays must be defined manually as shown below. With fixed length arrays, the size is imported from the type library and captured in the MarshalAs attribute applied to the parameter.

For example:

HRESULT Foo1(int ar[10]);
HRESULT Foo2(double ar[10][20]);
HRESULT Foo3(LPWSTR ar[10]);

Is converted to:

void Foo1([MarshalAs(UnmanangedType.LPArray, SizeConst=10)] int[] ar);
void Foo2([MarshalAs(UnmanangedType.LPArray, SizeConst=200)] double[] ar);
void Foo2([MarshalAs(UnmanangedType.LPArray, ArraySubType=UnmanangedType.LPWSTR, SizeConst=10)] String[] ar);

Unfortunately, only fixed size arrays can be imported from type libraries with TlbImp because the type libraries lack the information needed to marshal variable length arrays. Even though IDL allows an array to be attributed with the size_is or length_is attribute to convey the size to the callee, the MIDL compiler does not propagate that information to the type library. Without knowing the size, the marshaling service cannot marshal the array elements.

Consequently, the variable length arrays are simply imported as reference arguments. For example:

HRESULT Foo1(int ar[]);
HRESULT Foo2(int ArSize, [size_is(ArSize)] double ar[]);
HRESULT Foo3(int ElemCnt, [length_is(ElemCnt)] LPSTR ar[]);

Is converted to:

void Foo1(ref int ar);
void Foo2(ref double ar);
void Foo3(ref String ar);

However, it is possible to provide the marshaler with the array size by defining the managed method signature manually (as opposed to using TlbImp) using the MarshalAs attribute.

The MarshalAs attribute is applied to the array parameter of the managed method definition. The attribute can be used to indicate the number of elements in the array in 3 different ways.

  1. By identifying that the size of the array is determined by the value passed in another parameter. The parameters are identified positionally starting with the first parameter as number 0. For example:
    void Foo(
   int ElemCnt,
   [MarshalAs(UnmanagedType.LPArray, SizeParam=0)] int[] ar );
  2. By identifying that the size of the array is determined by the value passed in another parameter times a multiplier. For example:
    void Foo(
   int ElemCnt,
   [MarshalAs(UnmanagedType.LPArray,SizeParam=1,SizeMultiplier=4] int[] ar);
  3. By identifying that the size of the array is constant. For example:
    void Foo(
   [MarshalAs(UnmanagedType.LPArray, SizeConst=128)] int[] ar );

At runtime, the marshaler refers to the MarshalAs attribute associated with the parameter to determine the actual array size.

Passing Arrays to COM

All managed array types can be passed into unmanaged code from the NGWS runtime. Depending on the managed type and the attributes applied to it, the array may be accessed as a SafeArray or a C-style array.

Managed Array Type | Exported as…
ET_SZARRAY <type> | UnmanagedType.SafeArray(type) UnmanagedType.LPArray Type provided in signature. Rank always 1, low bound always 0. Size always known at runtime.
ET_ARRAY <type> <rank>[<bounds>] | UnmanagedType.SafeArray(type) UnmanagedType.LPArray Type, Rank, Bounds provided in signature. Size always known at runtime.
ET_CLASS <System.Array> | UT_Interface UnmanagedType.SafeArray(type) Type, rank, bounds and size always known runtime.

ET_SZARRAY

When a method containing an ET_SZARRAY parameter (single dimensional array) is exported from a NGWS runtime assembly to a type library, the array parameter is converted to a SafeArray of a given type. Again the same conversion rules apply to the array element types. The contents of the managed array are automatically copied from managed memory into the SafeArray. For example:

void Foo(long[] ar );
void Foo(String[] ar );

Is exported as:

HRESULT Foo([in] SAFEARRAY( long ) ar);
HRESULT Foo([in] SAFEARRAY( BSTR ) ar);

The rank of the SafeArrays is always 1 and the lower bound is always 0. The size is determined at runtime by the size of the managed array being passed.

The array can also be marshaled as a c-style array by using the MarshalAs attribute. For example:

void Foo([MarshalAs(UnmanagedType.LPArray)] long [] ar );
void Foo([MarshalAs(UnmanagedType.LPArray)] String [] ar );
void Foo([MarshalAs(UnmanagedType.LPArray, ArraySubType= UnmanagedType.LPSTR)] String [] ar );

Is exported as:

HRESULT Foo(long ar[]);
HRESULT Foo(BSTR ar[]);
HRESULT Foo(LPSTR ar[]);

Although the marshaler has the length information needed to marshal the array, the array length is usually passed as a separate argument in order to convery the length to the callee.

ET_ARRAY

When a method containing an ET_ARRAY parameter is exported from an NGWS runtime assembly to a type library, the array parameter is converted to a SafeArray of a given type. The contents of the managed array are automatically copied from managed memory into the SafeArray. For example

void Foo( long [,] ar )
void Foo( String [,] ar )

Is exported as:

HRESULT Foo([in] SAFEARRAY( long ) ar);
HRESULT Foo([in] SAFEARRAY( BSTR ) ar);

The rank, size and bounds of the SafeArrays are determined at runtime by the characteristics of the managed array.

The array can also be marshaled as a c-style array by using the MarshalAs attribute. For example:

void Foo([MarshalAs(UnmanagedType.LPARRAY)] long [,] ar )
void Foo([MarshalAs(UnmanagedType.LPARRAY, ArraySubType= UnmanagedType.LPSTR)] String [,] ar )

Is exported as:

HRESULT Foo(long ar[]);
HRESULT Foo(LPSTR ar[]);

Nested arrays are not supported.

For example:

void Foo(long [][][] ar )

Would cause an error when exported with TlbExp.

ET_CLASS <System.Array>

When a method containing a System.Array parameter is exported from a NGWS runtime assembly to a type library, the array parameter is converted to an _Array interface. The contents of the managed array are only accessible through the methods and properties of the _Array interface. System.Array can also be marshaled as a SafeArray by using the MarshalAs attribute. When marshaled as a SafeArray the array elements are marshaled as variants.

void Foo1( System.Array ar )
void Foo2( MarshalAs(UnmanagedType.SafeArray)], System.Array ar )

Is exported as:

HRESULT Foo([in] _Array *ar);
HRESULT Foo([in] SAFAERRAY(VARIANT) ar);


Delegate Types

Unmanaged Type | Description of unmanaged format
UnmanagedType.FunctionPtr | An unmanaged function pointer

Table 12 – Class Types


Other Types

Unmanaged Type | Description of unmanaged format
UnmanagedType.SysInt | Not used
UnmanagedType.SysUInt | Not used
UnmanagedType.LPVoid | Not used
UnmanagedType,AsAny | Not used

Table 13 – Class Types


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Other Marshaling Considerations

© 2000 Microsoft Corporation. All rights reserved.


Missing information

  • How does the miOle flag effect marshaling.
  • Explain how signatures are mangled
  • Add some common examples
  • Eliminate the NO_SPEC charset from the code
  • Explain how out, retval parameters add another level of indirection
  • Discuss lifetime issues of delegates
  • Discuss S_FALSE


Appendix A - Setting Marshal Information with ReflectionEmit

Marshaling information can be set with the Reflection Emit classes in the Runtime class library. Two class in particular allow marshaling information for parameters and fields to be supplied. Both ParameterBuilder and FieldBuilder have a SetMarshal method.

public class ParameterBuilder {
   …
   public virtual void SetMarshal(NativeMarshal nativemarshal);
}

public class FieldBuilder: FieldInfo: FieldInfo {
   …

   public virtual void SetMarshal(NativeMarshal nativemarshal);
}

Both SetMarshal methods take a NativeMarshal object as their only argument. Calling the one of the static methods on the NativeMarshal class creates the native marshal object. The DefineNativeMarshal method is used for most simple unmanaged types. DefineByValStr is used for fixed length string buffer. DefineSafeArray is used for COM SafeArrays.

public class NativeMarshal {
   // used to define all type except those listed below
   static NativeMarshal DefineNativeMarshal(UT t);

   // used to define a fixed length string buffer within a class.  The elemCount argument identifies the size of the buffer.  The Type of characters within the unmanaged buffer is determined by the charset of the class.  This type can only be used on fields.  It is not permitted on parameters.
   static NativeMarshal DefineByValTStr(int elemCount)

   // used to define a COM SafeArray.  The elemType argument is optional.  It is only needed when the array elements are string types.  This type can only be used on Parameters
   static NativeMarshal DefineSafeArray([optional]
                           UT elemType)

   // used to define a fixed length array field within a class.  The elemCount argument identifies the number of elements within the array  The elemType argument is optionally used to identify the type of the elements within the array.  It is only necessary for elements that are string types.  This type can only be used on fields.  It is not permitted on parameters.
   static NativeMarshal DefineByValArray(int elemCount);

   // used to define a variable length array being used as a parameter.  The elemType argument is optionally used to identify the type of the elements within the array.  It is only necessary for elements that are string types.  This type can only be used on parameters.  It is not permitted on fields.
   static NativeMarshal DefineLPArray(UT elemType);

   // used to define a custom marshaler.  The MarshalProvider argument identifies the type that will server as the marshal provider.  This type must implement ICustomMarshal.  The IID argument is identifies the COM type that the marshal provider will convert this parameter to.  The cookie optionally used to pass data to the marshal provider when it is constructed.  See the Attributed Marshaling spec for more details.
   static NativeMarshal DefineCustomMarshaler(Type MarshalerProvider,

                           String IID; String Cookie)
}


Appendix B - PInvoke Background

In order to define a PInvoke method the you need the following information:

  • Name of the native dll containing the method
  • Name method name or ordinal within the dll
  • Calling convention used by the method
  • The character set to be used when marshaling the methods string arguments
  • Whether the method will call SetLastError before returning

In order to pass a struct to through a PInvoke call you need the following informationfor the struct

  • The character set to be used when marshaling the fields within the struct
  • The packing size for the struct
  • The struct must have sequential or explicit layout information
  • If the struct has explicit layout then layout information is needed for each field

The following metadata API's are used to provide this information when defining Pinvoke methods and structs:

To define a PInvoke method you would use:

DefinePinvokeMap: used to set the target dll, the name or ordinal of the method in that dll, flags for controling name mangling, character set, calling converntion and SetLastError support (pmNoMangle, pmCharSetAnsi, pmCharSetUnicode, pmCharSetAuto, pmSupportSetLastError, )

To define a PInvoke struct you would use:

DefineTypeDef: used to indicate swhether layout is sequential or explicit and to set the flags that control the default marshaling behavior of struct members (tdAnsi, tdUnicode, tdAutoClass)
SetClassLayout: used to set the packing size of the struct and the offset of the individual fields within the struct

Compilers have varying ways of expressing this information within their respective languages

C#:

      [dllimport(KERNEL32, EntryPoint="MoveFile",  SetLastError=true, CharSet=CharSet.Auto, ExactSpelling=true, CallingConvention=StdCall)]
      public static extern void MoveFile(String src, String dst);

[structlayout(LayoutKind.Sequential, CharSet=CharSet.Unicode), Pack=4]
public struct TypeDesc {
       public int lpValue;
      public WORD vt;
}

[structlayout(LayoutKind.Explicit, CharSet=CharSet.Unicode)]
public struct TypeDesc {
       [structureOffset(0)]public int lpValue;
      [structureOffset(4)]public WORD vt;
}

Note: dllimport and structlayout are C# specific keywords not custom attributes

MC++:

[sysimport(dll="user32", name="MessageBox", charset=unicode)]
int MessageBox(DWORD hWnd, String strMessage, String strCaption, unsigned int uiType);

Struct supported not defined yet

Note: sysimport is a MC++ specific keyword not a custom attribute

VB:

Declare Sub MessageBeep Lib "User" Alias "SomeBeep"(ByVal N As Integer)
Declare Function GetWinFlags Lib "Kernel" Alias "#132"() As Long
Struct supported not defined yet

Note: Declare is a VB specific keyword

The BCL currently has the following enums defined in the System.Runtime.InteropServices namespace for use in the languages as shown above:

namespace System.Runtime.InteropServices {
public enum CallingConvention
{
    Winapi          = 1,
    Cdecl           = 2,
    StdCall         = 3,
    ThisCall        = 4,
    FastCall        = 5,
}

public enum LayoutKind
{
    Sequential;
    Union;
    Explicit;
}

public enum CharSet
{
    None = 1,      // User didn't specify how to marshal strings.
    Ansi = 2,      // Strings should be marshalled as ANSI 1 byte chars.
    Unicode = 3,            // Strings should be marshalled as Unicode 2 byte chars.
    Auto = 4,      // Marshal Strings in the right way for the target system.
}
}

Proposed New attributes

The following new attributes would give us a more consistent mechanism for attributing PInvoke classes. It would allow all compiler that support custom attributes to define PInvoke methods without changing their language syntax. I'm proposing that we add these new custom attributes and that we do the appropriate attribute to metadata mapping within the DefineCustomAttribute API.

namespace System.Runtime.InteropServices {

public class DllImportAttribute
{
    public DllImportAttribute(String dllName);      // used to set the dll name passed to DefinePinvokeMap

    public String         EntryPoint;      // used to set the entry point passed to DefinePinvokeMap
    public CharSet      CharSet;      // used to set the pmCharSetAnsi, pmCharSetUnicode, pmCharSetAuto on DefinePinvokeMap
    public boolean      SetLastError;      // used to set the pmSupportSetLastError flag on DefinePinvokeMap
    public boolean      ExactSpelling;      // used to set the pmNoMangle flag on DefinePinvokeMap
    public CallingConvention   CallingConvention;   // used to set the pmNoMangle flag on DefinePinvokeMap

    // accessors
    public String         DllName
}

public class StructLayoutAttribute
{
    public StructLayoutAttribute(LayoutKind kind);      // used to set the Layout kind passed to DefineTypeDef

    public int      Pack;            // used to set the packing size passed to SetClassLayout
    public CharSet   CharSet;         // used to set the charset passed to DefineTypeDef
    public boolean   CheckFastMarshal;      // Not sure how this is used?

    // accessors
    public LayoutKind   StructLayoutKind;
}
}

Scrapbook

Explicitly specifying an unmanaged type is optional because every managed type is marshaled to a corresponding unmanaged type by default. So, for example, there’s no need to explicitly supply the unmanaged type for a method parameter typed as a long because a long is automatically marshaled to a long in the unmanaged space. The unmanaged type attribute is only needed to explicitly control the unmanaged type.

The marshaling done by the runtime’s marshaling service is limited. That is, it is not meant to be a generic marshaler mechanism that can marshal any type to any other type. Therefore, only a very restricted set of unmanaged types is allowed for each managed type. For example, a System.String class can be marshaled to BStr, LPStr or an LPWStr but not to something like a long or a Date.

As mentioned above, each managed type has a corresponding unmanaged data type. This corresponding type can be different depending on where the data type is used and what the ambient default settings are. There are three “places” marshaling attributes can be applied: on parameters & return values of P/Invoke functions, parameters & return values of functions defined in interfaces (for COM Interop) and fields of classes and value types. The ambient setting that can affect marshaling behavior are listed in sections toward the end of this document. An example is the CharSet attribute that controls whether managed strings map to ANSI, Unicode, or BStr, etc unmanaged strings


Class Library Design Guidelines

Writing Managed Class Libraries for the Runtime

Overview

Moving to a managed execution environment offers an opportunity to improve the programming model to a wide range of functionality. The goal of this document is to encourage consistency and predictability in the public APIs while enabling web and cross la


Introduction to Class Library Design Guidelines

Designing managed, object oriented software is hard and designing it to fit with other managed libraries outside of your direct control is even harder. The problem is exacerbated by the fact that most your functionality has been exposed in an unmanaged way for years. There is considerable code, design experience and most importantly, faithful customer’s hard learned knowledge invested in the legacy design. What parts of your libraries should be managed? How does this interoperate with Com classic and other unmanaged functionality? How does this fit into the overall Microsoft strategy? What will impress your customers about a managed design?

This document will not address all these issues. Instead we define a set of guidelines as a way help class library designers more fully understand the trade-offs of different solutions. “Rules are made to be broken” as the saying goes, and we understand that there are situations where it is necessary to stray from these guidelines. However, our hope is to force designers to make an explicate decision to violate these guidelines and for these violations to be rare and not far from the spirit of this document.

A well-designed managed class library is:

Consistent Similar design patterns across libraries

- Predicable- Easily discoverable functionality. Typically only one way to perform a given task.

Web centric - Designed with cross context, cross process and cross machine execution in mind

- Callable from semi-trusted code.

Language neutral - Functionality accessible to many different programming languages


Justification

Moving to a managed execution environment offers an opportunity to improve the programming model to a wide range of functionality. For several reasons, we strongly advise designers to treat these guidelines as if they were prescriptive:

  • First, because we believe that developer productivity can be seriously hampered by inconsistent design.
  • Second, because development tools and add-ins will turn some of these guidelines into de facto prescriptive rules, thus reducing the value of non-conforming components. These components will function, but not to their full potential

It is very important to follow the guidelines given here, however there are instances where good library design dictates that these guidelines need to be broken. In such as case it is important to provide solid justification.


Relationship to the VOS

The Virtual Object System (VOS) is the model that defines the rules the Runtime follows when declaring, using, and managing types. The VOS establishes a framework that enables cross-language integration, type safety, and high performance code execution. The VOS is the raw materials out of which designers can build class libraries.

For more information on see the Virtual Object System spec.


Relationship to the CLS

Targeting the Common Language Specification (CLS) is an excellent way to ensure cross language interop. The CLS defines a set of programmatically verifiable rules that governs the interop of types authored in different programming languages. Although the CLS tries to encourage good library design, it does not enforce it.

The CLS is a set of rules for managed class library designers to guarantees that their APIs will be callable across a wide range of languages. There are two guiding principals for deciding what is in and what is out.

Does feature X facilitate the type of API development we want to encourage in the managed space?

Although we want the CLS to be rich enough for any managed library to be written exclusively in the CLS, more is not always better. Having ten ways to do the same task leads to confusing and bad design/usage discussions. For example providing both safe and unsafe constructs forces users to decide which they want to use. The CLS only offers type safe constructs, thereby encouraging the "correct" behavior.

Is it burdensome for a compiler to expose feature X?

All languages will take some modification to target the runtime and our VOS/VES (for example, look at the changes we made to VC and VB). But we don't have to create a huge overhead of additional work needed to be CLS compliant. The goal is for the CLS to be as small as possible and still meet the first goal. For example, parameterized types are NOT in as it is burdensome for a compiler to expose this feature.

For more information, see the Virtual Object System.


Naming Guidelines

One of the most important elements of predictability and discoverability is the use of a consistent naming pattern. Many of the common user questions don’t even need to come up once these conventions are understood and widely used. There are three elements of naming guidelines:

casing – use of the correct capitalization style
mechanical -- use nouns for classes, verbs for methods, etc.
word choice -- use consistent terms across libraries.

The following section lays out rules for the first two elements, and some philosophy for the third.


Capitalization Styles

The following section describes different ways of capitalizing identifiers. We will refer to these terms throughout the rest of this document.

Pascal Casing

This convention capitalizes the first character of each word. For example:

  BackColor

Camel Casing

This convention capitalizes the first character of each word except the first word. For example:

  backColor

Upper case

Only use all upper case for identifiers if it contains an abbreviation. See section x.y for a list of exceptions to this rule. For example:

  System.IO
  System.WinForms.UI

Capitalization summary

The following describes the capitalization of different types of identifiers.

Type | Case | Notes
Class | PascalCase |
Enum values | PascalCase |
Enum type | PascalCase |
Events | PascalCase | ends with EvenArgs
exception class | PascalCase | ends with Exception
Final Static field | PascalCase |
interface | PascalCase | starts with I
Method | PascalCase |
namespace | PascalCase |
property | PascalCase |
Instance Field | PascalCase |
parameter | camelCase |


Word Choice

  • Do avoid using class names duplicated in heavily used namespaces. Don’t use the following for a class name.
    System       Collections   Forms       UI
  • Do avoid using identifier that conflict with common keywords. For example, avoid using:
    abstract

    Access

    AddressOf

    Alias

    And

    AndThen

    Any

    Append

    Array

    As

    Assert

    Assign

    Attribute

    B

    base
    Base

    BF

    Binary

    bool
    break
    byte

    Boolean

    ByRef

    Byte

    ByVal

    Call

    CantOverride

    Case

    case
    catch
    char
    checked
    class

    Catch

    CBool

    Cbyte

    Ccur

    Cdate

    CDbl

    Cdec

    Cdecl

    Char

    ChDir

    Cint

    Circle

    Class

    CLng

    Close

    Compare

    Const

    const
    continue
    decimal
    default
    delegate

    CSng

    CStr

    CurDir

    CurDir$

    Currency

    Cvar

    CVDate

    CVErr

    Database

    Date

    Date$

    Debug

    Decimal

    Declare

    Default

    DefBool

    DefByte

    DefCur

    DefDate

    DefDbl

    DefDec

    DefInt

    DefLng

    DefObj

    DefSng

    DefStr

    DefVar

    Delegate

    Deprecated

    Dim

    Dir

    Dir$

    Do

    do
    double
    else
    enum
    explicit

    DoEvents

    Double

    Each

    Else

    ElseIf

    Empty

    End

    EndIf

    Enum

    Eqv

    Erase

    Error

    Error$

    Event

    Exit

    Explicit

    extern
    false
    finally
    fixed
    float

    F

    FALSE

    Finally

    Fix

    For

    for
    foreach
    goto
    if
    implicit

    Format

    Format$

    FreeFile

    Friend

    Function

    Get

    Global

    Go

    GoSub

    Goto

    Handles

    If

    Imp

    Implements

    Imports

    In

    Inherits

    Input

    Input$

    InputB

    InputB$

    InStr

    InStrB

    Int

    int
    interface
    is
    long
    namespace

    Integer

    Interface

    Is

    LBound

    Left

    Len

    LenB

    Let

    Lib

    Like

    Line

    LINEINPUT

    Load

    Local

    Lock

    Long

    Long8

    Loop

    LSet

    Me

    Mid

    Mid$

    MidB

    MidB$

    Mod

    Module

    MustOverride

    MyBase

    MyClass

    Name

    New

    new
    null
    object
    operator
    out

    Next

    Not

    Nothing

    Null

    Object

    On

    Open

    Option

    Optional

    Or

    OrElse

    Output

    Overloads

    override
    private
    protected
    public
    readonly

    Overrides

    ParamArray

    Preserve

    Print

    Private

    Property

    Protected

    Pset

    Public

    Put

    RaiseEvent

    Random

    Randomize

    Read

    ReDim

    ref
    return
    sealed
    sizeof
    short

    Rem

    Resume

    Return

    RGB

    RSet

    Scale

    Seek

    Select

    Set

    Sgn

    Shared

    Single

    Spc

    Static

    static
    string
    struct
    switch
    this

    Step

    Stop

    StrComp

    String

    String$

    Sub

    Tab

    Text

    Then

    throw
    true
    try
    typeof
    unchecked

    To

    TRUE

    Try

    Type

    TypeOf

    UBound

    union
    unsafe
    using
    variant
    virtual

    Unknown

    Unload

    Unlock

    Until

    Variant

    void
    while
    Abs

    Wend

    When

    While

    Width

    With

    WithEvents

    Write

    Xor
  • Weed this out. A lot of these are not VB revered words. Action: AlanC
  • Do not use abbreviations in identifiers (including parameter names).
  • Do use only generally accepted abbreviations in identifiers. For example, use the following abbreviations with the given casing:
    IO           Url        Http        UI       DB   Net Html   ID


Namespaces

Do avoid the possibility of two published namespaces having the same name by prefixing namespace names with a company name or other well-established brand. E.g., Microsoft.Office for the Office Automation Classes provided by Microsoft.

  • Do use PascalCasing, and separating logical components with periods (e.g., Microsoft.Office.PowerPoint). If your brand employs non-traditional casing, do follow the casing defined by your brand, even if it deviates from normal namespace casing (e.g., NeXT.WebObjects, and ee.cummings).
  • Do use plural namespace names where appropriate. For example, use System.Collections not System.Collection. Exceptions to this rule are brand names and abbreviations. For example, use System.IO not System.IOs.
  • Do not have name namespaces and classes with the same name. For example, don’t use “Debug” for a namespace name and have a class named “Debug”.

Overall Guidelines

The general rule for namespace naming we have been using for over a year now is: CompanyName.TechnologyName. That would mean we should expect to see Microsoft.Office, PowerSoft.PowerBuilder, etc. Notice this is merely a guideline, 3rd party companies are free to choice other names.

The Root Namespace

For the most part Microsoft follow the general guidance above for naming its technologies exposed to the managed world. We expect to see namespaces such as Microsoft.Office, Microsoft.Exchange, Microsoft.StreamingMedia, Microsoft.MSCom in the next couple of years.

The NGWS runtime has an obligation to identify certain technologies as being fundamental building blocks of the platform. These technologies conceptually belong more to the NGWS frameworks and runtime than to Microsoft. Consider the possibility of turning the NGWS frameworks and runtime over to a standards body: what technologies would go with it and what would Microsoft continue to hold on to?

System.*

System is the root namespace for the set of functionality that is part of the NGWS platform. The definition of what’s in a platform is by necessity subjective. There is a certainly a judgment call to be made on what is part of the platform and what is not, however we have outlined to following guidance for functionality in the System namespace.

Expose a generic set of functionality - The functionality must be infrastructure peaces upon which we expect our customers to build on. This restriction helps insure the integrality of System; that is, it cannot be polluted by any random functionality Microsoft builds. Essentially, it is an area we don’t expect competition in.

For example, the Office object model should NOT be in system because the functionality it exposes is not generic it is specific to the office implementation.

Be Rationalized with Similar Technology – We don’t want to expose more than one way to accomplish the same goal in the base system classes. Therefore what we do expose needs to be rationalized with similar functionality. For example we can not ship a messaging story in System that we already know will be completely replaced in V2. We can ship the building blocks or a small part of the messaging story in System.

Does not Re-Introduce Existing Functionality – If we already ship a way to solve a problem, do not add a new way in system.

  • Is Logically part of the Redistributable – It is clear that we will not ship a single redistributable for this platform. Different configurations on different platforms will included different subsets of functionality, but all will include some subset of System.*. If a set of classes is not in any redistributable they are not in System.
  • Be Free – That is, be an indirect source of income for Microsoft. Anything in the System namespace must be shipped under the exact same licensing agreements as the runtime. We will not charge for development or redistribution of anything in System. System classes may provide access to for-profit functionality (such as SQLServer), but the System classes them selves must be free.
  • Be compliant with the Common Language Specification – Follow the CLS guidelines on.
  • Be Secure – Be callably type-safe (assured by the CLS) and protect System resources as appropriate using code access security. System represent a set of functionality that can be safe called from a less than fully trusted application. This will be verified by a code inspection by the NGWS runtime Security team.
  • Follow the Design Guidelines - Follow the Managed Code Design Guidelines
  • Ship Source code – To aid in debugging, developer education etc. (Note, it may not be necessary to ship all source, just what is needed to meet the goals).
  • Depend only on System.* – To keep the set logically consistent, stuff in System can only publicly depend on other stuff in System.

Naming in System

  • When choosing names for the System namespace consider:
  • Names should be durable, concise, and objectively reflect semantics
  • Don’t use product names in a namespace if the purpose behind this is solely branding

Microsoft.*

This set of functionality represents the value Microsoft is adding to the NGWS platform. It is typically more domain specific (i.e. word processing, finical) and our customers find it valuable enough to pay for.

Although we certainly expect for the vast majority of functionality in the Microsoft.* namespace to have most of the attributes listed above, they are not required. The ones that are required of any managed library Microsoft ships is:

  • Be compliant with the Common Language Specification
  • Follow the Design Guidelines - Follow the Managed Code Design Guidelines on


Class and Class members

Class Naming Guidelines

  • Do name classes with nouns or noun phrases.
  • Do use PascalCasing.
  • Do use abbreviations in class names sparingly.
  • Do not use any class prefix (such as “C”).
  • Do not use an underscore.
    public class FileStream {
}
public class Button {
}
public class String {
}

Interface Naming Guidelines

  • Do name interfaces with nouns or noun phrases, or adjectives describing behavior. E.g., IComponent (descriptive noun), ICustomAttributeProvider (noun phrase), and IPersistable (adjective).
  • Do use PascalCasing.
  • Do use abbreviations in interface names sparingly.
  • Do not use the underscore.
  • Do prefix interface names with the letter “I”, to indicate that the type is an interface.
    Don’t prefix class names with the letter “C”. Occasionally, it will be necessary to have a class name that begins with I that is not an interface. This is acceptable as long as the next character is lower case. Such as IdentityStore.
  • Do use similar names when defining a class/interface pair where the class is a standard implementation of the interface. The names should differ only by the “I” prefix on the interface name. E.g., these guidelines are used for the interface IComponent and its standard implementation, the class Component.
    public interface IComponent {
}
public class Component : IComponent {
}
public interface IServiceProvider{
}
public interface IFormatable {
}

Enum Naming Guidelines

  • Do use PascalCasing for enums.
  • Do use PascalCasing for enum value names.
  • Do use abbreviations in enum names sparingly
  • Do not use a prefix on enum names (e.g. adXXX for ADO enums, rtfXXX for rich text enums, etc.).
  • Do not use any “Enum” suffix on enum types.
  • Do use a singular name for enums
  • Do use a plural name for bit fields
  • Do define enumerated values using an enum if they are used in a parameter or property. This allows the tool to know the possible values for a property or parameter.
    public enum FileMode{
    Create,
    CreateNew,
    Open,
    OpenOrCreate,
    Truncate
}
  • Do use the Flags custom attribute if the numeric values are meant to be bitwise or'ed together
    [Flags]
public enum Bindings {
    CreateInstance,
    DefaultBinding,
    ExcatBinding,
    GetField,
    GetProperty,
    IgnoreCase,
    InvokeMethod,
    NonPublic,
    OABinding,
    SetField
    SetProperty,
    Static
}

    An exception to this rule is when encapsulating a Win32 API, it's common to have internal definitions that come from a Win32 header. It's ok to leave these with the Win32 casing, which is usually all-caps.

  • Do use Int32 as the underlying type of an enum.
    An exception to this rule is if the enum represents flags and there are many flags (>32) or the enum may grow to many flags in the future or the type needs to be different than int for backward compatibility.
  • Do use enums only if the value can be completely expressed as a set of bitflags. Do not use enums for open sets (such as Operating system version etc).

ReadOnly and Const Field Names

  • Do name static fields with nouns, noun phrases, or abbreviations for nouns.
  • Do name static fields with PascalCasing.
  • Do not prefix static field names Hungarian type notation.

Parameter Names

  • Do use descriptive parameter names. Parameter names should be descriptive enough that in most scenarios the name of the parameter and its type can be used to determine its meaning.
  • Do name parameters with camelCasing.
  • Do prefer names based on a parameter’s meaning to names based on the parameter’s type. We expect development tools to provide the information about type in a handy manner, so the parameter name can be put to better use describing semantics rather than type. Occasional use of type-based parameter names is entirely appropriate.
  • Do not use reserved parameters. If more data is need in the next version a new overload can be added.
  • Do not prefix field names Hungarian type notation.
    Type GetType (string typeName)
string Format (string format, object [] args)

Method Naming Guidelines

  • Do name methods with verbs or verb phrases.
  • Do name methods with PascalCasing
    RemoveAll(), GetCharArray(), Invoke()

Property Naming Guidelines

  • Do name properties using noun or noun phrase
  • Do name properties with PascalCasing
  • Recommendation not using properties and types with the same name.

    Defining a property with the same name as a type can cause some ambiguity languages. It is best to avoid this ambiguity unless there is a clear justification for not doing so.

    For example: System.WinForms has an Icon property even though there is an Icon class because Form.Icon is so much easier to understand than Form.FormIcon or Form.DisplayIcon etc.
    However, System.WinForms.UI.Control has a color property. Because there is a Color class, the Color property was named BackgroundColor as it’s a more meaningful name that does not conflict.

    Text, LastIndex, Value[5]

Event Naming Guidelines

  • Do name Event handlers with the “EventHandler” suffix.
    public delegate void MouseEventHandler(object sender, MouseEvent e);
  • Do use two parameters named sender and e.

    The sender parameter represents the object that raised the event. The sender parameter is always of type object, even if it is possible to employ a more specific type.

    The state associated with the event is encapsulated in an instance of an event class named e. Use an appropriate and specific event class for its type.

    public delegate void MouseEventHandler(object sender, MouseEvent e);
  • Do name event argument classes with the “EventArgs” suffix.
    public class MouseEventArgs : EventArgs {
    int x;
    int y;
    public MouseEventArgs(int x, int y)
       { this.x = x; this.y = y; }
    public int X { get { return x; } }
    public int Y { get { return y; } }
}
  • Do use present and past tense for pre and post events.
  • Consider naming events with a verb.
  • Do name event names that have a concept of pre and post will be prefixed using the present and past tense. For example, a close event that could be canceled would have a Closing and Closed event.


Case Sensitivity

  • Don’t use names that require case sensitivity. Components must be fully usable from both case-sensitive and case-insensitive languages. Since case-insensitive languages cannot distinguish between two names within the same context that differ only by case, components must avoid this situation.

Examples of what not to do:

  • Don’t have two namespaces whose names differ only by case.
    namespace ee.cummings;
namespace Ee.Cummings;
  • Don’t have a function with two parameters whose names differ only by case.
    void foo(string a, string A)
  • Don’t have a namespace with two types whose names differ only by case.
    System.WinForms.Point p;
System.WinForms.POINT pp;
  • Don’t have a type with two properties whose names differ only by case.
    int Foo {get, set};
int FOO {get, set}
  • Don’t have a type with two methods whose names differ only by case.
    void foo();
void Foo();


Avoiding type name confusion

Different languages use different terms to identify the fundamental managed types. Designers must take care to avoid language-specific terminology. This section describes a set of rules that help avoid type name confusion.

  • Do use semantically interesting names rather than type names.
  • In the rare case that a parameter has no semantic meaning beyond its type, use a generic name. E.g., a class that supports writing a variety of data types into a stream might have:
    void Write(double value);
void Write(float value);
void Write(long value);
void Write(int value);
void Write(short value);

    rather than a language-specific alternative such as:

    •

void Write(double doubleValue);
void Write(float floatValue);
void Write(long longValue);
void Write(int intValue);
void Write(short shortValue);

  • In the extremely rare case that it is necessary to have a uniquely-named method for each fundamental data type, do use the following universal  type names:

C# type name
 | VB type name
 | VC type name
 | IL representation
 | Universal type name

sbyte
 | NA
 | char
 | I1
 | SByte

byte
 | Byte
 | unsigned char
 | U1
 | Byte

short
 | Integer
 | short
 | I2
 | Int16

ushort
 | NA
 | unsigned short
 | U2
 | UInt16

int
 | Long
 | int
 | I4
 | Int32

NA
 | unsigned int
 | U4
 | Uint32

long
 | LongLong
 | __int64
 | I8
 | Int64

ulong
 | NA
 | Unsigned __int64
 | U8
 | UInt64

float
 | Single
 | float
 | R4
 | Single

double
 | Double
 | double
 | R8
 | Double

bool
 | Boolean
 | bool
 | I4
 | Boolean

char
 | Char
 | wchar_t
 | U2
 | Char

string
 | String
 | String
 | System.String
 | String

object
 | Object
 | Object
 | System.Object
 | Object

    •

E.g., a class that supports reading a variety of data types from a stream might have:

double ReadDouble();
float ReadSingle();
long ReadIn64();
int ReadInt32();
short ReadInt16();

rather than a language-specific alternative such as:

double ReadDouble();
float ReadFloat();
long ReadLong();
int ReadInt();
short ReadShort();


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Class Members

© 2000 Microsoft Corporation. All rights reserved.


Property Usage Guidelines

  • Do use PascalCasing for Property names
    public class Foo {
    public Color BackColor { }
}
  • Do not use Hungarian notation
  • Do see section 0 on choosing between properties and methods.
  • Recommendation not using properties and types with the same name.

    Defining a property with the same name as a type can cause ambiguity in some languages. It is best to avoid this ambiguity unless there is a clear justification for not doing so.

    For example: System.WinForms.Form has an Icon property even though there is an Icon class because Form.Icon is so much easier to understand than Form.FormIcon or Form.DisplayIcon etc.
    However, System.WinForms.UI.Control has a color property. Because there is a Color class, the Color property was named BackgroundColor as it’s a more meaningful name that does not conflict.

  • Do preserve the previous value if a property set throws an exception.
  • Do allow properties to be set in any order. Properties should be stateless with respect to other properties.

    It is often times the case that a particular feature of an object will not take effect until the developer has specified a particular set of properties, or until an object has a particular state. Until the object is in the correct state, the feature is not active. Once in the right state, then the feature automatically activates itself without requiring an explicit call. The semantics are be the same no matter what order the developer sets the property values or how the developer get's the object into the active state.

    For example, a TextBox control may have two related properties: DataSource and DataField. DataSource specifies the table name, and DataField specifies the the column name. Once both are specified, the control can automatically bind data from the table into the Text property of the control. However, the developer is able to set the properties in any order, for example:

    TextBox t = new TextBox();
t.DataSource = "Publishers";
t.DataField = "AuthorID";
// data binding feature now active

    The following is equivalent:

    TextBox t = new TextBox();
t.DataField = "AuthorID";
t.DataSource = "Publishers";
// data binding feature now active

    In addition, the developer can set the properties to null, meaning 'not specified':

    TextBox t = new TextBox();
t.DataField = "AuthorID";
t.DataSource = "Publishers";
// data binding feature now active

t.DataSource = null;
// data binding feature now inactive

    Implement this by tracking the state of the data binding feature and automatically activating or deactivating it at the appropriate times. For example:

    public class TextBox {
    string dataSource;
    string dataField;
    boolean active;

    public string DataSource {
        get {
            return dataSource;
        }
        set {
            if (value != dataSource) {
                // set the property value first, in case activate fails
                dataSource = value;

                // update active state
                SetActive(dataSource != null && dataField != null);
            }
        }
    }

    public string DataField {
        get {
            return dataField;
        }
        set {
            if (value != dataField) {
                // set the property value first, in case activate fails
                dataField = value;

                // update active state
                SetActive(dataSource != null && dataField != null);
            }
        }
    }

    void SetActive(boolean value) {
        if (value != active) {
            if (value) {
                Activate();
                Text = dataBase.Value(dataField);
            }
            else {
                Deactivate();
                Text = "";
            }

            active = value; // set active only if success
        }
    }

    void Activate() {
        // open database
    }

    void Deactivate() {
        // close database
    }
}

    In the previous example, the following expression determines whether or not the object is in a state in which the data binding feature can activate itself:

     dataSource != null && dataField != null

    It may make sense to make this into a method which will determine whether the object can be activated given it's current state, and then activate as necessary.

    void UpdateActive {
    SetActive(dataSource != null && dataField != null);
}

    The expression that appears in this method is an indicator as to what parts of the object model that needs to be examined in order to enforce these state transitions. In this case, the DataSource and DataField properties are affected.

  • Add and example that is not UI specific
  • Recommendation raising PropertyChanged events for component authors.

    If there is a good scenario for a user wanting to know when a property of another component is changing programmatically, then the component should raise a PropertyChanged event for the property. For example, a Control may raise a PropertyChanged event when the Control's position changes. This would allow a tooltip provider Control to respond to the change in position and update the location of the tooltip window. The PropertyChanged event is raised after the property value has changed. There is a protected helper routine RaisePropertyChanged which is used to raise this event. However, it is unlikely to be worth the overhead to raise a PropertyChanged event for a Hashtable item addition.

    class Control : Component {
    Rect position;

    [Bindable(True)]
    public Rect Position {
        get { return position; }
        set {
            if (!position.Equals(value)) {
                position = value;
                RaisePropertyChangedEvent(“position”);
            }
        }
    }
}
    • ChrisAn has some additional comments to make on this section

    Data binding uses this pattern to allow two way binding of the property. If these events do not exist, then data binding works in one direction (if the database changes, the property is updated). Each property that raises the PropertyChanged event should provide metadata to indicate that the property supports data binding.

  • Recommendation raising change/changing events if the value of a property changes via external forces.

    If a property value changes via some external force (e.g. not by calling methods on the object), then raise events telling the developer that the value is changing and has changed. An good example is the Text property of a Edit control. When the user types, the property value automatically changes.

    An event is raised before the value of the property has changed. It does not pass the old or new value, but can be canceled by the developer by throwing an exception. The name of the event is the name of the property followed by Changing. For example:

    class Edit : Control {
    public string Text {
        get { return text; }
        set {
            if (text != value) {
                OnTextChanging(Event.Empty);
                text = value;
            }
        }
    }

}

    An event is also raised after the value of the property has changed. It can not be cancelled. The name of the event is the name of the property followed by Changed. The generic PropertyChanged event should also be raised (see previous section on PropertyChanged events). The pattern for raising both of these is to raise the specific event from the OnPropertyChanged method. For example:

    class Edit : Control {
    public string Text {
        [Bindable(True)]
        public string Text {
            get { return text; }
            set {
                if (text != value) {
                    OnTextChanging(Event.Empty);
                    text = value;
                    RaisePropertyChangedEvent(Edit.ClassInfo.text);
                }
            }
        }

        protected void OnPropertyChanged(PropertyChangedEvent e) {
            if (e.PropertyChanged.Equals(Edit.ClassInfo.text))
                OnTextChanged(Event.Empty);
            if (onPropertyChangedHandler != null)
                onPropertyChangedHandler(this, e);
        }
    }
}

    There are cases when the underlying property value is not stored as a field and thus it's value is more difficult to track changes to. When raising the changing event, find all the places that the property value can change and provide the ability to cancel. For example, the previous Edit control example is not actually accurate because the Text value is actually stored in the HWND. In order to raise the TextChanging event, we must examine Windows messages for when the text may change, and allow for an exception thrown in OnTextChanging to cancel. If it is too difficult to provide a changing event, it's reasonable to just support Changed.

Properties vs. Methods

Designers sometimes face a decision between a property and a method. Guidelines for choosing between these options:

Do use a Property:

  • If the member has a logical backing store.

    For example:

    string Type.Name
//should be a property, string Name because the name logical attribute of a //class but:
Guid Guid.GetNext()
//should not be a property, because a Guids don’t naturally have a “next” Guid.

Do use a Method:

  • If the operation is a conversion (such as Object.ToString())
  • If the getter has an observable side effect.
  • If calling the member twice in succession results in different results.
  • If order of execution is important.
    class Connection
{
   //It’s good for these three to be properites as they can be set in any order
   String DNSName {get{};set{};}
   String UserName {get{};set{};}
   String Password {get{};set{};}

   //Howver, it would be bad for this to be a properity because it has to be done in a particular order
   //(namly after the others have been set).  We use a method here.
   bool Execute ();
}
  • If the member is static but it returns a mutable value.
  • If the member must return an array.

    Properties that return arrays can be very misleading. Usually it is necessary to return a copy of the internal array so that the user cannot change internal state. This, coupled with the fact that a user could easily assume it's an indexed property, leads to really inefficient code such as:

    Type type = //get a type somehow
for (int i = 0; i < type.Methods.Length; i++)
{
   if (type.Methods[i].Name.Equals ("foo"))
   {...}
}

    In this code example, each call to the Methods property creates a copy of the array. That would be n+1 copies for this loop.

Read-Only and Write-Only Properties

  • Do use Read-only properties when the user cannot change the logical backing data field.
  • Do not use Write-only properties.

Indexed Property Usage

  • Do use only one indexed property per class and make it the default indexed property for that class.
  • Do not use non default Indexed properties.
  • Do use the name “Item” for indexed properties unless there is an obviously better name (for example a Chars property on String)
  • Do use indexed properties when the logical backing store is an array
  • Do not mix indexed properties and overloaded methods.
    MethodInfo Type.Method[string name]
MethodInfo Type.GetMethod (string name, boolean ignoreCase)


Event Usage Guidelines

  • Do name events using PascalCasing
  • Do not use Hungarian notation
  • Do use the terminology “raise” for events rather than fired.

    When referring to events in documentation, use the 'an event was raised' terminology instead of 'an event was fired'.

  • Consider naming events with a verb.
  • Do nameEvent handler names ending in “EventHandler”.
    public delegate void MouseEventHandler(object sender, MouseEventArgs e);
  • Do use a return type of void for Event handlers.
    public delegate void MouseEventHandler(object sender, MouseEventArgs e);
  • Do use two parameters named sender and e.

    The sender parameter represents the object that raised the event and calls the delegate. The sender parameter is always of type object, even if it is possible to employ a more specific type.

    The state associated with the event is encapsulated in an instance of an event class named e. Use an appropriate and specific event class for its type.

    public delegate void MouseEventHandler(object sender, MouseEvent e);
  • Do make Event classes extend System.EventArgs.
     public class MouseEvent : EventArgs {
}
  • Do name Event class names end in “EventArgs”.
    public class MouseEventArgs : EventArgs {
    private int x;
    private int y;
    public MouseEventArgs(int x, int y)
       { this.x = x; this.y = y; }
    public int X { get { return x; } }
    public int Y { get { return y; } }
}
  • Do implement AddOnEventName and RemoveOnEventName for each event.

    Each event has a set of methods that are used to add and remove event handlers. These start with AddOn and RemoveOn followed by the name of the event. The parameter is called handler.

    class Button : Control {
    MouseEventHandler onClickHandler;

    public void AddOnClick(MouseEventHandler handler) {
        onMouseEvent = (MouseEventHandler)Delegate.Combine(onMouseEvent, handler);
    }

    public void RemoveOnClick(MouseEventHandler handler) {
        onMouseEvent = (MouseEventHandler)Delegate.Remove(onMouseEvent, handler);
    }
}

    Classes with many events can store the handlers more efficiently than just declaring a field for each, since the developer will probably not handle all the events for a given class all at once. For example, the Component class implements a more space efficient way of storing handlers which is exposed via the protected (family) methods AddEventHandler and RemoveEventHandler:

    public class Foo : Component {
    static final object EventClick = new object();
    static final object EventMouseDown = new object();
    static final object EventMouseUp = new object();
    static final object EventMouseMove = new object();

    public void AddOnClick(MouseEventHandler handler) {
        AddEventHandler(EventClick, handler);
    }

    public void RemoveOnClick(MouseEventHandler handler) {
        RemoveEventHandler(EventClick, handler);
    }
}
  • Do use present and past tense for pre and post events.
  • Do name event names that have a concept of pre and post will be prefixed using the present and past tense. For example, a close event that could be canceled would have a Closing and Closed event.
  • Do use a protected (family) virtual method to raise each event.

    For each event on a class, include a corresponding protected (family) virtual method that raises the event. This is not appropriate for sealed classes, because they cannot be subclassed. The purpose of the method is to provide a way for a subclass to handle the event using an override. This is more natural than using delegates in the case where the developer is sub classing. The name of the method is OnEventName where EventName is the name of the event being raised. For example:

    public class Button {
   ButtonClickHandler onClickHandler;

   protected virtual void OnClick(ClickEvent e) {
       if (onClickHandler != null)
           onClickHandler(this, e);   // call the delegate if non-null
   }
}

    The subclass can choose not to call the base during the processing of OnEventName. Be prepared by this by not including any processing in the OnEventName method that is required for the base class to work correctly.

  • Do assume that anything goes in an event handler.

    Classes are ready for the handler of the event to do pretty much anything, and in all cases the object is left in a good state after the event has been raised. Consider using try/finally at the point where the event is raised. Since the developer can call back on the object to perform other actions, do not assume anything about the object state when control returns to the point at which the event was raised. For example:

    public class Button {
   ButtonClickHandler onClickHandler;

   protected void DoClick() {
       PaintDown(); // paint button in depressed state

       try {
           OnClick(); // call event handler
       }
       finally {
           // window may be deleted in event handler
           if (windowHandle != null)
               PaintUp(); // paint button in normal state
       }

       PaintUp(); // paint button in normal state
   }

   protected virtual void OnClick(ClickEvent e) {
       if (onClickHandler != null)
           onClickHandler(this, e);
   }
}
  • Do raise a CancelEvent to allow the developer to control behavior.
  • Use or extend System.ComponentModel.CancelEvent to allow the developer to control the default behavior of an object.

    For example, the treeview control fires an CancelEvent when the user is about to edit a node label. The developer can use this event to prevent a node from being edited:

    public class Form1 : Form {
    TreeView treeView1 = new TreeView();

    void treeView1_BeforeLabelEdit(object source, NodeLabelEditEvent e) {
        e.cancel = true;
    }
}

    Note that in this case, no error is generated to the user. The label is just read only.

    In cases where the developer would like to cancel the operation and return an exception, then CancelEvent is not appropriate. In these cases, the event does not decend from CancelEvent, and the developer just raises an exception inside of the event handler in order to cancel. For example, the user may want to write validation logic in a edit control:

    public class Form1 extends Form {
    TreeView edit1 = new Edit();

    void edit1_TextChanging(object source, Event e) {
        throw new RuntimeException("Bad edit");
    }


Method Usage Guidelines

  • Do name methods with PascalCasing
  • Do not use Hungarian notation
  • Do default to having non-virtual methods
  • Do see the Inheritance section of this doc for more information.

Method Overloading Guidelines

Definition: When there are two members accessible from the same class with the same name but different signatures.

  • Do use overloading when providing different methods that do semantically the same thing.
  • Do favor overloading to default arguments (as default arguments are not in the CLS).
    int String.IndexOf (String name);
int String.IndexOf (String name, int startIndex);
  • Do use a default values correctly.
    In a family of overloaded methods the complex method should use parameter names that indicate a change from the default state assumed in the simple method.
    For instance, in the example below, method one assumes the look-up will not be case sensitive. In method two we choice the name “ignoreCase” rather than “caseSensitive” because the former indicates how we are changing from the default behavior.
    1: MethodInfo Type.GetMethod(String name);  //ignoreCase = false
2: MethodInfo Type.GetMethod (String name, boolean ignoreCase);

    It is very common to use a zero’ed state for the default value (such as: 0, 0.0, false, “”, etc).

  • Do be consistent in ordering and naming of method parameters.

    It's common to have a set of overloaded methods with an increasing number of parameters to let the developer specify a desired level of information. The more parameters specified, the more detail that is specified. All the related methods have a consistent parameter order and naming. Each of the method variations has the same semantics for their shared set of parameters.

    public class Foo {
    readonly string defaultForA = "default value for a";
    readonly string defaultForB = "default value for b";
    readonly string defaultForC = "default value for c";

    public void Bar() {
        Bar(defaultForA, defaultForB, defaultForC);
    }

    public void Bar(string a) {
        Bar(a, defaultForB, defaultForC);
    }

    public void Bar(string a, string b) {
        Bar(a, b, defaultForC);
    }

    public void Bar(string a, string b, string c) {
        // core implementation here
    }
}

    This consistency is present even if the parameters have different types.

    The only method in such as a group that may be virtual is the one that has the most parameters.

  • Do use overloading for variable numbers of parameters.

    Where it is appropriate to have variable numbers of parameters to a method, use the convention of declaring N methods with increasing numbers of parameters, and also provide a method which takes an array of values for numbers greater than N. N=3 or N=4 is appropriate for most cases. For example:

    public class Foo {
    public void Bar(string a) {
        Bar(new string[] {a});
    }

    public void Bar(string a, string b) {
        Bar(new string[] {a, b});
    }

    public void Bar(string a, string b, string c) {
        Bar(new string[] {a, b, c});
    }

    public void Bar(string[] args) {
        // core implementation here
    }
}


Constructor Usage Guidelines

  • Do have only a default private constructor if there are only static methods and properties on a class.
    public sealed class Runtime
{
   private Runtime(); //prevents the class from being created
   //…
}
  • Do minimal work in the constructor

    Constructors do not do much work other than to capture the constructor parameter(s). The cost is delayed until the user uses a particular feature of the instance.

  • Do provide a protected (family) constructor for subclass able types.
  • Recommendation not having an empty constructor for structs (value types)

    If no constructor is supplied the runtime will zero init all the fields of the structs. This makes array and static field creation faster.

  • Do use parameters in constructors as shortcuts for setting properties.

    There should be no difference in semantics between using the empty constructor followed by some property sets, or by using a constructor with multiple arguments. For example, the following is equivalent:

    Foo foo = new Foo();
foo.A = "a";
foo.B = "b";

Foo foo = new Foo("a");
foo.B = "b";

Foo foo = new Foo("a", "b");
  • Do be consistent in ordering and naming of constructor parameters.

    A common pattern for constructor parameters is to have an increasing number of parameters to let the developer specify a desired level of information. The more parameters that are specified, the more detail is specified. For all the constructors, there is a consistent order and naming of the parameters.

    public class Foo {
    private const string defaultForA = "default value for a";
    private const string defaultForB = "default value for b";
    private const string defaultForC = "default value for c";

    private string a;
    private string b;
    private string c;

    public Foo():this(defaultForA, defaultForB, defaultForC) {}
    public Foo(string a) : this(a, defaultForB, defaultForC) {}
    public Foo(string a, string b) : this(a, b, defaultForC) {}
    public Foo(string a, string b, string c)
    { /* do work here */ }
}


Field Usage Guidelines

  • Do not use instance fields that are public or protected (family).

    Consider providing property accessors for fields instead of making them public.

    public struct Point {
    int x;
    int y;

    public Point(int x, int y) {
        this.x = x;
        this.y = y;
    }

    public int X {
        get { return x; }
        set { x = value; }
    }

    public int Y {
        get { return y; }
        set { y = value; }
    }
}

    To expose a field to a subclass, use a protected (family) property that returns the value of the field.

    public struct Control : Component {
    int handle;

    protected int Handle {
        get { return Handle; }
    }
}

    By not exposing fields directly to the developer, the class can be versioned more easily. The reasons are (a) that a field cannot be changed to a property while maintaining binary compatibility and (b) that the presence of executable code in getters and setters allows later improvements, e.g., demand-creation of an object upon usage of the property, or a property change notification.

  • Recommendation using read-only static fields instead of properties where the value is a global constant.
    public struct Int32
  public static readonly int MaxValue = 2147483647
  public static readonly int MinValue = -2147483648
}
  • Do not abbreviate field names.

    Spell out all the words used in a field name. Only use abbreviations if developers generally understand them. Do not use uppercase for field names. For example:

    class Foo {
    string url;
    string destinationUrl;
}
  • Do not use Hungarian notation for field names. Good names describe semantics, not type.
  • Do not use a prefix for field names.
  • Do not use a prefix for static field names.
  • Do not include a prefix on a field name, for example 'g_' or 's_' to distinguish static vs. non-static fields.
  • Do use public static readonly fields for predefined object instances.

    If there are predefined instances of an object, declare them as public readonly static fields of the object itself. Use pascal casing because they are public. For example:

    public struct Color {
    public static readonly Color Red = new Color(0x0000FF);
    public static readonly Color Green = new Color(0x00FF00);
    public static readonly Color Blue = new Color(0xFF0000);
    public static readonly Color Black = new Color(0x000000);
    public static readonly Color White = new Color(0xFFFFFF);

    public Color(int rgb) { }

    public Color(byte r, byte g, byte b) { }

    public byte RedValue { get {...} }
    public byte GreenValue { get {...} }
    public byte BlueValue { get {...} }
}


Parameter Usage Guidelines

  • Do use camel casing for parameter names.
    public class Foo {
    void Repaint(int backColor, int foreColor) { }
}
  • Do not use Hungarian notation for parameter names. Good names describe semantics, not type.
  • Do check arguments for validity.

    Perform argument validation for every public or protected (family) method and property setter, and throw meaningful exceptions to the developer. The System.ArgumentException exception is used in these cases.

    class Foo {
    public int Count {
        get { return count; }
        set {
            if (count < 0 || count >= MaxValue)
                throw new ArgumentOutOfRangeException(Sys.GetString("InvalidArgument", "value", count.ToString());
        }
    }

    public void Select(int start, int end) {
        if (start < 0)
            throw new ArgumentException(Sys.GetString("InvalidArgument", "start", start.ToString()));
        if (end < 0)
            throw new ArgumentException(Sys.GetString("InvalidArgument", "end", end.ToString()));
    }
}

    Note that the actual checking does not necessarily have to happen in the public/protected (family) method itself (e.g. it could happen at a lower level in some private routines). The bottom line is that the entire surface area that is exposed to developers checks for valid arguments.


Type Usage Guidelines

Types are the unit of encapsulation in the runtime. There are three basic kinds of Types. Classes are by far the most common kind of types used. Classes can be abstract (subclasses must provide and implementation) and sealed classes (no subclass are allowed). Value types describe values that are represented as sequences of bits stored on the stack. An interface type is a partial description of a value, potentially supported by many object types.


Class Usage Guidelines

  • Do favor using classes over any other type. Other types systems (such as Com classic) used Interfaces heavily. In the managed world, Classes are the most commonly used element.

Base Class Usage Guidelines

Base classes are useful way to group objects that share a common set of functionality. Base classes can provide a default set of functionality, while allowing customization though extension.

  • Recommendation using base classes.

    Abstract classes are inherently more versionable than interfaces because methods can be added with a default implementation without requiring all the derived classes to implement the method.

    However, this is only appropriate where the object that needs to implement the interface would not have any other base class. For example, make IByteStream an interface so a class an implement multiple stream types. Make ValueEditor an abstract class because ValueEditor subclasses classes have no other purpose than to edit values.

  • Recommendation providing customization through family (protected) methods.

    The public interface of a base class should provide a rich set of functionality for the consumer of that class. However, customizers of that class often want to implement the fewest methods possible to provide that rich set of functionality to the consumer. To meet this goal provide a set of sealed public methods that call through to a single family method with the “Impl” suffix that provides implementations for such a method.

    Public Control
{
   //…
        public void sealed SetBounds(int x, int y, int width, int height)
         {
           ….
           SetBoundsImpl (…);
         }
        public void sealed  SetBounds(int x, int y, int width, int height, BoundsSpecified specified)

         {
           ….
           SetBoundsImpl (…);
         }

        protected virtual void SetBoundsImpl(int x, int y, int width, int height, BoundsSpecified specified)
       {
           //do the real work here.
       }
}
  • Do define a protected (family) constructor on all abstract classes. Many compilers will insert a public constructor if you don’t. This can be very misleading to users as it can only be called from subclasses.

Sealed Class Usage Guidelines

  • Do use sealed classes if sub classing will not be required.
  • Do use sealed classes if there are only static methods and properties on a class.
    public sealed class Runtime
{
   private Runtime(); //prevents the class from being created
   public String GetCommandLine() {}
   //…
}


Value Type Usage Guidelines

Struct Usage Guidelines

  • Recommendation use structs for types that meet any of the following criteria:
    • Act like primitive types and,
    • Inherit directly from ValueType and,
    • Have an instance size under 16 bytes and,
    • Are immutable and,
    • Value semantics are desirable
    public struct Int32: IComparable, IFormattable
{
   //Constructors
 public Int32 (int value);
   //Fields
 public readonly const int MinValue = -2147483648;
 public readonly const int MaxValue = 2147483647;
   //Properties
 public int Value { virtual get; }
   //Methods

 public static string ToString(int i);
 public override string ToString();
 public static int Parse(string s, NumberStyle style, NumberFormatInfo info);
 public static int Parse(string s, NumberStyle style);
 public static int Parse(string s);
 public override int GetHashCode();
 public static int FromString(string s);
 public virtual string Format(string format, IServiceObjectProvider sop);

 public static string Format(int value, string format, NumberFormatInfo info);

 public static string Format(int value, string format);
 public override bool Equals(object obj);
 public virtual int CompareTo(object object);
 public static object Box(int value);
}
  • Do not provide a default constructor. The runtime will “insert” one that initializes the value to an all zero state. This allows arrays of structs to be created more efficiently.
  • Do allow a state where all instance data is zero filled to be valid without running the constructor.

Enum Usage Guidelines

  • Do use Enums to strongly type parameters, property and return type.

    Always define enumerated values using an enum if they are used in a parameter or property. This allows the tool to know the possible values for a property or parameter.

        public enum SpecialFolder  {
        ApplicationData,
        UserData ,
        Cookies,
        DesktopDirectory,
        Favorites,
        History,
        InternetCache,
        Programs,
        Recent,
        Sendto,
        StartMenu,
        Startup,
        Templates
  }

Type.CreateInstance (IBinder binder, BindingEnum bindAttributes, Variant [] args)
  • Do use the System.Flags custom attribute for enums who’s numeric values are meant to be bitwise or'ed together.
    [Flags(true)]
public enum Bindings
{
   IgnoreCase      = 0x01;
   NonPublic      = 0x02;
   Static         = 0x04;
   InvokeMethod   = 0x0100;
   CreateInstance   = 0x0200;
   GetField      = 0x0400;
   SetField      = 0x0800;
   GetProperty      = 0x1000;
   SetProperty      = 0x2000;
   DefaultBinding      = 0x010000;
   DefaultChangeType   = 0x020000;
   Default            = DefaultBinding | DefaultChangeType;
   ExactBinding      = 0x040000;
   ExactChangeType      = 0x080000;
   BinderBinding      = 0x100000;
   BinderChangeType   = 0x200000;
  • Do not assume enum arguments will be in the defined range. (Do argument validation)
    public  void SetColor (Color color)
{
   if (!EnumInfo.IsValid (typeof(color), color) throw new ArgumentOutOfRangeException();
}
  • Do favor using enums over static final constants
  • Do use Int32 as the underlying type of an enum.

    unless:

    The enum represents flags and there are many flags (>32) or the enum may grow to many flags in the future.

    The type needs to be different than int for backward compatibility.

  • Do not use a non-integral enum type. Only use Byte, Int16, Int32, Int64.
  • Do not define methods, properties or events on enums.
  • Do not use any “Enum” suffix on enum types.


Interface Usage Guidelines

  • Recommendation using a class or abstract class.
  • Do use interfaces to provide extensibility and customizability.
  • Recommendation providing a default implementation of an interfaces where appropriate. For example: System.Reflection.Expando is the default implementation of the System.Reflection.IExpando interface
  • Do see section 0 on the versioning issues with interfaces and abstract classes
  • Do implement interfaces privately if you only want the interface methods available when cast to that interface.
    public class ButtonEnumerator: private IEnumerator
{
 public boolean GetNext();
 public Button GetValue ();
 public Object IEnumerator.GetValue ();
}


Delegate Usage Guidelines

Delegates are a powerful tool for the managed code object model designer to encapsulate method calls. They are used in two basic areas:

Event notifications – See section x.y on event conventions

Callbacks – Passed to a method so that user code can be called multiple times during execution to provide customization. The classic example of this is passing a Compare callback to a sort routine. These methods should use the “Callback” conventions

  • Do use an Event design pattern for events (even if it is not UI related).
  • Do use end callbacks with the suffix “Callback”.


Error Raising and Handling

  • Do end Exception class names end in “Exception”.
    public class FileNotFoundException : Exception {
}
  • Recommendation using at least these common constructors.
    public class XxxException : Exception {
    XxxException() { }
    XxxException(string message) { }
    XxxException(string message, Exception inner) { }
}
  • Recommendation using the predefined exceptions types. Only define new exception types for programmatic scenarios.

    Introduce a new exception class so a programmer can take a different action in code based the exception class. Do not define a new exception class unless there is a scenario for a developer needing the exception class.

    For example, it makes sense to define FileNotFoundException because the programmer might decide to create the missing file, whereas FileIOException is not something that would typically be handled specifically in code.

  • Do make sure that in exceptions which don't have an explicit message (read: non-null), the message property traverses down the tree to the next real text.
    SethDe: add an example
  • Do not derive new exceptions from the base class Exception. The base class will have multiple subclasses according to individual namespace.
  • Do remember that under the system exception will be another container exception called Core, which will contain the handful of EE exceptions that bring the system completely down. Users cannot throw these exceptions.
  • Do group new exceptions off the base class Exception by namespace. For example, there will be subclasses for XML, IO, Collections, etc.

    Each of these areas will subclass their own exceptions as appropriate. Any exceptions that other library or application writers wish to add will extend exception directly. A single name for all related exceptions should be made, and all exceptions related to that application or library should extend from that bucket

  • Do use a localized description string. When the user sees an error message, it will be derived from the description string of the exception that was thrown, and never from the exception class. Include a description string in every exception.
  • Do use grammatically correct error messages including ending punctuation.

    Each sentence in a description string of an exception ends in a period. Code that generically displays an exception message to the user handles the case where a developer forgot the final period.

  • Do provide Exception properties for programmatic access. Include extra information in an exception (besides the description string) only when there is a programmatic scenario where that additional information is useful. It's rare to need to include additional information in an exception.
  • Do throw exceptions only in exceptional cases.
    • Do not use exceptions for normal or expected errors.
    • Do not use exceptions for normal flow of control.
  • Recommendation returning null for extremely common error cases. For example, File.Open returns a null if the file is not found, but throws and exception if the file is locked.
  • Do design classes such that in the normal coarse of use there will never be an exception thrown. For example, a FileStream class exposes another way of determining if the end of file has been reached to avoid the exception that will be thrown if the developer reads past the end of the file. For example:
    class Foo {
    void Bar() {
        FileStream stream = File.Open("myfile.txt");
        byte b;

        // ReadByte returns -1 at EOF
        while ((b =
         stream.ReadByte())  > = 0) {
            // do something
        }
    }
}
  • Do throw an InvalidOperationException if in an inappropriate state.

    The System.InvalidOperationException exception is supposed to be thrown if the property set or method call is not appropriate given the objects current state.

  • Do throw an ArgumentException or subclass there of if passed bad parameters.

    When bad parameters are detected, throw a System.ArgumentException or a subclass there of.

  • Do realize that the stack trace starts at the throw.

    The stack trace is from where the exception is thrown, not where it is newed. Be aware of this fact when deciding where to throw.

  • Recommendation using exception builder methods.

    It's common for a class to throw the same exception from different places in its implementation. To avoid code bloat, use helper methods that new the exception and return it. For example:

    class File {
    string fileName;

    public byte[] Read(int bytes) {
        if (!ReadFile(handle, bytes))
            throw NewFileIOException();
    }

    FileException NewFileIOException() {
        string description = // build localized string, including fileName
        return new FileException(description);
     }
}

    Another alternative is to use the constructor of the exception to build the exception. This is more appropriate for global exception classes like ArgumentException.

  • Do throw Exceptions in favor of returning an error code (or HResult)
  • Do throw the most specific exception possible
  • Do favor using existing exceptions over creating new ones
  • Do set all the fields on the exception you use
  • Do use Inner exceptions (chained exceptions).
  • Do set meaningful message text targeted at the developer in the exception.
  • Don’t have methods that throw NullReferenceException or IndexOutOfRangeException.
  • Do argument checking on protected (family) members. Clear state in the documentation if the protected method does not do argument checking. Unless otherwise stated, assume argument checking is done. There may be performance gains in not doing argument checking.
  • Do cleanup intermediate results when throwing an exception. Callers should be able assume that there are no side-effects when an exception is thrown from a function.

    Example: If Hashtable.Insert throws an exception, then the caller can assume that the item was not added to the Hashtable (the hashtable implementation breaks this rule, but that's another issue).


Standard Exception Types

The following table breaks down the standard exceptions provided by the runtime and the conditions they should be subclassed/thrown.

Exception Type | Base Type | Description | Example
Exception | Object | Base class for all Exceptions. | None (use a subclass of this exception).
SystemException | Excetion | Base class for all runtime generated errors. | None (use a subclass of this exception).
CoreException | SystemException | Base class for all fetal runtime errors | None (use a subclass of this exception).
IndexOutOfRangeException | CoreException | Thrown only by the runtime when an array is indexed improperly. | Indexing an array outside of it’s valid range: arr[arr.Length+1]
NullReferenceException | CoreException | Thrown only by the runtime a null object is referenced. | object o = null; o.ToString();
InvalidOperationException | SystemException | Thrown by methods when in an invalid state. | Calling Enumerator.GetNext() after removing an Item from the underlying collection.
ArgumentException | SystemException | Base class for all Argument Exceptions. Subclasses of this exception should be thrown where applicable. | None (use a subclass of this exception).
ArgumentNullException | ArgumentException | Thrown by methods which do not allow an argument to be null. | String s = null; ”foo”.IndexOf (s);
ArgumentOutOfRangeException | ArgumentException | Thrown by a methods which verify arguments are in a given range. | String s = “foo”; s.Chars[9];
InteropException | SystemException | Base class for exceptions that occur or are targeted at environments outside of the runtime. | None (use a subclass of this exception).
ComException | InteropException | Exception encapsulating Com classic Hresult information. | Used in Com interop.
SEHException | InteropException | Exception encapsulating Win32 structured Exception Handling information | Used in unmanaged code interop.


Wrapping Exceptions

Errors that occur at a layer below where your component sits should be wrapped in a meaningful exception to your target users and thrown.

  • Get a “real” example here
public class TextReader
{
public String ReadLine ()
{
try
{
         //read a line from the stream
} catch (Exception e)
{
         throw new IOException (“Could not read from stream”, e);
}
}
}


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Array Usage Guidelines

© 2000 Microsoft Corporation. All rights reserved.


Arrays vs. Collections

Class library designers are sometimes faced with a decision about when to use an arrays and when to return a collection. These solutions have very similar usage models; but somewhat different performance characteristics.

Collection

Do use a collection when Add, Remove or other methods for manipulating the collection are supported. This scopes all related methods to the collection.

Do use collections to add readonly wrappers around internal arrays.

Indexed Properties in Collections

Use only as the default member of a collection class or interface.

Do not create families of functions like this. Use a collection instead.

int FooCount { get; }
SomeType Foo[int index] { set; get; }
void ApppendFoo(SomeType foo)

Array Valued Properties

Recommendation using collections to avoid the inefficiencies in the following naive code:

for (int i = 0; i < obj.Foo.Count; i++)
      DoSomething(obj.Foo[i])


Returning Empty Arrays

Nulls should only be returned by “reference” properties (i.e. things that refer to another object or component). String and Array properties should never return null. The reason for this is that users don’t understand null in this context. They always assume the following code will work:

   public void DoSomething(…) {
      string s = SomeOtherFunc();
      if (s.Length > 0) {
         // do something else
      }
   }

The general rule is that null, empty string, and empty (0 item) arrays should effectively be treated the same.

Do return an Empty array instead of a null reference.


Operator Overloading Usage Guidelines

Do define operators on Value types that are logically a built in language type (such as System.Decimal)

Do provide operator-overloading methods only involving the class the methods are defined on.

Do use the names and signature conventions described in the CLS.

Don't be cute.

Operator overloading is useful in the cases where it is immediately obvious what the result of the operation would be. For example, it makes a lot of sense to be able to subtract one Time value from another and get a TimeSpan. However, it's not appropriate to use an or operator to union two database queries, or to use shift to write to a stream.

Do provide alternate signatures.

Not all languages support calling overloaded operators. For this reason, always include a secondary method with an appropriate domain-specific name that has equivalent functionality. For example:

class Time {
   TimeSpan operator -(Time t1, Time t2) { }
   TimeSpan Difference(Time t1, Time t2) { }
}


Casts

Do not lose precision in an Implicit casts.

For example, there should NOT be an implicit cast from Double to Int32, but there may be one from Int32 to Int64.

Don’t throw exceptions from implicit casts because it will be very difficult for the developer to understand what is happening.

Do provide casts that operate on the whole object. The casted value represents the whole value being casted, not one sub part. For example, it is not appropriate for for a Button to a cast to string by returning it's caption.

Do use casts to convert a value into another value.

Do not generate a semantically different value.

For example it is appropriate to convert a Time or TimeSpan into an int. The int still represents the time or duration. It does not make sense to convert a file name string, e.g. "c:\mybitmap.gif" into a Bitmap object.

Do cast values are in the same domain. Don’t cast values from different domains. Casts operate within a particular domain of values. For example, numbers and strings are different domains. So it makes sense that an Int32 can cast to a double. It does not make sense for an int to cast to a string, because they are in different domains.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Common Design Patterns

© 2000 Microsoft Corporation. All rights reserved.


Finalize and Dispose

Class instances often encapsulate control over resources not managed by the runtime (Hwnds, Database connections etc). There needs to be both an explicit and an implicit way to free those resources. The implicit control is provided by implementing the protected method Finalize() on Object. The GC calls this method when there are no longer any valid references to the object. The explicit control is provided by a Dispose() method. This method must be called by the consumer of the object, and can be done even if other references to the object are alive.

Finalize

  • Only implement Finalize on methods that need finalization. There is some cost associated with it
  • Do not make the Finalize method more visible. It should NOT be public.
  • Do free any external resources in the finalizer
  • If you only have managed references don’t implement the finalize method.

Dispose

  • Do use the name “Dispose” for this method unless a domain specific name is in common usage (“Close” for files).
  • Do provide a public dispose method for users to free external resources
  • Do free any external resources in the dispose method
  • Do not make the Finalize method more visible. It should NOT be public.
  • Do free any external resources in the finalizer
  • Do not assume that Dispose will be called. External resources should also be released in the finalizer.
  • Do propagate dispose through containment hierarchies.
  • Do consider the following design pattern:
    void Dispose()
{
     Finalize();
     GC.SuppressFinalize(this);
}


Immutable

  • It might be worth describing how/when class writers should utilize an immutable design pattern in the runtime,
  • and provide examples demonstrating where its used in the base class libraries. The concurrency implications
  • of an immutable pattern should also be highlighted.


Static Methods and Singleton Design

  • It might be worth describing the implications of static methods (they can't be overriden) as well as how
  • developers can utilize them with a private constructor to implement a singleton design pattern. Examples
  • of where you're using this today in the base class library (for example: reflection) could also be useful.


Callbacks

Delegates, Interfaces and Events can each be used to provide callback functionality. Each has it’s own specific usage characteristics that make it better suited to particular situations.

Do use Events if:

  • One signs up for the callback up front (typically through a separate Add and Remove methods
  • Typically more than one object will care

Do use a Delegate if:

  • You basically want a C style function pointer.
  • Single callback
  • Registered in the call or at construction time (not through separate Add method)

Do use an Interface if:

  • The callback entails complex behavior
  • We should have an example of each


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Collections

Do see the collections spec on http://comrtime/specs/classlibs/Doc/Collections.doc


Security in Class Libraries

Class library authors need to consider two perspectives with respect to security - which if any apply will depend on the class itself. Some classes - for example, Syste.IO.FileStream - that represent objects that need protection with permissions; the implementation of these classes is responsible for checking the appropriate permissions of the caller required for each action and only allowing duly authorized callers to perform actions they have permission for. The System.Security namespace contains some classes to help make these checks easier. Additionally, class library code often is fully-trusted or at least highly-trusted code and as such any flaws in the code represent a serious threat to the integrity of the entire security system, and a certain extra degree of care is required as detailed below.


Protecting objects with permissions

Permissions are defined to protect specific resources, and any class library that performs operations of such protected resources needs to be responsible to enforce this protection. Before acting on any request on a protected resource such as delete a file, the code need to first check that the caller (and usually all callers, by means of a stack walk) has the appropriate delete permission for that resource. If so the action is considered authorized and may be done, or if not a security exception should be raised. In code, protection is typically implemented with either a declarative or imperative check of the appropriate permissions.

For example, use of files is protected by the FileIOPermission which grants separately the ability to create, read, write, and delete selected named groups of files. When the File class (or possibly any other class that exposes the same functionality) method to delete files is called to delete a specific file, the caller stack walk check for delete permission on that file must pass before the action delete operation will happen. The code of the File class is responsible for performing the necessary check that binds together the semantics of the permission object to the potential actions taken by the protected object itself.

It is important that classes protect all possible kinds of exposure of resources, not necessarily only direct access. For example, a cached file object is responsible for checking file read permission even if the actual data is only retrieved from a cache in memory and no actual file operation happens since the effect of handing the data to the caller is the same as an actual read.


Fully-trust class library code

Many class libraries are implemented as fully trusted code that wraps platform specific functionality such as COM or system APIs as managed objects; being fully-trusted this code is absolutely a potential weakness to the security of the entire system and consequently deserves careful design and review. In a very real sense, placing a heavy security burden on a relatively small set of class libraries and the underlying core runtime security allows the larger body of managed code to acquire the security benefits in that managed code only represents a potential security threat in proportion to the level of trust granted to it.

A common class library security scenario is a fully trusted class that exposes a resource that is protected by a permission; the actual resource is accessed by a native code API. Again, the typical example would be a File class that uses Win32 file API to do some file operation such as delete. The following shows the steps taken to protect the resource.

  • Caller requests delete file "c:\test.txt" by calling appropriate method of the File class
  • The delete method creates a permission object representing "delete c:\test.txt" permission
  • Test using IPermission.Check if all callers on the stack have this permission; if not a security exception arises
  • Assert full-trust - the File class asserts its right to call native code (since its callers may not have this permission)
  • File class uses native API to perform the file delete operation
  • File class returns to its caller, completing the file delete request successfully


Precautions for highly trusted code

Any code in a trusted class library will be granted permissions not normally available to most application code, and hence particular care must be taken to avoid creating a weakness in security. Note that there may be classes that do not especially require special permissions that may be granted them simply by virtue of being in an assembly with code that does, in which case the same potential for trouble exists.

Any such trusted code needs to be designed so it can potentially be called by any kind of semi-trusted code on the system without exposing security holes. Protection of resources is normally safe by virtue of the stack walk - is the caller has insufficient permissions, an attempted access will be blocked - however, any time trusted code asserts permission it is taking responsibility and requires checking. Normally, the assert should only follow the necessary permission check of the caller as described above. The "window" of asserted high permission should be kept as small as possible to reduce risk of unintended exposure (but see note on performance below).

Fully trusted code is implicitly granted all other permissions, plus it is allowed to violate rules of type safety and object usage. Independent of protection of resources, any aspect of the programmatic interface that might break type safety or afford access of data not normally available to the caller could lead to a security problem.


Performance

Security checks involve checking the stack for permissions of all callers, so depending on the stack depth are potentially very expensive operations. If one operation actually consists of a number of actions at a lower level that require security checks, it may greatly improve performance to check caller permissions once and then assert the necessary permission before performing the actions. The assert will stop the stack walk from propagating further up the stack so the check will stop there and succeed; typically this results in performance improvement when three or more permission checks can be covered together.


Summary of security issues for class libraries

Do:

  • Any class library that uses protected resources must ensure that it only does so within the permissions of its callers.
  • Assertion of permission should only be done when strictly necessary, preceded by necessary checks.
  • Consider how a semi-trusted malicious caller might potentially use the class to get around security.
  • To improve performance aggregate operations that will involve security checks and consider use of assert to limit stack walks without compromising security

Don’t:

  • Assume that the code will only be called by callers with certain permissions.
  • Define non-type-safe interfaces that might be used to circumvent security elsewhere.
  • Expose functionality that allows a semi-trusted caller to take advantage of higher trust of the class.


Threading Design Guidelines

  • Avoid providing static methods that mutate static state

    In common server scenarios, static state is shared across requests which means multiple threads could be executing that code at the same time (which opens up this possibility for threading bugs). Consider using a design pattern that encapsulates data into instances that are not shared across requests.

  • Design for server environment

    Design your library for the stress of running in a server scenario. Avoid taking locks if at all possible.

  • Instance state does not need to be thread safe

    By default, the library is NOT thread safe. Adding locks to create thread safe code decrease perf and increase lock contention (as well as opening up dead lock bugs). In common application models, only on thread at a time is executing user code which makes the need for thread safety minimal. For this reason the frameworks are not thread safe by default. In cases where it's interesting to have a thread safe version a GetSynchronized() method can be used to return a thread safe instance of that type. (See System.Collections for examples).

  • Static state must be thread safe

    If you must use static state make it thread safe. In common server scenarios, static data is shared across requests which means multiple threads could be executing that code at the same time. For this reason it is necessary to protect static state.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Asynchronous execution

© 2000 Microsoft Corporation. All rights reserved.


Introduction to Asynchronous execution

Asynchronous programming is a feature supported by many areas such as Remoting, ASP+, and Win Forms. As it is a core concept, it is important for the NGWS frameworks and runtime to establish a common design pattern to avoid confusion among developers. This document proposes such a design pattern.

The basic ideas behind this proposal are as follows:

It is the client that decides if a particular call should be asynchronous.

It is not necessary for a server to do additional programming for supporting asynchronous behavior by its clients. Runtime infrastructure should be able to handle the difference between the client and server views. This avoids the IDispatch mishap where the server has to implement IDispatch and do lots of work to support dynamic invocation by clients.

The server can choose to explicitly support asynchronous behavior either because it can implement it more efficiently than a general architecture or it wants to support only asynchronous behavior by its clients. However, it is recommended that such servers follow the design pattern outlined in this document for exposing asynchronous operations.

Type safety.

The NGWS runtime will provide services needed for supporting asynchronous programming model. An example partial list of such services is:

  • Synchronization primitives such as critical sections and RWLocks.
  • Synchronization constructs such as containers that support WaitForMultipleObjects.
  • Thread pools.
  • Exposure to the underlying infrastructure pieces such as Message objects and Thread Pools.


Async Design Pattern

As an example, consider the following server class

class PrimeFactorizer
{
public boolean Factorize(unsigned long factorizableNum,
            ByRef unsigned long primefactor1,
            ByRef unsigned long primefactor2)
      {
         // Ensure that the given number is factorizable to prime numbers
         …..
         …..
         if(IsPrimeFactorizable == false)
         {
            primefactor1 = primefactor2 = 0;
            return(false);
         }

         // Factorize
         ……
         primefactor1 = …..;
         primefactor2 = …..;

         return(true);
}
}

The following code snippet shows client defining a pattern for invoking Factorize method asynchronously:

delegate FactorizingDelegate(unsigned long factorizableNum,
       ByRef unsigned long primefactor1,
       ByRef unsigned long primefactor2);
PrimeFactorizer pf = new PrimeFactorizer();
FactorizingDelegate fd = new FactorizingDelegate(pf.Factorize);

Compiler will define the following FactorizingDelegate class after parsing its definition at the first line above:

   class FactorizingDelegate extends delegate
   {
public boolean Invoke(unsigned long factorizableNum,
         ByRef unsigned long primefactor1,
         ByRef unsigned long primefactor2);
public IAsyncResult BeginInvoke(unsigned long factorizableNum,
                        ByRef unsigned long primefactor1,
                        ByRef unsigned long primefactor2,
               AsyncCallbackDelegate cb);
public boolean EndInvoke(ByRef unsigned long primefactor1,
              ByRef unsigned long primefactor2,
              IAsyncResult ar);
}

The following classes are defined inside classlibs:

delegate AsyncCallback (IAsyncResult ar);

public interface IAsyncResult
    {

        /**
         * True if the asynchronous operation has been completed.
         */
        bool IsCompleted { get; }

        /**
         * Handle to block on for the results
         */
        WaitHandle AsyncWaitHandle { get; }

        /**
         * The delegate object on which the async call was invoked.
         */
        Object     AsyncObject     { get; }

        /**
         * The state object passed in via BeginInvoke.
         */
        Object     AsyncState      { get; }

        /** true if the call completed synchronously
        */
        bool       CompletedSynchronously { get; }

Note that the object that implements IAsyncResult must be a waitable object and its underlying synchronization primitive should be signaled after the call is canceled or completed. This enables the client to wait for the call to complete instead polling. Runtime is supplying a number of waitable objects that mirror Win32 synchronization primitives. It also will be supplying the methods that support waiting for such synchronization objects to become signaled with “any” or “all” semantics. Such methods will be context aware to avoid deadlocks.

The Cancel method is a request to cancel processing of the method after the desired timeout period has expired. Note that it is only a request by the client and server is recommended to honor it. Further, client should not assume that the server has stopped processing the request completely after it got the notification that the method has been canceled. In other words, the client is recommended not destroy the resources such as file objects as the server may be actively using them. The IsCancelled property will be set to “true” if the call was canceled and IsCompleted property will be set to “true” after the server has completed processing of the call. It is illegal for the server to use any client supplied resources outside of the agreed upon sharing semantics after it sets the IsCompleted property to “true”. Thus, it is safe for the client to destroy the resources after IsCompleted property returns “true”.

The Server property returns the server object that provided the IAsyncResult.

The following code snippet demonstrates the client-side programming model for invoking the Factorize method asynchronously:

class FactorizeCallback
{
public FactorizeCallback(unsigned long number)
{
   _ulNumber = number;
}
// Note the qualifier one-way. See last paragraph of this section for
// its explanation.
public one-way boolean FactorizedResults(IAsyncResult ar)
   {
unsigned long factor1, factor2;
      FactorizingDelegate fd = ar.AsyncObject;
      fd.EndInvoke(factor1, factor2, ar);

// Output results
Console.Writeline(“Factors of” + _ulNumber + “:” + factor1 + “ ” +
   factor2);
   }

   private unsigned long _ulNumber
}

// Client code
delegate FactorizingDelegate(unsigned long factorizableNum,
       ByRef unsigned long primefactor1,
       ByRef unsigned long primefactor2);
PrimeFactorizer pf = new PrimeFactorizer();
FactorizingDelegate fd = new FactorizingDelegate(pf.Factorize);

// --------------------------------------------------------------------------------------------------------
// Async Variation 1
unsigned long factorizableNum = 1000589023, temp;

// Define the asynccallback delegate
FactorizeCallback fc = new FactorizeCallback(factorizableNum);
AsyncCallbackDelegate cb = new AsyncCallbackDelegate(fc.FactorizedResults);

// Asynchronously invoke the Factorize method on pf
// Note: If we have pure out parameters, we do not need temp
fd.BeginInvoke(factorizableNum, temp, temp, cb);

// Proceed to do other work
……

// --------------------------------------------------------------------------------------------------------
// Async Variation 2
// Asynchronously invoke the Factorize method on pf
// Note: If we have pure out parameters, we do not need temp
unsigned long factorizableNum = 1000589023, temp, factor1, factor2;
IAsyncResult ar = fd.BeginInvoke(factorizableNum, temp, temp, NULL);

// Do other work
……

// Wait for the callback
SynchronizationServices.WaitForSingleObject(ar);

// Output results
fd.EndInvoke(factor1, factor2, ar);
Console.Writeline(“Factors of” + factorizableNum + “:” + factor1 + “ ” + factor2);

Note that if the FactorizeCallback is a context-bound class that requires synchronized/thread- affinity context, the callback is dispatched through the context dispatcher infrastructure. In other words, the callback itself may execute asynchronously with respect to its caller for such contexts. That is precisely the semantics of the one-way qualifier on method signatures. It means that any such method call may execute synchronously or asynchronously with respect to caller and the caller cannot make any assumptions about completion of such a call when execution control returns to it.

Also, calling EndInvoke before the asynchronous operation is complete will block the caller. Calling it second time with the same AsyncResult is undefined.


Summary of Async Design Pattern

The server splits asynchronous operation into its two logical parts: the part that takes input from the client and kicks off the asynchronous operation and the part that supplies results of the asynchronous operation to the client. In addition to the input needed for the async operation, the first part also takes an AsyncCallbackDelegate object to be called when the asynchronous operation is completed. The first part returns a waitable object that implements IAsyncResult interface used by the client to determine the status of the asynchronous operation. Server typically also utilizes the waitable object it returned to the client to maintain any state associated with asynchronous operation. The client uses the second part to obtain the results of the asynchronous operation by supplying the waitable object.

The options available to client for initiating asynchronous operations are:

  1. Supply the callback delegate when beginning asynchronous
  2. Not supply the callback delegate when beginning asynchronous operation.

The options available to the client for completing asynchronous operations are:

  1. Poll the returned IAsyncResult object for completion.
  2. Attempt to complete the operation prematurely thereby blocking until the operation completes.
  3. Wait on the IAsyncResult object. The difference between this and previous option is that the client can use timeouts to wake up periodically.
  4. Complete the operation inside the callback routine.

File IO is an example where both synchronous and asynchronous read/writes to file objects is desirable. An example of how the File object inside classlibs implements read-write operation will server as a good illustration of the design pattern.

class File
{
// Other methods
………

// Synchronous read
unsigned long Read(Byte[] buffer, unsigned long NumToRead);

// Asynchrnous read
IAsyncResult BeginRead(Byte[] buffer,
unsigned long NumToRead,
AsyncCallbackDelegate cb);
unsigned long EndRead(IAsyncResult ar);

// Synchronous read
unsigned long Write(Byte[] buffer, unsigned long NumToWrite);

// Asynchrnous read
IAsyncResult BeginWrite(Byte[] buffer,
unsigned long NumToWrite,
AsyncCallbackDelegate cb);
unsigned long EndWrite(IAsyncResult ar);

}

The client cannot easily associate state with a given asynchronous operation without defining a new callback delegate for each operation. This can be fixed by making the begin method take an extra object parameter that represents the state and which is captured into the IAsyncResult.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Items that maybe covered in the future

© 2000 Microsoft Corporation. All rights reserved.


Exposing Platform Specific

Some managed libraries want to take advantage of the cross-platform nature of the runtime. These classes should not directly expose platform specific constructors in their APIs.


Synchronization and other Concurrency Patterns

  • Add concurrency patterns that class designers should use in the Managed world. Specific explanations of class, method and inter-method synchronization could be useful, along with a description of the Interlocked class.
  • It would also be useful to highlight some of the decisions the base class libraries have made WRT synchronization (for example: for performance collections do not provide built-in synchronization symantics), and how developers can help communicate these design choices to customers.


Remoting and Serialization

  • How do class designers plan for remoting (what the rules/implications). How/when should developers
  • provide serialization capabilities on their classes?


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Internationalization

  • How do developers design for localization or external resource usage?


Deployment

  • What are the recommended patterns for packaging up class frameworks into assemblies. WinForms seems to
  • have adopted a strategy whereby all namespace units (System..Data,, etc) are
  • packaged into separate DLLs. Should other class developers follow this?


Component Model

  • What is it
  • How and when should these classes be used
  • How does this aid in tools support


Callback mechanisms

  • Events vs. callback delegates vs. callback interfaces
  • We provide several "callback" mechanisms:
  • events (example: button)
  • delegate (example: refelection member filters)
  • interface (example: IComparer)
  • What are the guidelines for selecting between these mechanisms?


Remoting vs. messages

  • What are the guidelines for using explicit message transfer instead of a remote method call?


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Enabling Profiling and Debugging

© 2000 Microsoft Corporation. All rights reserved.


Enabling Profiling

A code profiler is a software tool, implemented as a DLL, that is called by the Runtime, during execution of an image – the profiler receives notifications of ‘interesting’ events such as each time a function is entered or exited. By gathering statistics on these events, the profiler tool can build a comprehensive picture of which routines burned most cpu time, when garbage collections occured, exceptions thrown, etc, etc.

‘Connecting’ to a profiler is only done during the initialization phase for the Runtime within each process. In this phase, it checks two environment variables to determine:

  • Whether to ‘connect’ with a code profiler (Cor_Enable_Profiling)
  • Which code profiler to send any notifications to (Cor_Profiler)

If Cor_Enable_Profiling does not exist, or is set to the value 0, the Runtime will make no attempts to ‘connect’ to a profiler.

If Cor_Enable_Profiling has the value 1, the Runtime will attempt to ‘connect’ to the profiler whose CLSID is specified in the Cor_Profiler string. For example:

set Cor_Profiler={32E2F4DA-1BEA-47ea-88F9-C5DAF691C94A}

Then the Runtime will ‘connect’ with the DLL, by instantiating the COM class with that CLSID – the class must implement the ICorProfilerCallback interface (used to receive notifications from the Runtime, during execution, of ‘interesting’ events, such as function enters and leaves)


Enabling JIT-attach Debugging

JIT-attach debugging is the phrase used to describe attaching a debugger to an image which throws an uncaught exception. In unmanaged code, it’s what happens when you are prompted with a message box that invites you to:

  • Click on OK to terminate the program
  • Click on CANCEL to debug the program

If you click CANCEL, a debugger is started and attached to the process. Control for this is made via the registry key:

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\Current Version\AeDebug

For any App that includes managed code, the NGWS runtime will offer a similar feature to JIT-attach a debugger. The registry key that controls these options is called:

HKEY_LOCAL_MACHINE\Software\Microsoft\ComPlus\DbgJITDebugLaunchSetting

Alternatively, you can set the environment variable CORDBG_DbgJITDebugLaunchSetting to achieve the same effect.

The Runtime inspects its value, as follows:

  • If value = 0, prompt the user via a message box for what to do. Choices are:
    • Continue, which will result in a stack dump and process termination, as before
    • Attach a debugger. In this case, the Runtime spawns the debugger listed in the DbgManagedDebugger registry key. If none, we return control, and terminate the process, as above
  • If value = 1, simply return control. This will result in a stack dump; the process will then terminate
  • If value = 2, we skip checking with the user. We go straight to spawning the debugger listed in the DbgManagedDebugger registry key

The name of the debugger to attach is stored in the registry key:

HKEY_LOCAL_MACHINE\Software\Microsoft\ComPlus\DbgManagedDebugger

Set this key to hold the command-line launch for the required debugger. For example, to use the CorDbg sample debugger that ship with the SDK, set the DbgManagedDebugger key with a value like this:

c:\Program Files\Com20SDK\Bin\CorDbg.exe !a 0x%x or !a %d

The argument will hold the ID of the process to be debugged. It will be filled in automatically by the Runtime dispatch code.


Making an Image Easier to Debug

For unmanaged code, you control how easy it is to debug the resulting image via IDE switches, or command-line options. For example, the /Zi command-line option for VC++ asks it to emit debug symbol files (PDBs). Similarly, the /Od command-line option tells the compiler to disable optimization – the resulting code runs more slowly, but it’s easier to debug, should the need arise.

In the NGWS runtime, compilers such as VC++, VB and C# compile their source program into MSIL (Microsoft Intermediate Language). This MSIL is subsequently JIT-compiled, before execution, into native machine code. In the same way as you can trade off performance for ‘debuggability’ with conventional source-to-MSIL compilers, you have that same choice for the JIT-compilers.

There are two aspects to this control:

  • Request the JIT-compiler to generate ‘tracking’ info. This allows a debugger to match up chains of MSIL with their machine code counterpart; and to track where local variables and function arguments are stored
  • Request the JIT-compiler to prevent itself from optimizing

Normally, your ‘first level’ compiler will set these options automatically, based upon the IDE switches or command-line options you specify (eg /Od)

But, as a programmer, you may wish to change the behavior of the JIT-compiler so that the machine code it generates is easier to debug. You do so with a little text file. If the assembly you wish to debug is called MyApp.Exe, then create a text file called MyApp.Ini, in the same directory as MyApp.Exe, like this:

[NGWS Runtime Debugging Control]
GenerateTrackingInfo=1
AllowOptimize=0
Ambivalent=1

You may set the value of each option to 0 or 1, and any option may be absent (their value defaults to 0). Setting GenerateTrackingInfo to 1 and AllowOptimize to 0 provides best debuggability. If the Ambivalent option is present, then the assembly’s modules won’t generate tracking info, and will enable optimizations. If all the assemblies that are loaded into an appdomain have this set, then all the modules will exhibit this behavior. If even a single assembly is marked as Ambivalent=0, then all ambivalent assemblies will follow the behavior specified in their respective .INI files.

[Note: settings for an assembly are carried via the System.Diagnostics.DebuggableAttribute. It has two fields, called m_fEnableJITcompileTracking and m_fDisableJITcompileOptimizer. For a retail build, compilers don’t attach any DebuggableAttribute – this defaults the JIT-compiler to behave as if both fields were false – highest performance, hardest to debug. Enabling tracking lowers performance a little; disabling optimization lowers performance a lot]

The DebuggableAttribute applies to a whole assembly at a time (not to individual modules within the assembly). Tools must therefore attach that Custom Attribute to the Assembly metadata token, if an assembly has already been created, or to the class called System.CompilerServices.AssemblyAttributesGoHere – the ALink tool will then ‘promote’ these DebuggableAttribute attributes from each module to the assembly they become a part of. If there is any conflict, the ALink operation will fail.


NGWS Runtime Hosting Interfaces

This specification describes the classes and interfaces provided by the runtime that hosting environments like IE and ASP+ use to launch the runtime, create Application Domains and execute code in assemblies.


Overview of Runtime Hosting Interfaces

This specification describes the classes and interfaces provided by the NGWS runtime that hosting environments use to launch the runtime, create Application Domains and execute code in assemblies. Examples of hosting environments include IE running web pages that contain managed code and/or script, ASP+ running web applications, and VBA running Office macros.

In each of these cases, the hosting environments need to run managed code on behalf of the user and take advantage of the application isolation features provided by the runtime. For details on the runtime's support for application isolation see: http://comrtime/Specs/admin/AppDomains.doc.

The NGWS runtime provides a set of unmanaged interfaces we call the hosting interfaces that hosts use to configure and start the Runtime. Once the Runtime has been started and the initial app domain(s) has been created, much of the interaction between the host and the managed code in the process is through COM Interop to instances of classes like System.AppDomain or System.Reflection.Assembly.

The primary interfaces described in this document are ICorRuntimeHost and ICorConfiguration. These interfaces serve two primary purposes: they allow an unmanaged host to start and configure the runtime, and they simplify access to the managed classes described above from unmanaged code.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Requirements of Runtime Hosting Interfaces

© 2000 Microsoft Corporation. All rights reserved.


Application Isolation Features for Hosting Environments

Hosting environments must be able to programmatically create an App Domain in which to run managed code. In addition, hosting environments should be able to run code in an existing App Domain.


Run Code in an App Domain

When a domain is created, the hosting environment must be able specify the initial code to run and be able to get back a reference to that code. The initial code must be either a type, a global function, or the assembly's entry point.


Security Requirements

App Domains can be created from either fully-trusted or semi-trusted code. Fully trusted hosts must be able to supply evidence when creating a domain. This evidence is mapped through security policy to establish a "top of stack" permission set for all code running in the domain. Partially trusted hosts cannot present evidence, but they can explicitly specify the "top of stack" permission set. This permission set must be equal to, or a subset of, the permissions of the host.

Hosts with the appropriate level of trust must also be able to establish security policy to be applied to all code running in the domain. For example, a host may use the domain-specific policy to only load code that comes from a particular location. Domain-specific policy is ”merged with” the per-machine and per-user security policy at Runtime.


Unloading of applications

Many hosting environments have requirements to unload running code. For example, Web Servers needs to unload user code when that code is replaced with a newer version on disk and the WFC Forms designer needs to unload and reload controls that are actively being developed in the editor. The runtime must support two forms of unloading. A host must be able to immediately unload a domain regardless of its current state. Additionally, a host must be able to tell the runtime to unload a domain when there are no more active threads in the domain.


Performance Requirements

The transition between unmanaged hosting environment and the managed assembly must not be greater than the transitions before these features were added. (is this realistic??).

The cost of tearing down an App Domain and creating a new one cannot be prohibitive. We should also have the ability to "reset" a domain so it can be pooled and reused.


"Naming" an App Domain

The Visual Studio debugger team would like to be able to associate a human readable string with a domain so they can display lists of domains in a user interface.


Pooling of Domains

The runtime must provide the infrastructure needed for hosts to pool domains for subsequent reuse.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Scenarios for Hosting Interfaces

© 2000 Microsoft Corporation. All rights reserved.


IE Hosting Web Pages

IE often must display pages that contain managed script and/or code. In these cases, IE must be able to create an App Domain in which to run the code and unload the code when the page is removed from the IE cache (or at whatever times they deem appropriate).


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

ASP+ Hosting Web Server Applications

ASP+ uses App Domains to isolate applications on a web server.


Creating Controls in the WFC Forms Designer

The WFC Forms Designer supports creating controls to be contained in forms, web pages, etc .. As changes are made to a control in the editor, the Designer must unload the current version of the control, rebuild it, and reload the newly compiled version.


Running VBA Macros from Office Applications

Given the VBA code in an Office macro, the VBA environment must create an in-memory assembly, load it in an App Domain, run it and shut it down.


Running Regular Expressions

An application does a large search using a regular expression supplied by the user (e.g., through UI or a configuration file). Because the search is expected to be very large, the application tells the System.Regex object to use full optimization, including compilation to IL. When the search is done, the regular expression might not be used again.


SQL Server

SQL Server hosts to Runtime to allow customers to write stored procedures in NGWS runtime languages and to run IL that is emitted when processing various expressions.


Hosting the Runtime

Unmanaged hosts use ICorRuntimeHost as the starting point for hosting the runtime. The initial pointer to this interface is obtained either by calling CoCreateInstance or by calling through the version-aware shim. To start a specific version of the Runtime, the host must go through the shim.

When calling CoCreateInstance, use a clsid of CLSID_CorRuntimeHost and an IID of IID_ICorRuntimeHost.

When calling through the shim, call CorBindToRuntime with an IID or IID_ICorRuntimeHost. See http://comrtime/specs/sbs/Startup%20Shim.doc for details.

ICorRuntimeHost is then used to configure and start the runtime and to create AppDomains in which to run code.

Managed hosts can create AppDomains directly.

The following picture shows the relationship between Win32 processes, AppDomains, Assemblies, and ICorRuntimeHost.

The process boundary in the picture is somewhat arbitrary. The unmanaged host could be in the same process with the managed hosting objects. The process boundary was included to emphasize the fact that a single process may contain multiple AppDomains


ICorRuntimeHost

ICorRuntimeHost provides the following capabilities:

  • Configure the Runtime
Get and set global events and configure parameters such as GC heap allocation. Configuration is done through the ICorConfiguration interface.
  • Create Domains
Create Application Domains in which to run code. The methods on ICorRuntimeHost that are used to create domains return interface pointers to instances of System.AppDomain. Hosts call methods on this interface through COM interop to load assemblies into the domain, etc …
  • Enumeration of Domains in a Process
  • Get the default App Domain for the process

Any configuration of the runtime must be done before it is started. Subsequent configuration requests will be ignored and an error returned.

ICorRuntimeHost and associated interfaces are defined in mscoree.idl:

typedef void* HCORENUM;

interface ICorRuntimeHost : IUnknown
{

    HRESULT GetConfiguration([out] ICorConfiguration** pConfiguration);

    HRESULT Start();

    HRESULT Stop();

    HRESULT CreateDomain([in] LPWSTR pwzFriendlyName,   // Optional
                         [in] IUnknown* pIdentityArray, // Optional
                         [in] IUnknown* pPolicyArray,   // Optional
                         [in] LPWSTR pwzAppRelativeSearchPath,     // Optional
                            [in] LPWSTR pwzGlobalSearchPath, // Optional
                         [in] bool   bShadowCopyFiles,    // Optional
                         [in] long   snapshotID,          // Optional
                         [out] void ** pAppDomain);

    HRESULT GetDefaultDomain([out] IUnknown** pAppDomain);

    HRESULT EnumDomains([out] HCORENUM *hEnum);

    HRESULT NextDomain([in] HCORENUM hEnum,
                       [out] void** pAppDomain);

    HRESULT ResetDomainEnum([in] HCORENUM hEnum);

    // These routines are being depricated
    HRESULT CreateFiberInfo();
    HRESULT DeleteFiber();
    HRESULT SwitchFiberIn(
[in] DWORD pFiberCookie);
// [in] Cookie that indicates the fiber to use.

    HRESULT SwitchFiberOut(
[out]   DWORD *pFiberCookie);
// [out] Cookie that indicates the fiber being switched out.

    HRESULT MapFile(
[in]   ULONG       hFile,           // [in]  HANDLE for file
[out]  HMODULE*    hMapAddress);
// [out] HINSTANCE for mapped file

    HRESULT GetCompileInfo(
[out] ICorCompileInfo  **ppCompileInfo);

}

GetConfiguration

Returns an interface (ICorConfiguration) that can be used to configure the runtime. All configuration must be done before calling ICorRuntimeHost::Start. ICorConfiguration currently has just one method. This interface will be expanded with additional configuration methods as needed.

Parameters:

Name | Description
pConfiguration | [out] A pointer to an ICorConfiguration interface

Start

Starts the runtime. This is equivalent to CoInitializeCor();

Stop

Terminates the runtime. This is equivalent to CoUninitializeCor();

CreateDomain

Creates an AppDomain. The caller receives an interface pointer to an AppDomain object that can be used to further control the domain.

Parameters:

Name | Description
pwzFriendlyName | [in] An optional parameter used to give a friendly name to the domain. This friendly name can be displayed in user interfaces to identify the domain (like in the debugger).
pIdentityArray | [in] An optional array of pointers to IIdentity objects that represent evidence that gets mapped through security policy to establish a "top of stack" permission set.
pPolicyArray | [in] An optional array of context policies to be added to the basic policies of the domain. Note: This is not implemented in the prerelease.
pwzAppRelativeSearchPath | [in] A semi-colon separated list of directories to search for privately referenced assemblies. These directories must be contained within the application directory. If not supplied, the default search path is the current app directory.
pwzGlobalSearchPath | [in] A semi-colon separated list of directories to search for global, strongly named assemblies.
bShadowCopyFiles | [in] This flag tells the runtime to make a shadow copy of a module, and load the copy instead of the original. The original is left unlocked and can be overwritten. The default is false.
SnapshotID | [in] This property relates to the one above. When we make a copy of a file to load, we associate a snapshotID with it. If the next request to copy the file contains the same snapshotID, we don't bother copying it again. The default is 0xffff. This default tells us to "always copy".
pAppDomain | [out] A interface pointer to an instance of System.AppDomain that can be used to further control the domain. See the class description for AppDomain below.

GetDefaultDomain

Returns an interface pointer to the default domain for the process.

Parameters:

Name | Description
pAppDomain | [out] A pointer to an instance of System.AppDomain representing the default domain for this process. See System.AppDomain.

EnumDomains

Returns an enumerator that a host can use to enumerate the domains in the process.

Parameters:

Name | Description
hEnum | [out] The enumerator

NextDomain

Returns the next domain in the enumeration. This method returns S_FALSE when there are no more domains in the enumeration.

Parameters:

Name | Description
hEnum | [in] The enumerator
PAppDomain | [out] An interface pointer to the instance of System.AppDomain representing the next domain in the enumerator.

ResetDomainEnum

Resets the enumerator back to the beginning of the list.

Parameters:

Name | Description
hEnum | [in] The enumerator to reset.


ICorConfiguration

SetAppDomainLoadEvent

This method is used to register a callback that will be invoked whenever a new domain is created.

Parameters:

Name | Description
pAppStartEvent | [in]A callback that is invoked whenever a domain is created.

SetGCThreadControl

Sets a callback for scheduling threads for non-runtime tasks when they would otherwise be blocked for a GC. This callback will be called whenever the Runtime is about to run GC on a particular thread.

Name | Description
pGCThreadControl | [in]The callback for scheduling threads.

SetDebuggerThreadControl

Sets a callback that to receive notifications when threads are blocking and un-blocking during debugging.

Name | Description
pDebuggerThreadControl | [in]The callback for getting notified that a thread is about to block within the debugging services.

AddDebuggerSpecialThread

The host may indicate to the debugging services that a particular thread should be allowed to continue to execute while the debugger has an application stopped during managed/unmanaged debugging scenarios. The specified thread will not be allowed to run managed code, or to enter the Runtime in any way. An example of such a thread would be an in-process thread to support legacy script debuggers.

Name | Description
dwSpecialThreadId | [in]The id of the thread you wish to keep running.


Verification

Hosts may invoke the verification through the IValidator interface as described in http://comrtime/specs/security/Verification.doc


Security

A complete description of how hosts can affect the security policy of code loaded into domains can be found in http://comrtime/specs/security/hostload.doc


Managed Classes

After the Runtime has been started and the initial domain(s) has been created, most of the interaction between the host and the Runtime is through COM Interop to a set of managed classes defined in the mscorlib assembly. Most typically the host interacts with System.AppDomain and System.Reflection.Assembly and various classes in System.Security. To access these classes through Interop, run tlbexp on mscorlib.dll and import the resulting typelib in your source file.

See http://urtsdk/docs for a complete description of these classes.


How the NGWS Runtime Locates Assemblies

This document describes how the NGWS runtime locates an assembly given an assembly reference. Topics including probing, version policy and partial references are discussed.


Overview of Locating Assemblies

This document describes how the NGWS runtime locates assemblies when they are referenced at runtime. The process of finding and loading an assembly begins with a reference to that assembly. We call this an AssemblyRef. An AssemblyRef contains the text name, version, culture and an optional Originatorthat identify the assembly.

Given an AssemblyRef, the process of binding to that assembly is influenced by:

  • The ApplicationBase (or AppBase) – the root directory in which the application making the reference is located. For executables, this is the directory containing the exe. For web apps, the AppBase is the root directory of the application as defined by the web server.
  • Whether the reference is to an assembly with a shared name vs. a private name. (See the Assembly Overview for the distinction between shared and private assemblies).
  • Version policies specified in the application configuration file, including safe mode.
  • Any codebase locations provided in the application configuration file.
  • Any QFE updates to the assembly that are present in the application directory of in the global assembly cache.
  • The administrator’s version policies.

The following picture shows the steps taken to locate an assembly. The details of each step are described in the following sections.


Step 1: Initiating the Bind

The process of locating an assembly begins when the NGWS runtime class loader is asked to resolve a reference to another assembly. This reference may be static or dynamic. Static references are recorded in metadata at build time by the compiler. Specifically, a Typeref with a resolution scope that is an AssemblyRef is a static reference to an assembly. Dynamic references are constructed “on the fly” as a result of calling various API’s such as System.Reflection.Assembly.Load. For example:

Assembly a = Assembly.Load("myDll, Ver=5.5.5.5,
                            PK=92735c03372ac342");

In either case, the reference contains the name, version, culture and originator of the desired assembly as describe above. The loader passes this information to the AssemblyResolver, which locates the assembly following the steps described in the following sections. There is no difference in the resolution process based on whether the AssemblyRef was static or dynamic.


Step 2: Version Policy in the Application Configuration

The first step in binding to an assembly is to retrieve the application configuration file, if one exists. For client executables, the configuration file resides in the same directory as the executable and has the name of the exe (minus the .exe extension) + .cfg. So the configuration file for c:\program files\myapp\myapp.exe would be c:\program files\myapp\myapp.cfg. In the browser scenario, the configuration file must explicitly be pointed to in the html file (see Code Download Using IE5 for details). Specifying version policy at the application level currently cannot be done in ASP+ scenarios.

The <BindingPolicy> tag in the application configuration file can be used to redirect the reference to a particular version, or all versions, of an assembly to a different version (a complete description of the schema for BindingPolicy is available in Versioning). These version polices only apply when the reference is to an assembly with a shared name. References to private assemblies cannot have such policies applied. In fact, version checking is not done at all for binds to private assemblies.

The following entry from an application configuration file redirects references to all versions of an assembly called calcr to version 2.1.0.0:

   <BindingPolicy>
      <BindingRedir Name="Calcr"
                    Originator="32ab4ba45e0a69a1"
                    Version="*" VersionNew="2.1.0.0"
                    UseLatestBuildRevision="yes"/>
   </BindingPolicy>

If a version redirect for the requested assembly is encountered, the AssemblyResolver proceeds as if the original bind had requested the version specified in the redirect. In the example above, the bind will continue as if the original bind request was for version 2.1.0.0 of calcr.

The default behavior of the AssemblyResolver is to match versions based on major and minor number, and take the latest build and revision numbers (see Versioning for the semantics of each portion of the version number). The policy of taking the latest build and revision is often referred to as Automatic QFE Policy. This QFE policy allows software vendors to release critical bug fixes without requiring dependent applications to be rebuilt or reconfigured. It is worth noting that this policy is implicit in that there is nothing that NGWS runtime does to enforce that a QFE is in fact backward compatible with the version it is replacing. Also, we never implicitly take the latest major and minor numbers because they are, by definition, incompatible with what is being requested.

The automatic QFE policy can be overridden in one of two ways using the application configuration file. First, QFE policy can be disabled for references to particular assemblies using the “UseLatestBuildRevision” attribute of BindingPolicy. For example, the following entry turns off QFE policies for all references to an assembly named Parser:

   <BindingPolicy>
      <BindingRedir Name="Parser"
                    Originator="32ab4ba45e0a69a1"
                    Version="*" VersionNew="12.3.0.0"
                    UseLatestBuildRevision="no"/>
   </BindingPolicy>

Second, the application configuration file can be used to specify that all references to assemblies made within the application should bind to the exact versions that were present at build time, as recorded in the manifest. This is called Safe Mode. When running an app in Safe Mode, automatic QFE policy is never applied. Safe Mode is turned on using the <BindingMode> tag (see Versioning for detailed schema).

   <BindingMode>
      <AppBindingMode Mode="safe"/>
   </BindingMode>


Step 3: Locating the Assembly through Codebases or Probing

After the desired version has been determined by combining the information in the reference with that in the application configuration file, the AssemblyResolver attempts to find the assembly either in the AppBase (called probing) or at a codebase specified in the configuration file or through the current binding context.


Locating the Assembly through Codebases

The application configuration file can be used to point directly to the file containing a particular assembly’s manifest. This prevents the AssemblyResolver from probing for it under the AppBase. For example, the following entry tells the AssemblyResolver that version 1.0.0.1 of calcr can be found at http://internaltools/utils/caclr.dll :

   <Assemblies>
      <CodeBaseHint Name="Calcr"
                    Originator="8e47bf1a5ed0ec84"
                    Version="1.0.0.1"
                    CodeBase="http://internaltools/utils/caclr.dll"/>
   </Assemblies>

When the file at the given codebase is loaded, the AssemblyResolver checks to be sure that the name, version, culture and Originator match what is in the reference. Of course, by default the revision and build numbers can be higher that what is requested. Regardless of whether the given assembly matches the bind criteria, Step 3 is done at this point. If no match is found, the bind fails – no further steps are taken. If we do have a match, we proceed to Step 4.

In some scenarios, codebase information may also be available in the current binding context. This most often applies in the browser download scenario. Given an assembly that references another assembly, the first assembly’s codebase is considered a “hint” as to where to find the referenced assembly. For example, if Asm1 references Asm2 and Asm1 were downloaded from http://mystuff/utils then http://mystuff/utils will be considered a hint as to where to find Asm2.dll. If Asm2 is not found at that location, the AssemblyResolver will continue by probing for the assembly as described below.


Locating the Assembly through Probing

In the absence of a codebase describing the location of the assembly, the AssemblyResolver probes for the assembly under the AppBase. The term probing refers to the set of heuristics used to locate the assembly based on its name and culture.

Note that the version of the assembly probed for may differ from the original reference if rules in the application configuration file redirected the original bind to a different version.

By default, the assemblies that are actually deployed in the application directory need not match the bind reference exactly. It is possible for an application author to deploy new assemblies in the application directory that will still satisfy the bind, provided the newer assemblies only contain updates in the revision and/or build number of the assembly.

The AssemblyResolver always probes for the name of the assembly concatenated with three extensions (listed in order of priority):

  1. .MCL
  2. .DLL
  3. .EXE

The AssemblyResolver will always begin probing in the AppBase directory. That is, the first probing URLs generated will always the concatenation of the AppBase, the assembly name, and extension. For example, if the AppBase is c:\program files\myapp and the assembly name is FOO, then the first probing URLs generated will be:

c:\program files\myapp\foo.mcl,
c:\program files\myapp\foo.mcl, and
c:\program files\myapp\foo.exe

If the assembly is not found in the AppBase, probing continues in the directories on the assembly search path. The search path is specified using the <AppDomain> tag in the application configuration file:

<AppDomain PrivatePath="utils"/>

The directories given on the search path are relative to the AppBase and must be subdirectories of the AppBase. You cannot use this path to bind to assemblies outside the AppBase. Assemblies outside the AppBase must have shared names and must either be installed in the global assembly cache or pointed to using a codebase as described above.

In addition to the subdirectories given in the configuration file, the NGWS runtime always prepends a directory corresponding to the name of the assembly being probed for. For example, given the path specified above, if the NGWS runtime were probing for “caclr” it would modify the search path to be:

calcr;bin

The AssemblyResolver also considers the culture of the assembly when generating probing URLs. Specifically, the AssemblyResolver will look in culture-specific subdirectories under each directory on the search path. The name of the this subdirectory is the culture name as specified by RFC1766.

Probing is best understood by looking at all the URLs generated for a specific example:

NAME=MyName

APPBASE=http://www.wingtiptoys.com

SEARCH PATH=bin (The AssemblyResolver changes this to: MyName;bin)

LOCALE=de

Explicit Codebases:

Binding context CODEBASE=http://www.tailspintoys.com/codebase.exe

App.cfg CODEBASE=[none]

Probing URLs:

  1. http://www.tailspintoysr.com/codebase.exe
  1. http://www.wingtiptoys.com/MyName.MCL
  2. http://www.wintiptoys.com/MyName.DLL
  3. http://www.wingtiptoys.com/MyName.EXE
  1. http://www.wingtiptoys.com/MyName/MyName.MCL
  2. http://www.wingtiptoys.com/MyName/de/MyName.MCL
  3. http://www.wingtiptoys.com/bin/MyName.MCL
  4. http://www.wingtiptoys.com/bin/de/MyName.MCL
  1. http://www.wingtiptoys.com/MyName/MyName.DLL
  2. http://www.wingtiptoys.com/MyName/de/MyName.DLL
  3. http://www.wingtiptoys.com/bin/MyName.DLL
  4. http://www.wingtiptoys.com/bin/de/MyName.DLL
  1. http://www.wingtiptoys.com/MyName/MyName.EXE
  2. http://www.wingtiptoys.com/MyName/de/MyName.EXE
  3. http://www.wingtiptoys.com/bin/MyName.EXE
  4. http://www.wingtiptoys.com/bin/de/MyName.EXE

If the reference did not contain an Originator (i.e was to a private assembly), the binding process stops at this point. If a match was found, that assembly is returned to the loader. If no match was found, an error is raised. This error ends up as a TypeLoadException in managed code.

If the reference was to an assembly with a shared name, the binding process continues by looking in the global assembly cache.


Step 4: The Global Assembly Cache and Auto-QFE Policy

The global assembly cache stores assemblies that are meant to be used by several applications on the machine. All assemblies in the global cache must have shared names.

If an assembly was found via probing or through a codebase, and QFE policy has not been turned off, the AssemblyResolver looks in the global assembly cache to see if any QFE’s exist for that assembly. If there is a version of the assembly in the cache that has a higher build and/or revision number, this assembly will be returned instead of the one found by probing.

If no assembly was bound via probing or through a codebase, the AssemblyResolver will look in the global cache for an assembly that matches the criteria given in the AssemblyRef. If automatic QFE policy has not been turned off, the AssemblyResolver will take the match that has the higher build/revision number.

This policy of always looking for latest revision/build number in the global assembly cache, allows an administrator to put certain shared assemblies in a place that will affect all assembly bindings on the system. The most common scenario for this would be to apply a security fix to a well-known shared assembly.


Step 5: Administrator Policy

The final step in the binding process is to apply any version policy specified by the administrator. Admin policy is specified in a file called admin.cfg located in the windows directory. The syntax for specifying admin version policy is that same as that used in the application configuration file. The following example redirects all references to “calcr” made by any application on the machine to version 5.0.11.0:

   <BindingPolicy>
      <BindingRedir Name="Calcr"
                    Originator="32ab4ba45e0a69a1"
                    Version="*" VersionNew="5.0.11.0"
                    UseLatestBuildRevision="yes"/>
   </BindingPolicy>

Administrator policy is the strongest form of policy. The version determined by the administrator policy file is final, and cannot be overridden. It is required that if a policy redirect occurs as a result of administrator policy, the target version MUST pre-exist in the global assembly cache. That is to say that the AssemblyResolver will not re-issue a probe for the post-administrator-policy assembly, but rather it is assumed that such version redirect is a result of an explicit administrator action, which involved pre-installing the desired target assembly in the global assembly cache.


Partially Specified References

A complete AssemblyRef includes the textual name of the assembly, the version, the culture and the Originator (if the assembly has a shared name). Strictly speaking, only the text name of the assembly is absolutely required to do a bind. If not all parameters are specified, the AssemblyResolver will attempt to resolve the bind based on first match.

Partial references are often convenient to use in source code, but they do introduce some variability into the binding process. For example, if you issue a bind to a shared named assembly but don’t include the originator in your reference, you have no assurance that the assembly you get back came from the publisher you expect. Also, references that do not include the Originator are not subject to version policy, so the application and administrator config files are never checked. To be absolutely sure you’re getting what you want, include all of the parameters when referencing an assembly.

Given a partial reference, the AssemblyResolver will start by probing as it always does. The first assembly it finds that matches the criteria will satisfy the bind. For example, consider the scenario where there are two assemblies in the global assembly cache whose identities differ only by originator. If the desired originator is not specified on the bind, the AssemblyResolver will return whichever assembly it happens upon first.


Code Download Using IE5

This document describes how to download and run NGWS runtime assemblies from html pages using Internet Explorer. The NGWS runtime provides several improvements over the ActiveX model, particularly in the areas of code access security and deployment. Topics discussed in this specification include specifics of how NGWS runtime assemblies are referenced from html pages, how AppDomains are used to provide isolation between web apps and how code is cached when downloaded to the machine.


Overview of Code Download Using IE5

This document describes how to download and run NGWS runtime assemblies from html pages using Internet Explorer. The NGWS runtime provides several improvements over the ActiveX model, particularly in the areas of code access security and deployment.

Security in ActiveX required that a user either fully trust the downloaded code or it couldn't be installed on the machine. In the NGWS runtime, mobile code can be partially trusted, allowing finer grained control over what code is allowed to do.

If a user did decide to let an ActiveX control be installed, the impact was machine wide. Updates were made to the machine registry meaning that any other applications on the machine were impacted by the changes. A primary goal of the NGWS deployment model with respect to downloaded code is that it is "no-impact". That is, no other applications are affected and no "footprint" of any kind is left on the machine.

This specification includes the following topics:

  • Scenarios we need to support
  • The syntax for referencing assemblies in html
  • The packaging formats we allow
  • How AppDomains are created and references to assemblies resolved
  • A brief description of how downloaded code is cached on the machine

The features described in this document require version 5.0 of Internet Explorer on Windows2000 and NT4, and version 5.01 of Internet Explorer for Windows9x.

This document specifically does not address support for other browsers.


Requirements

  • Allow NGWS classes to be downloaded and accessed through web pages in Internet Explorer 5.0
  • Evidence for downloaded code must be passed through the security system to determine the set of granted permissions.
  • No footprint can be left on the machine. Nothing can be written to the registry as a result of downloading and running a class.
  • Code downloaded on behalf of one application cannot impact other applications on the machine. For example, downloading a QFE as a side effect of browsing to a particular page cannot cause other apps on the machine to pick up the QFE.
  • Must support download of both private and shared assemblies
  • The model for locating assemblies, specifying references to assemblies, and configuration must be consistent with the that of our other application scenarios (i.e exe and xsp).
  • Download of an assembly must be incremental. At a minimum, the assembly manifest must be downloaded to access types and resources within the assembly. As individual types and resources are accessed, the files in which they live are downloaded.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Scenarios

© 2000 Microsoft Corporation. All rights reserved.


Simple Web Site

Joe's toy shop, Wing Tip Toys, has a relatively simple web site consisting of a number of html pages and a few controls that get downloaded and executed on the client. The html pages are mostly kept in the site directory, but the downloadable controls are kept in a subdirectory that is secured so they can't be pointed at directly with a browser.

Joe's web site uses one shared assembly, Calendar from a third party vendor and two private assemblies that provide utility code for the app. The private assemblies are used from different pages but need to exchange data as part of the user interface experience of the web site. The web app directly references the third party website to download the control.

Joe's web site has no special configuration needs above the defaults provided by the NGWS runtime.

Optionally, Joe's web site may contain some aspx pages that are processed on the server and generate html that is sent to the client.

Joe's site is laid out as follows:

Exposed URL Namespace File System Layout
http://www.wingtiptoys.com

default.htm

custdata.htm

ordering.aspx

finalize.aspx

\img

logo.gif

toys.gif

(Notes:

  • The assemblies in the "bin" directory are for download to the client. For security reasons the “bin” directory is inaccessible by a browser.
  • The shared assembly is not physically stored at this web site.

)

 C:\inetpub\wwwroot

default.htm

custdata.htm

ordering.aspx

finalize.aspx

\img

logo.gif

toys.gif

\bin

customer.dll

accounts.dll


Multiple Web Apps Per Site

The HumanResources web site contains two applications: Benefits and Payroll. Each application consists of a number of html and aspx pages as well as NGWS runtime assemblies. Some of these assemblies run on the server and some are downloaded and run on the client. Allow they are published under the same site, these two applications are separate entities. There is no communication between them and in fact they must be isolated from each other.

Some of the html pages make reference to shared assemblies that are present on other websites.

The HumanResources site is laid out as follows:

Exposed URL Namespace File System Layout
http://HumanRes

\Payroll

default.htm

empdata.htm

absence.aspx

salary.aspx

\img

logo.gif

absence.gif

\Benefits

default.htm

empdata.htm

benlist.aspx

benadd.aspx

\img

logo.gif

health.gif

(Notes:

  • For security reasons the “code” and "dlls" directory is inaccessible by a browser
  • The shared assemblies used by both apps are not physically stored at this web site.

)

 C:\inetpub\wwwroot

\Payroll

default.htm

empdata.htm

absence.aspx

salary.aspx

\img

logo.gif

absence.gif

\code

employee.cab

statistics.dll

\Benefits

default.htm

empdata.htm

benlist.aspx

benadd.aspx

\img

logo.gif

health.gif

\dlls

employee.dll

benutils.dll

As a further example, consider an ISP that hosts a web site that allows individuals to post home pages for free. Each customer is assigned a directory on the web server. The contents of that directory and any subdirectories is completely up to them. However, the ISP needs to enforce that code used by one customer cannot interact with code from another customer.


Object Tag Syntax

The OBJECT tag is used to download an NGWS runtime assembly to the client using Internet Explorer. The location of the "package" containing the assembly manifest, and the name of the class to run are specified using the CLSID attribute of the OBJECT tag:

<object    id="myCtl"
         classid="http://www.contoso.com/foo.dll#fooClass"
</object>

The CODEBASE attribute of the OBJECT tag is ignored.

The path to the "package" may be absolute or relative to the "Application Base" (see later section on how the AppBase is established). Both shared assemblies and private assemblies may be downloaded.


Constraints

This solution requires that the NGWS runtime be installed on the target machine. IE5 will not know how to download and start the runtime.


Changes required in IE5

IE5 has been changed to make the bind context available to Mime filters. This is necessary for the Runtime to be able to report status of the download.


Download Formats

The NGWS runtime supports the following download formats:

  • PE file (dll or exe) The PE pointed to must contain an assembly manifest.
  • Design Ongoing! Cab. For Beta1 we have some support for cab files, but our current architecture won't allow a cab file to be referenced directly from the OBJECT tag using the syntax described above. Design details on how to solve this problem are currently being discussed. In general, the Assembly Resolver’s download support includes the capability to download a single assembly per cab file. The name of the file inside the cab that contains the manifest must be the name of the cab (minus the ".cab") + ".dll" or ".exe". For example, given a reference to foo.cab, Fusion will look for a file inside the cab called foo.dll (then foo.exe).


Application Domains and Probing

Application Domains are used to isolate "applications" running within a process. An important aspect of this isolation is the ability for the app to control where the NGWS runtime finds code to load on its behalf. In addition to scoping requests to load code, it is also important to scope configuration information to the application.

It is the job of the host to determine where the Application Domain boundaries lie for their particular type of application. In "browser-download" scenarios we call this the "IE Host". This section describes how the IE host establishes domain boundaries, associated configuration information with the domain, and sets up the corresponding Assembly Resolver context to scope requests to load code.


Determining the "AppBase"

Each Application Domain has an associated "AppBase" and configuration file. The AppBase is a directory that serves as the root directory when probing for assemblies on disk. The configuration file contains config information specific to the code running in the domain.

The scenarios above show that we must provide some flexibility in creating the Domain/AppBase/Config file. We can't always tie it to the site, nor can we always tie it to an individual html page or a directory on the web server. Instead we provide default behavior that is appropriate for most scenarios and allow developers to customize this behavior for more advanced scenarios.

By default the IE Host creates an AppDomain per site and sets the AppBase to the site directory. This handles many common scenarios, including the "Joe's Flower Shop" scenario described above.

Developers can change this default behavior by pointing to a configuration file using the LINK tag in their html pages. The syntax is:

<LINK REL=Configuration HREF="http://stevenpr/myapp/app.cfg">

The IE host will place all pages that point to the same configuration file in the same domain. The ability to create an AppDomain at something other than the site level is needed to support scenarios where there is more than one app per site, like the HumanResources and some ISP hosting scenarios.

If the HumanResources example above uses ASP+ it is likely they will have application config files in both the Payroll and Benefits subdirectories. These config files will be the same ones pointed to by the LINK tag in the HTML pages. This way, we unify the config of the server portions of the app with the client portions of the app. Stated differently, the concept of "application" as defined by the server portion is consistent with that of the client portion.


Probing

Once the AppBase is established, the following points describe how assemblies are located.

  • The assembly manifest files pointed to using the OBJECT tag may either be absolute or relative to the "app":
    <object    id="myCtl"
         classid="http://www.contoso.com/foo.dll#fooClass"
</object>

<object    id="myCtl"
         classid="./bin/foo.dll#fooClass"
</object>
  • Relative references are found relative to the docbase of the page. This is standard IE behavior and is not influenced by either the NGWS runtime or the Assembly Resolver. For example, if the relative reference to foo.dll above were contained in the page www.wingtiptoys.com/pages/customer.htm, IE would expect to download www.wingtiptoys.com/pages/bin/foo.dll.
  • Assemblies that are downloaded using the OBJECT tag may also reference other assemblies. When resolving such a reference, the Assembly Resolver will first look at the codebase that the calling assembly came from. For example, suppose an OBJECT tag causes an assembly named foo.dll to be downloaded from http://www.wingtiptoys.com/bin. Foo.dll then references bar.dll. The Assembly Resolver will first look for http://www.wingtiptoys.com/bin/bar.dll.
  • If no such file exists, the standard Assembly Resolver probing rules apply using the AppBase as the probing root. If no configuration file is specified, the Assembly Resolver will probe only in the Appbase. If a configuration file is specified, the Assembly Resolver will probe for the assembly using the "Private Path" specified in the configuration file. For example, assuming again that the AppBase is c:\inetpub\wwwroot, and given the following reference:
    <object    id="myCtl"
         classid="foo.dll#fooClass"
</object>

and cfg file:
<?xml version ="1.0"?>
<configuration xmlns="x-schema:catalog.xms">

<AppDomain
    PrivatePath="code;dlls"
/>
</configuration>

    The Assembly Resolver would look for the following files:

    C:\inetpub\wwwroot\foo.dll

    C:\inetpub\wwwroot\code\foo.dll

    C:\inetpub\wwwroot\dlls\foo.dll

    1. A few notes are worth mentioning here. First, probing over http is really not desirable for performance reasons -- we should provide recommendations to our customers that reduce the number of probing misses (this is true in general, however the performance cost of probing is exaggerated over http). Second, the Assembly Resolver probing rules also take into account directories named by version number and culture. See How the NGWS runtime Locates Assemblies for details.


Implementing the Scenarios

This section maps the implementation described in the previous section to the scenarios from the beginning of the document.

Simple Web Site – Wing Tip Toys

Wing Tip Toys does nothing to change the default behavior of the NGWS runtime with respect to creating AppDomains and locating assemblies. In fact, it does not even have a configuration file. Their default web page looks something like:

<html>

<head>
   <title>Wing Tip Toys</title>
</head>

<body>
<!-- lots of stuff here />
<object    id="customerCtrl"
      classid=".\bin\customer.dll#custClass"
   </object>

<object    id="calendarCtrl"       classid="http://www.contoso.com/public/calendar.dll#calendarClass"
   </object>
</body>

</html>

Notes:

  • Customer.dll is a private assembly found relative to the doc base of the page
  • Calendar.dll is a shared assembly downloaded from the vendor of the control

HumanResources

The HumanResources site contains two applications: Benefits and Payroll. The get the desired isolation behavior, the developers of the applications will include a LINK tag in each html page that points to the cfg file for that app. For example:

<html>

<head>
   <link rel="Configuration" href="http://HumanRes/Payroll/app.cfg">
   <title>AcmeCorp Payroll</title>
</head>

<body>
<!-- lots of stuff here />
<object    id="employeeCtrl"
      classid=".\code\employee.cab#empClass"
   </object>

<object    id="calendarCtrl"       classid="http://www.contoso.com/public/calendar.dll#calendarClass"
   </object>
</body>

</html>


Downloading Executables

Most types that are downloaded using the OBJECT tag are UI controls that appear visually on the web page itself. The NGWS runtime also supports the download of managed executables. There are two interesting scenarios:

  • A user types the url of a managed exe into the browser (i.e http://stevenpr/samples/MyApp.exe)
  • An html page contains a link to a managed executable (i.e HREF="MyApp.exe")

In both of these scenarios the NGWS runtime creates a new AppDomain in which to run the executable. For purposes of probing, the AppBase is set to the location that executable came from. Also, if the application has a configuration file, it must be in the same location as the executable.


IE Security

The NGWS security system allows an administrator to set security policy based on Authenticode signatures. The Authenticode signatures are carried in the assembly - specifically in the file containing the assembly's manifest.

As described above, the NGWS runtime allows assemblies to be packaged in CAB files for download purposes. Although the Authenticode signing tools allow CAB files to be signed, the signature on a CAB file cannot be used to make policy decisions in NGWS, i.e the NGWS runtime does not "apply" the signature on the CAB to each assembly in the CAB while evaluating policy. The Authenticode signatures must be on the assemblies themselves.


IE Admin UI

The IE Security dialog (tools->internet options->security->custom level) has been extended in NGWS to provide control over whether managed code is downloaded at all through the browser. This is in addition to the control that can be obtained through code access security.

Specifically, the following choices have been added to the security dialog:

Managed Controls
  1. Enable
  2. Disable
  3. Prompt
Enable: enables the download and hosting of managed controls
Disable: disables the download and hosting of managed controls
Prompt: prompts browser user before managed control is downloaded and run


Config Hierarchy

The hierarchy of configuration files for the IE host is as follows:

Application Config: This file resides on the server. It is either pointed to explicitly from the html page or assumed by default to be in the site directory.
Machine Config: This file is called admin.cfg. It resides on the client in the windows directory.

Any configuration data (i.e version policy) that is specified per machine is picked up on the client, not the server.


Error Reporting

If the download cannot complete for any reason, IE displays the standard box to indicate a failed download as shown in the following picture:


IE5 Activation Reference Documents

  • NGWS Application Domains: http://comrtime/Specs/admin/AppDomains.doc
  • Fusion Probing Heuristics: http://fusion/Projects/Helium/Helium%20Specs/probing.doc
  • Alan's web binding doc (not sure where this is)
  • HTML 4.01 spec: http://www.w3.org/TR/html4/cover.html


Appendix A - NGWS Runtime MIME Filter

This section describes the mechanics of how the NGWS runtime works with IE to actually perform the download. The primary component is a MIME filter developed by the Runtime team. This filter is packaged in a separate dll and is not shipped with IE5.

The filter is registered to handle the following MIME types:

  • application/octet-stream
  • application/x-msdownload
  • application/x-mscomplus

When IE starts the download of code with one of these MIME types, the NGWS runtime MIME Filter gets inserted in the chain of filters called by IE.

The filter is responsible for the following tasks:

  • Determining if the code being downloaded is a runtime PE file.
  • Passing activation information into the runtime, including starting (or routing to) the AppDomain in which the code will be run.
  • Returning the resulting Class Factory pointer to Trident.

The following picture describes the overall architecture and the specific steps that are involved in activating runtime classes in IE5:

  1. As described above, the Mime filter is registered for ‘application/octet-stream’, ‘application/x-msdownload’, and ‘application/x-mscomplus’.
  2. The Mime filter looks at the bits coming down to determine if the file is a runtime PE file. It does this by looking at directory entry 14.
  3. If the file is not a runtime executable, the filter passes control onto the rest of the filter chain and code download happens as before.
  4. If the file is a runtime executable, the filter starts the runtime and passes it the information required for activation. This information includes the full URL specified in the CLSID attribute of the OBJECT tag, and the IBindCtx interface it can use to report progress. The Runtime must be present on the machine. IE does not know how to download and start the Runtime.
  5. At this point, the Runtime determines which domain the code will run in (starting a new one if necessary), loads the DLL described in the URL, and creates a class factory that can be used to create the actual objects.
  6. The Runtime gets the IBindStatusCallback out of IBindCtx and sets the resulting class factory in the call to IBindStatusCallback::OnObjectAvailable. This makes the Mime filter aware that the object has been created.
  7. The Mime filter then reports back to the download transaction that the download is complete.
  8. IE’s processing of the mime filter chain now stops and the class factory interface pointer is returned to Trident.
  9. Trident now uses the IP to create objects as it needs


COM Interoperability Specification

This document highlights the COM Interoperability features of the NGWS runtime. It describes how unmanaged COM applications can gain access to the services exposed by NGWS classes and how managed NGWS applications can gain access to the services of COM servers in a seamless and transparent manner.


Overview of Interoperability

This document describes how the NGWS runtime interacts with all external entities, including classic COM, external libraries, and the operating system. The document covers how the NGWS runtime exposes its services externally, as well as how Runtime can access services exposed by the external environment.

The external view of the runtime will be especially important to classic COM clients that need access to objects inside the runtime. Since it’s unrealistic to expect both the client and server portions of existing applications to be upgraded to NGWS at the same time, we have to assume there will be a period when servers live within the Runtime and clients remain classic COM. During this period, it’s important that these hybrid applications continue to operate normally without affecting the classic COM client. The fact that these new objects have a different implementation must be completely transparent to existing clients.

Equally important is the need for new Runtime clients to have access to the services provided by classic COM servers. Again, it’s unrealistic to expect all classic COM servers to be upgraded to NGWS overnight. For one reason or another, some classic COM servers may never be upgraded to NGWS. Others may be upgraded to NGWS over an extended period of time. Regardless of how simple it is to migrate from classic COM to NGWS, there will be a period when NGWS clients need the services provided by classic COM servers. Therefore, the objects contained within a classic COM server need to be exposed within the runtime in such a way that they appear to be true NGWS objects.

One determining factor in how fast servers are upgraded is the amount of effort needed to do the upgrade. Therefore, NGWS interoperability must also ensure that the process of migrating from classic COM to NGWS is as simple and effortless as possible. While most of the work involved in the migration is language or tool dependent, the NGWS frameworks and runtime team can also take steps to simplify this process.


Goals, Non-Goals and Limitations of Interoperability

The overriding goal of NGWS interoperability is to insure that the NGWS runtime is 100% compatible with classic COM clients and servers. At the most basic level that means that the release of the NGWS frameworks and runtime will not interfere in any way with the continued use of applications that abide by existing COM specifications. It also means that either the client or the server portion of existing applications can be upgraded to NGWS independently without impacting the functionality of the application.

The following goals and requirements describe how the NGWS runtime will interact with its external environment. To the runtime, this external environment includes both classic COM clients and server objects.


Goals

Coexistence of classic COM and the NGWS Runtime

With the NGWS frameworks and runtime fully installed on a computer, all classic COM applications must continue to work, without modification, in exactly the same way they worked before the runtime was installed.

Upgraded COM Server Compatibility

Classic COM clients must continue to work in exactly the same way when their respective classic COM servers are upgraded to run in the NGWS runtime environment. The client must be completely unaffected by the upgrade of the server. This implies that the set of interfaces exposed by the runtime on behalf of the new NGWS runtime server must be binary compatible to the set of interfaces exposed by the older version of the same server. In addition, all other aspects of the interaction between the client and server must also be identical, including discovery, activation, location transparency, error handling, synchronization, eventing, and lifetime management.

This requirement does not imply that the NGWS runtime server must be source code compatible with the classic COM server. Source code compatibility depends on the tools used to create the server.

Upgraded COM Client Compatibility

Classic COM servers must continue to work in exactly the same way when their respective classic COM clients are upgraded to run in the NGWS runtime environment. The server must be completely unaffected by the upgrade of the client. This implies that the NGWS classes exposed by the runtime on behalf of the classic COM server are binary compatible to the set of interfaces exposed to the older version of the same client. In addition, all other aspects of the interaction between client and server must be identical, including discovery, activation, location transparency, error handling, sinkhronization, eventing and lifetime management.

This requirement does not imply that the NGWS runtime client must be source code compatible with the classic COM client. Source code compatibility depends on the tools used to create the client.

Mixed Client/Server Compatibility

A NGWS runtime server that is capable of serving multiple clients must be capable of serving both classic COM clients and runtime clients simultaneously. It should be transparent to the server that some of its clients are classic COM, while others are Runtime clients. Both types of clients must be able to use the same instance of a particular NGWS runtime server at the same time.

The reverse also applies. A NGWS runtime client that is capable of interacting with multiple COM servers must be able to interact with both classic COM servers and NGWS runtime servers simultaneously. This flexibility allows some classic COM servers to be upgraded without requiring that every server used by a particular application be upgraded at once. The fact that one server was upgraded while others remain classic should be transparent to the client applications.

Performance Targets

The performance of the NGWS runtime interoperability layer is absolutely critical to the success of the NGWS frameworks and runtime. While marshaling calls between classic COM and the NGWS runtime is expected to be slower than marshaling in classic COM, we expect this extra overhead to be offset by improved performance within the runtime, so that the overall performance of a reasonably complex method call will remain the same. The raw overhead of an in-process, direct vtable bound method call between classic COM and the NGWS runtime with simple scalar data types for parameters should be on the order of 10 machine instructions. Considering that the overhead in classic COM is a single jump instruction, our goal of 10 instructions can add significantly to the call overhead when the amount work done within a method call is very small. However, typical method calls are more complex, and therefore the impact of 10 instructions should be negligible when compared to the number of instruction executed within such calls.

The overhead on late bound method calls should be not noticeable.


Non-Goals of Interoperability

Source Code Compatibility

The runtime’s Interop services operate at the binary level as opposed to the source level. It provides a mechanism for exposing managed types in a binary format that is compatible with COM and vice-versa. There are no guarantees regarding the level of effort required to build a type that are binary compatible with a COM type.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Limitations

The interoperability services provided by the runtime are not without limitation.


NGWS objects exposed to COM

When a NGWS runtime assembly is exposed to COM, the following limitations apply:

  • Assemblies may require a corresponding type library at runtime.

When types within a NGWS runtime assembly are exposed to COM, the following limitations apply:

To all types:

  • Parameterize constructors are not exposed.
  • Static methods are not exposed
  • Const fields are not exposed.
  • Data marshaled into and out of the type may be copied and or transformed
  • The inheritance hierarchy will be flattened
  • Type will not have the same versioning characteristics as other manage types.

Types that are creatable:

  • Require registration
  • Default constructors may be required.


COM objects exposed to the NGWS Runtime

When a COM type is exposed through a RCW to the NGWS runtime, the following restrictions are imposed on that type:

  • Success HRresults.
  • Typedefs
  • Variable length arrays
  • Module level methods


Interoperability Through Proxies

The fundamental difference between objects that live within the runtime and those that live outside the runtime is that they are managed objects. For more information on managed object see the Virtual Object System (VOS) Specification

Classic COM objects are allocated from a standard Windows heap. These objects are considered unmanaged objects. By convention, the object itself is responsible for freeing the memory it uses when no more references to the object exist. This is accomplished through COM reference counting.

In the NGWS runtime, managed objects are allocated from the runtime’s managed heap and are automatically garbage collected by the runtime. In order to do this, all references to managed objects are traced. When the last reference to the object is removed, the runtime garbage collector reclaims the memory occupied by the object This eliminates the need to reference count managed objects. The runtime essentially handles the reference counting internally.

Tracing is possible within managed code because the runtime is aware of the outstanding references that exist on an object. As each new object reference is declared within managed code, the runtime adds the reference to a list of live references. At any given time, the runtime is aware of all live references that exist on a given object. As the references fall out of scope or change their value, the list of live references is updated. As long as the reference is within managed code the runtime is able to trace it. References from unmanaged code are a different story.

There is no way for the runtime to trace references from unmanaged code to managed objects. Even if the runtime knew about the references, there would be no way to tell if a reference was live or dead. Because classic COM clients are written in unmanaged code, there’s no way of tracing the references they hold on managed objects living within the runtime.

Therefore, the runtime does not allow classic COM clients to reference managed objects directly, instead the runtime creates a wrapper called the COM Callable Wrapper (CCW) that acts as a proxy for the real managed object.

The CCW is not garbage collected and therefore it can be directly referenced by unmanaged clients. The object that it wraps though is garbage collected like any other managed object. The CCW is responsible for handling all aspects of the interaction between managed objects and COM clients. For example, the CCW handles data type marshaling and the translation of COM HResults into NGWS runtime exceptions. In general, the CCW does everything necessary to make the COM client think it is using a COM server and to make the NGWS runtime server appear as though a NGWS runtime client is using it. Section 4 of this document covers the CCW in detail. It addresses all aspects of the interaction between COM clients and NGWS objects. In other words, it addresses unmanaged to managed interoperability.

Making calls from NGWS to COM also presents a problem when it comes to managing object lifetime. COM objects expect to be reference counted, but in a managed environment reference counting is unnatural. It would be very confusing if developers writing managed code had to handle references to managed and unmanaged objects differently (not to mention the problems it introduces to the upgrade process). To abstract the differences between managed and unmanaged references the runtime uses another type of wrapper class called a Runtime Callable Wrapper (RCW). The RCW is a managed object that acts as a proxy for a reference counted COM object. The RCW is garbage collected in the usual way but also maintains an internal cache of interface pointers on a single COM identity. The internal interface pointers kept by the RCW are reference counted like any other COM interface but references to the RCW itself are traced. The RCW is responsible for handling all aspects of the interaction between managed clients and COM objects. This include maintain object identity, data marshaling and exception handling. In general, the RCW does everything necessary to make the NGWS runtime client think it is using a NGWS runtime server and to make the COM server appear as though a COM client is using it.

Section 5 of this document covers the RCW in detail. It addresses all aspects of the interaction between Runtime clients and COM objects. In other words, it addresses managed to unmanaged interoperability.


Exposing NGWS objects to Classic COM

This section describes the mechanism that makes it possible to use NGWS objects from unmanaged code. This is the mechanism that classic COM clients use to access the services provided by NGWS objects. When a classic COM client creates an instance of an NGWS object, the object is exposed through a COM Callable Wrapper (CCW) that acts as a proxy for the real managed object. These wrappers are not visible within the runtime by other NGWS classes; they are simply there for marshaling calls between managed (NGWS runtime) and unmanaged (classic COM) code. Each NGWS object being referenced outside the runtime will have a single CCW regardless of the number of references that exist on that object. This ensures that a single object identity is shared between the CCW and the COM object.

Figure 1Classic COM client and NGWS runtime client using a NGWS objects IFoo interface

The CCW is really a proxy for the NGWS object being referenced from unmanaged code. It is transparent to the managed object that it is being referenced by a CCW as opposed to a NGWS runtime client. The CCW’s reference on the managed object is traced like any other managed reference. The COM client’s reference on the CCW is reference counted like any other COM reference. If multiple COM clients share a reference to the same managed object, the runtime guarantees that they will share the same CCW. In other words, there will never exist more than one CCW for any instance of a managed class.

The bulk of this section covers the responsibilities of the CCW in detail.


Binding to NGWS classes

COM applications can bind to managed classes in either an early bound or late bound fashion. Late binding in this context refers to using the IDispatch interface exposed by the object. Early binding (also called v-table binding) implies binding through an interface other than IDispatch. Late binding requires compile time type information while late binding does not.

Early Binding

The runtime generates type information in the form of a COM type library for COM applications to bind to. The type libraries generated can then be imported into most development tools to provide complete compile-time type information about managed types. A COM type library can be generated by converting the metadata within any NGWS runtime assembly. This conversion is explained in section 4.1.4 below.

Late Binding

COM client are free to bind to managed types using the standard late binding mechanism provided by COM, namely IDispatch. All managed types automatically support IDispatch without having to write any additional code. The IDispatch implementation is automatically provided by the runtime by examining the metadata for the type. For details of IDispatch implementation the runtime provides, see section 4.9.2. Under some circumstances, the runtime may require a type library in order to provide an implementation of IDispatch.

Type Library Generation

A type library describing the types contained in a managed assembly may be needed for several reasons.

  • During development, so that clients can bind to the types defined within the assembly.
  • During development, so the development tools like Visual Studio can provide type checking and auto completion.
  • During execution, so that object can provide COM style type information to clients that request it (like Internet Explorer)
  • During execution, to save the performance cost of generating a type library on the fly (see section 4.9.2.2 on on-the-fly type library generation).
  • During execution, in order to use the OleAut marshaler to marshal remote instances of object defined within the assembly.

As section 4.9.2.2 describes, the runtime is capable of generating type libraries on-the-fly as needed. However, a static type library describing the types defined in a NGWS runtime assembly can also be generated in two ways:

  • Using the TlbExp utility.
  • Using the System.InteropServices.TypeLibConverter object described below.

Using TlbExp (Type Library Exporter)

The TlbExp utility is provided as part of the NGWS SDK. TlbExp is a command line tool that uses the API described in the following section to export a type library generated from a NGWS runtime assembly.

TlbExp has the following command line syntax:

TlbExp AssemblyName [/out: TlbFile]

The TlbExp utility generates a type library containing definitions of the public types defined in the NGWS runtime assembly named AssemblyName.

Options:

/out: TlbFile - Specifies the name of the tlb file to be generated. If the option is omitted, the type library is generated with the same name as the assembly (the actual assembly name not the name of the file containing the assembly) with the .tlb extension.

Examples

TlbExp mytest.dll

Generates a type lib with the name same name as the assembly found in mytest.dll.

TlbExp mytest.dll /out clipper.tlb

Generates the type library with the name clipper.tlb.

Using ITypeLibConverter to Export a Type Library

The TypeLibConverter class in the System.InteropServices namespace implements the ITypelibConverter interface that can be used by both managed and unmanaged applications that need to convert metadata to type information or vice-versa.

For a complete description of the TypeLibConverter class, see the reference pages for the specific type.

A complete description of the actual conversions made during type library generation is contained in the sections below.

Note on using type libraries

Most development tools are designed to easily consume type information in the form of a type library. Type libraries produced by TlbExp or the TypeLibConverter object are no exception. The Visual Basic 6.0 development environment for example, allows user to directly add a reference a type library (tools references menu). Using a type library from Visual C++ 6.0 is also possible but a little trickier. A C++ source file can reference a type library using the #import statement:

#import <windows.h>
#import <mylib.tlb>

//  reference types defined in mylib.tlb

When the C++ pre-compiler parses the #import statement it imports the type library and generates a corresponding .tlh file that contains the same type definitions in C++ syntax. The compiler then parses the .tlh file as a normal C++ header file. The .tlh file only needs to be converted once for a given .tlb file. Any changes to the .tlb file will cause a new .tlh file to be generated.

There are several options that can be used with the #import statement (see the Visual C++ documentation for details). Two options are of particular importance when using libraries generated with TlbExp. When importing a library, users must either 1) use the raw_interfaces_only option or 2) import the definitions of the types defined in the base class library (mscorlib.tlb).

Importing a library without mscorlib.tlb

   #import "thedll.tlb" raw_interfaces_only

Importing a library with mscorlib.tlb

   #import "mscorlib.tlb"
   #import "thedll.tlb"

Another issue to be aware of when using type libraries from within C++ has to do with case sensitivity of assembly names. NGWS runtime assemblies names are case insensitive. During exporting the assembly name is used create the type library name. During #import the type library name is used to create the namespace name. The namespace name then used to reference the types within the given library. Because the types are referenced within C++ source code, the type names are case sensitive. As a result, two assemblies that differ only in the case of their names are considered distinct assemblies from unmanaged C++. Developers should therefore be sure to use the same case when creating naming assemblies. The way the assembly name is provided varies according to the tools being use. In most cases the assembly name is provided as part of the build process. See your compiler documentation for details.

One final point worth mentioning is that names within type libraries are folded into a common, case insensitive string table. This includes types names, member names and parameter names. In situations where a type, member and/or parameter share the same name, the name has common casing. The case of the common name is the same as the case used in the first reference to that name. For example, if the following types were exposed to COM, the type named Foo, the method name FOO and the parameter named foo, would all be exported as Foo because that’s the case used in the first reference to the name foo.

class Bar {
   int FOO();
}

class Foo { …
}

class Baz {
   int Baz(int foo);
}

Assembly to Type library Conversion

The mechanisms described above convert NGWS runtime assemblies to COM type libraries. During the conversion, certain transformations are made. Because type libraries are unable to accommodate all the information found in assemblies, some data may be discarded. This section explains those transformations and identifies the source of each piece of information emitted to the resultant type library. The transformations described here are those made by the ITypeLibConverter interface.

Common conversions

There are some common pieces of information that are seen throughout an assembly and are converted the same way regardless of where they appear. Information like version and helpfile are examples. This information is always optional and can be provided for an entire assembly, an individual type or member of a type. Rather than repeat the conversion rules in each of the following sections, they appear once here.

The version information in an assembly consists of a 4-part version string similar to “2,1,23,443”. The first two parts are considered the product version while the last two parts make up the build version. In type libraries, a 2-part version string is used (like “2.1”). When converting from assembly to type library, the first two parts of the assembly version (the product version) are transferred to the type library version. The build version is discarded.

Assembly level conversion

At the outermost level of the conversion, the managed assembly is converted to a COM type library. Each individual assembly is converted into a single type library. There is always a one-to-one mapping between assembly and type library. An assembly cannot be broken up into multiple type libraries.

The conversion is deterministic. A given assembly will always generate the same type library regardless of when or where it was converted.

Type libraries are identified solely by their type library identifier (TLBID), version and LCID. The name of the file containing the type information is not part of the libraries identity nor is the friendly name defined within the library. Type Libraries are located through the registry via their TLBID, version and LCID.

Sample Library

[ uuid(0D26FC72-7EB1-4565-AA75-DA5F177EFA66),
  lcid(0x0409),
  version(2.1),
  helpstring("Acme Widget Library")
]
library Acme
{
…
};

Assemblies are identified by the combination of their SimpleName, OriginatorKey, version and locale. The Originator key is the public part of a public/private cryptographic key pair. The SimpleName is the friendly that’s used to refer to the assembly. The version is a four-part numeric string. The first two parts are the major and minor product version while the last two parts are the build numbers. The locale string indicates the locale that the assembly supports. Other information is not part of the assembly identity.

Sample Assembly

SimpleName:       Acme
OriginatorKey:    Public Key of Acme Corp
Version:          2,1,23,443
Locale:          en-us
Title:            Acme Widget Library
Processor:         x86
OS:               NT4

The conversion from assembly to type library preserves the identity of the library. In other words, each uniquely identified assembly converts to a uniquely identified type library. To ensure this uniqueness, the conversion process uses the SimpleName and OriginatorKey from the assembly to produce a unique TLBID. A given SimpleName and OriginatorKey combination will consistently yield the same TLBID. Conversely, no two assemblies with different SimpleName and OriginatorKey combination will every produce the same type library.

SimpleName + OriginatorKey => TLBID

Since the version and locale make up the rest of the type libraries identity, they are not used in the generation of the TLBID. The version information from the assembly is also passed along to the type library although type libraries contain only a two-part version number while assemblies contain a four-part version number. The assembly’s major and minor version numbers are carried forward to the type library. The assembly build numbers are discarded in the conversion. This does imply that two assemblies that differ only by their build numbers will yield the same type library.

The assembly locale id string (i.e. “us-en”) is converted to an LCID (i.e. 0x409) and stored in the LCIS field in the type library. If the locale is not specified the LCID is set to 0.

A type library is typically referred to by its library name (not the file name). The name is copied directly from the assemblies SimpleName. Since an assemblies simple name can (and often does) contain periods which are not permitted in type library names, all periods in the simple name are converted to underscores.

The helpstring (also called DocString) of the typelib is copied from assembly’s Title field. The HelpFile and HelpContext fields in the type library are not set. The LIBFLAGS in the typelib are also not set.

Other assembly fields such as Processor, OSInformation, CodeBase, Alias, AssemblyHash, Configuration and Flags are discarded during the conversion.

Other type library attributes such as hidden, restricted and control are not set.

In general, there are no guarantees about the content of a typelib from version to version. In other words, a type defined in version 1 of a particular library is not guaranteed to be in version 2.

Module level conversions

A NGWS runtime assembly may contain the definition of several external methods that are actually implemented in unmanaged code. The managed definition is provided so that managed applications can take advantage of code implemented in unmanaged dynamic link libraries. During export of a NGWS runtime assembly, a type library module will be created for each dynamic link library externally referenced from within the assembly. Each module will contain all the external methods that reference a single DLL. The module is marked with the dllname attribute to indicate the DLL containing the implementations of the methods defined within the module.

External Methods

class Utility
{
   [DllImport("kernel32",
EntryPoint=”CreateDirectory”,
CallingConvention=CallingConvention.StdCall]
   public static extern bool Create (string name);

   [DllImport("User32"]
EntryPoint=”MessageBox”,
CallingConvention=CallingConvention.StdCall]
   public static extern int MsgBox (string msg);
}

class MoreUtils
{
   [DllImport("kernel32"]
EntryPoint=”RemoveDirectory”,
CallingConvention=CallingConvention.StdCall]
   public static extern bool Remove (string name);
}

Converted to

   [dllname(“Kernel32.dll”), …]
   Module Kernel32Module
   {
      [entry(“CreateDirectory”) pascal bool Create([in] BSTR name);
      [entry(“RemoveDirectory”) pascal bool Remove([in] BSTR name);
   }

   [dllname(“User.dll”), …]
   Module User32Module
   {
      [entry(“MessageBox”) pascal int MsgBox([in] BSTR name);
   }
}

Type level conversions

Each type within the managed assembly can potentially be exported to the resulting type library. When exported, types carry the same name they had within their assembly (excluding the namespace portion of the their name). So, for example, the type A.B.IList:

namespace A {
   namespace B {
      interface IList {
            …
      }
   }
}

Would be exported with the name “IList”. This allows COM applications to refer to the type as simply “List” as opposed to “A.B.IList”. The using this approach, type names within an assembly can potentially collide since types within different namespaces may have the same name. When such collisions occur, the type will be exported using the its complete name (including namespace) to ensure its uniqueness. Since dots are not allowed in type library names, the dots are replaced with underscores. For example, the following types within the namespaces A, B and C:

Assembly Widgets
{
   namespace A {
      namespace B {
         class LinkedList : public IList {…}
         interface IList {…}
      }
   }
   namespacee C {
      interface List {…}
   }
}

Would be exported as shown below.

Library Widgets
{
   […]
   coclass LinkedList
   {
      interface A_B_IList;
   }

   […]
   interface A_B_IList {…}
   […]
   interface C_IList {…}
}

In addition to the type name, types that are COM creatable are also given a ProgId. The ProgId is generated automatically by combining the namespace combined with the type name. The ProgId of the LinkedList class shown above would be A.B.LiskedList. however, simply combining the namespace and type name can, under some circumstances, result in an invalid ProgId. For example, ProgId’s are limited to 39 characters and cannot contain punctuation characters other than periods. In that case, the ProgId can be specified explicitly with the ProgIdAttribute. If the ProgIdAttribute is applied, the class will be registered with the ProgId specified rather than using the generated ProgId. If for any reason the ProgId is invalid, the RegAsm utility will generate a warning at registration time and the class will not be registered.

There are 4 different type definitions that can be found in an assembly.

Classes

Managed classes are exported to type libraries as a coclass. The coclasses have no methods or properties. The coclass has the same name as the original managed class and implements the all interfaces that the managed class explicitly implemented. In addition, the coclass implements one other interface called the class interface that is generated during the conversion process.

The class interface has all methods and properties available on the original managed class. This allows COM clients to effectively access the methods and properties of the managed class by calling through the class interface. For example, the Button class shown below:

class Button : Control {
   void click();
}

Would produce the following type information:

[
   uuid(…), hidden, dual, odl, nonextensible, oleautomation
]
interface _Button : IDispatch {
   HRESULT ToString([out, retval] BSTR* pRetVal);
   HRESULT Equals( [in] VARIANT obj, [out, retval] VARIANT_BOOL* pRetVal);
   HRESULT GetHashCode([out, retval] long* pRetVal);
   HRESULT GetType([out, retval] _Type** pRetVal);
   HRESULT click();
}

[
   uuid(A03A6562-3425-3288-99AD-9533F36ADB14)
]
coclass Button
   [default] interface _Button;
   interfce _Object;
}

Notice that the class interface contains all the method implemented by the managed class as well as the methods of its base classes. The class interface always carries the hidden, dual, odl, nonextensible and oleautomation attributes. The uuid attribute assigned to the class interface is generated during export. It cannot be changed. For complete details on the class interfaces see section 4.9.3 below).

A managed class can be assigned an associated uuid by attributing the class with the GuidAttribute. When a managed class is exported, the value of the GuidAttribute is used for the uuid attribute in the type library. If the class does not have a GuidAttribute assigned, a uuid is automatically generated during the export process. Generated uuids are produced from a hash that includes class’s complete name (including the namespace). This ensures that a class with a given name in a given namespace will always generate the same uuid and that no two classes with different names will ever generate the same uuid.

Managed classes that are not COM creatable, such as abstract classes and those without public default constructors, are marked with the noncreatable attribute.

Other type library coclass attributes such as licensed, hidden, restricted, and control are not set.

Interfaces

Managed interfaces are export as to type libraries as COM interfaces. The interface has the same methods and properties as the managed view of the interface but the method signature differ considerable. See member conversions below for details. COM interfaces are identified by their IID. A managed interface can be assigned a fixed IID by attributing the interface definitions with the GuidAttribute. If the interface is not attributed with the attribute and IID is automatically assigned when the type library is exported. The assigned IID is based on the interface name (including namespace) as well as the complete signature of all the methods defined within the interface. This ensures that any changes made to the interface will result in a new IID being defined if the interface definition re-exported. Reordering the methods, change their argument types or return type will all affect the IID generated. Changes to method names do not affect the generated IID.

Interfaces that are not attributed with a fixed IID can still be still be requested through the runtime’s implementation of QueryInterface. QueryInterface will work for both fixed interface IID’s as well as generated IID’s. Generated IID’s are not persisted within the metadata for the type.

Value types

Value types (types that extend System.Value) are exported to type libraries as c-style structures along with typedef. The layout of the structure members is controlled with the StructLayoutAttribute, which is applied to the type. Only the fields of the value type are exported. If a value type has methods, they are inaccessible from COM.

For example:

[StructLayout(LayoutKind.Sequential)]
struct Point {
   int x;
   int y;
   public void SetXY(int x, int y){
      this.x = x;
      this.y = y;
   }
};

The Point value type is exported to COM as a point typedef shown below:

typedef struct tagPoint {
   int x;
   int y;
} Point;

Notice that the SetXY method is removed from the typedef.

Enumerations

Managed Enumerations are exported to type libraries as a COM as an enum with the member names altered in order to ensure the uniqueness of each members name. Managed enum members are scoped to the enum in which they belong. In other words, all references to Sunday in the DaysOfWeek enum shown below must be qualified with the name of the DaysOfWeek enum. Referencing just Sunday instead of DaysOfWeek.Sunday is not permitted. Because enum members must always be qualified, multiple enum’s are able to have member with the same name.

In COM and in unmanaged C++ however, enum members are scoped to the entire library. This allows members to be used without qualifying which enum the member belongs to. So, for example, Sunday can be used in place of DaysOfWeek.Sunday. Therefore, each enum member must have a unique name. Have two enums with the same member names is not permitted.

To ensure that each member name is unique, the name of the enum along with an underscore is prepended to each member when the enum is exported to a type library. For example:

Enum DaysOfWeek {
   Sunday = 0;
   Monday;
   Tuesday;
   …
};

Is exported as:

Enum DaysOfWeek {
   DaysOfWeek_Sunday = 0;
   DaysOfWeek_Monday;
   DaysOfWeek_Tuesday;
   …
};

Member are then be referred to as DaysOfWeek_Sunday by unmanaged applications.

COM Visibility of Types

Type Visibility: The COM visibility of managed types is largely determined by the managed visibility of that type. Simply put, public types are visible to COM while private, family and assembly scoped and nested types are not visible to COM.

Visibility of types can also be controlled explicitly with the ComVisibleAttribute. The attribute can be used to reduce the visibility of a type but cannot be used to increase the visibility of a type. In other words, the attribute cannot be used to make a non-public type visible.

When used to attribute an assembly, the ComVisibleAttribute applies to all type within the given assembly. For example, setting the ComVisibleAttribute of an assembly to false hides all the types within the assembly.

When used to attribute an individual type, the ComVisibleAttribute can be used to hide a public type from COM. An attribute applied to an individual type overrides the any attribute applied to the assembly. For example, the attribute can be used to make one type visible within an otherwise hidden assembly.

In the following assembly, only interface IB and class C would be COM visible.

Namespace Widgets
{
   private interface IA {}
   public interface IB {}
   [COMVisilble(False)] public interface IC {}

   protected class A : IB {…}
   public class C {…}
   [COMVisilble(False)] public class D {…}
}

The COM visibility of a type has the following implications.

Types that are not visible to COM:

Do not appear in the type libraries generated by TlbExp.

Cannot be obtained through the runtime implementation of QueryInterface.

Cannot be created from COM.

Cannot be used in the public interface of any type that is COM visible.

When a non-visible type is used in the public interface of a visible type the type is either replaced by an IUnknown interface or the member is not exported. Consider the following example:

Namespace BillingSystem
{
   private struct Address {…}
   private class Invoice{…}
   public class Customer {
      public void SendInvoice(Invoice I);
      public void SetAddress(Address A);
   }
}

In this case the public Customer type uses the private types Invoice and Address within its public interface. When exported to a COM type library, the customer class would appear as:

Library BillingSystem
{
   interface _Customer {
      HRESULT SendInvoice(IUnknown *I);
   }
   coclass Customer {
      interface _Customer;
   }
}

The Invoice type used in the SendInvoice method is replaced with an IUnknown. The SetAddress method is removed from the interface completely. A warning is generated when TlbExp tried to export the method.

Accessibility of members: All members of managed interfaces must be public and therefore are also publicly accessible from COM. There is no mechanism for selectively removing members of a managed interface from the COM representation of that interface.

Classes are also exposed to COM through the class interface. The class interface contains all the public, non-static members of the class. Class members that are private, protected or static are not accessible through the class interface. The same accessibility rules apply the member being access through the IDispatch portion of the class interface.

Users can access the static or protected member of a class from COM by wrapper the access to those member in another public non-static member. For example:

public   class IHaveStatics
{
   public static void StaticMethod() {…}
   public void InstanceStaticMethod() {
      IHaveStatics.StaticMethod();
   }
}

There is no mechanism for limiting the visibility or accessibility of a member through the IDispatch interface that the runtime exposes.

Member level conversion

Managed types may contain methods, properties, fields and events all of which are exported COM in one form or another.

Method Conversions

Classic COM clients expect to make method calls passing familiar COM data types as parameters and receiving HResults in return. NGWS classes on the other hand have no restriction on return types (and in fact don’t use HResults) and prefer to implement methods using familiar NGWS types.

In order to satisfy both models, every method of a managed type has an NGWS signature and an implied COM signature. The two signatures are typically very different. NGWS runtime clients interact with the server using the NGWS runtime signature while (possibly at the same time) COM clients interact with the server using the COM signature. The server implements the method with the NGWS runtime signature and the runtime marshaling service is responsible for providing a stub with the COM signature the delegates the call to the managed method.

HResult Translation

A managed signature is converted to an unmanaged signature by changing the managed return value to an out, retval parameter and changing the type of the unmanaged return value to HResult. For example, the DoSomething method might have the following signatures:

Managed Signature

long DoSomething(long i);

Unmanaged Signature

HRESULT DoSomething([in] long i, [out, retval] long *rv);

Notice that the COM signature returns an HResult and has an additional out parameter for the return value. The return value from the managed implementation is always returned as an out parameter added to the end of the unmanaged signature and the unmanaged signature always returns an HResult. If the managed method has a void return, the out retval parameter is omitted. For example:

Managed Signature

void DoSomething(long i);

Unmanaged Signature

HRESULT DoSomething([in] long i);

Under some circumstances, its preferable the leave the managed signature unchanged. The PreserveSignatureAttribute can be used to do this. For example:

Managed Signature

[PreserveSignature] long DoSomething(long i);

Unmanaged Signature

long DoSomething ([in] long i);

Having two distinct method signatures makes it easy for the class to be used by both COM and runtime clients seamlessly. In fact, both COM and Runtime clients can use a NGWS class simultaneously. The author of the class only ever implements the managed signature. The only place the COM signature is ever used is within the type library generated for the class.

Overloaded Methods

While the NGWS runtime does support overloaded methods, the IDispatch interface relies solely on method name for binding rather than the complete method signature and therefore is not capable of supporting overloaded methods. However, in order to provide access to overloaded methods of a type, TlbExp decorates the names of overloaded methods with an ordinal number so that each method name is unique. For example,

Managed Signature

interface IFoo {
public:
   void DoSomething();
   void DoSomething(short i);
   void DoSomething(long l);
   void DoSomething(float l);
   void DoSomething(double l);
}

Unmanaged Signature

interface IFoo {
   void DoSomething();
   void DoSomething_1(short i);
   void DoSomething_2(long l);
   void DoSomething_3(float l);
   void DoSomething_4(double l);
}

The COM signature for the methods appear as a single DoSomething method followed by a series of decorated DoSomething_x methods where x starts from 1 and continues for each overloaded form of the method. Note that some of the overloaded methods may be inherited from a base type. There is no guarantee that overloaded methods will retain the same number as a give type is versioned.

While Runtime clients can use the overloaded form of the method, classic COM clients will have to access the decorated methods. Object browsers would display all forms of the decorated method along with the method signature so that developers can choose the correct form of the method.

The late bound client can also call IDispatch::GetIdsOfNames() passing in the decorated name to get the dispid of any overloaded method.

Property and Field conversions

Managed types (classes and interfaces) can also have properties and fields. A managed property is of a specific data type that may have an associated get method and set method. The get and set methods are defined separately just like any other method. For example, the following code shows an interface containing a Height property. Classes that implement the interface are required to provide a get and set method for the property.

A managed field is simply a data member of a managed class. Fields can only appear on classes. In Managed C++ for example:

__managed __interface IMammal {
public:
   __property IMammal *Mother;
   __property IMammal *Father;
   __property int Height;
   __property int Weight;
};

__managed class Human : public IMammal {
   public:
      int Age;
      __property IMammal *Mother {get{…;};set{…};};
      __property IMammal *Father {get{…;};set{…};};
      __property int Height {get{…;};set{…};};
      __property int Weight {get{…;};set{…};};
};

When exporting managed properties to a type library, the property set method becomes a [propput], and the get method becomes a [propget]. The name of the property is the same as the managed property name.

If the property type is a class or interface, the property set method becomes a [propputref], and the parameters are given an added level of indirection.

If the property has no get or the set method, the property is omitted from the type library.

Managed fields are also exported to the type library. The runtime marshaling service automatically generates the get and set methods for all public fields. Fields are always have both a get and a set function.

interface IMammal : IDispatch {
        [propget]    HRESULT Mother([out, retval] IMammal** pRetVal);
        [propputref] HRESULT Mother([in] IMammal* pRetVal);
        [propget]    HRESULT Father([out, retval] IMammal** pRetVal);
        [propputref] HRESULT Father([in] IMammal* pRetVal);
        [propget]    HRESULT Height([out, retval] long* pRetVal);
        [propput]    HRESULT Height([in] long pRetVal);
        [propget]    HRESULT Weight([out, retval] long* pRetVal);
        [propput]    HRESULT Weight([in] long pRetVal);
        [propget]    HRESULT Age([out, retval] long* pRetVal);
        [propput]    HRESULT Age([in] long pRetVal);
};

Event Conversions

Events that are sourced by managed types can be sinked by unmanaged types. Managed types implement events using a delegate base event model. For example, the following Button class implemented in C# sources the Click and the Resize event:

delegate void ClickDelegate(int x, int y);
delegate int ResizeDelegate ();

public class Button
{
   public event ClickDelegate Click;
   public event ResizeDelegate Resize;
   public void OnClick(int x, int y) { if (Click != null) Click(x, y); }
   public int OnResize() { if (Resize != null) return Resize() else return 0; }
   public void Init() {;}
}

When exported to a type library, the exporter makes the following conversions:

  • An interface and is created for each public delegate type defined. The interface has the same name as the delegate with a prepended underscore. Since all managed delegates extend the type System.Delegate, the interface contains all the public methods of the Delegate class along with an additional member called Invoke that has the same signature as the managed delegate. For example, the Invoke member for the ClickDelegate looks like this:
    class _ClickDelegate : IDispatch {
   …
   HRESULT Invoke( [in] long x, [in] long y);
}

  • If the events members of the event source are public, an add_Event and remove_Event member is added to the class interface of the class that sources the events.  In the example above, the class interface for Button (called _Button) contain the method add_Click, remove_Click, add_Resize and remove_Resize.  Both the add and remove methods take a single Delegate type argument.   For example:

    class _Button : IDispatch {
   …
   add_Click(_ClickDelegate *handler)
   remove_Click(_ClickDelegate *handler)
   …
}

    •

The complete listing of the metadata produced for the Button class is shown below:

[uuid(…), …]
interface _ClickDelegate : IDispatch {
   // all member of type _Object
   // all members of type _Delegate
    [id(0x6002000b)] HRESULT Invoke([in] long x, [in] long y);
};

[uuid(…), …]
interface _ResizeDelegate : IDispatch {
   // all member of type _Object
   // all members of type _Delegate
    [id(0x6002000b)] HRESULT Invoke([out, retval] long* pRetVal);
};

[uuid(…), …]
interface _Button : IDispatch {
   // all member of type _Object
    [id(0x60020004)] HRESULT add_Click([in] _ClickDelegate* handler);
    [id(0x60020005)] HRESULT remove_Click([in] _ClickDelegate* handler);
    [id(0x60020006)] HRESULT add_Resize([in] _ResizeDelegate* handler);
    [id(0x60020007)] HRESULT remove_Resize([in] _ResizeDelegate* handler);
    [id(0x6002000a)] HRESULT Init();
};

[uuid(…), …]
coclass Button {
   [default] interface _Button;
   interface _Object;
};

Unmanaged applications that want to sink events that originate in managed code can implement the delegate interfaces defined by the exporter and pass them to the managed objects add and remove methods in order to register as an event sink.

Unfortunately, managed events cannot sink with COM connection points using the metadata shown above.  In order to use COM connection points, and additional event interface must be defined and identified as a source interface with the ComSourceInterfacesAttribute as shown below.

using System.Runtime.InteropServices
using “ButtonEventsLib”;  // definition of the ButtonEvents interface

delegate void ClickDelegate(int x, int y);
delegate int ResizeDelegate ();

[ComSourceInterfaces(“ButtonEventsLib.ButtonEvents”)]
public class Button
{
   public event ClickDelegate Click;
   public event ResizeDelegate Resize;
   public void OnClick(int x, int y) { Click(x, y); }
   public int OnResize() { return Resize(); }
   public void Init() {;}
}

When an event source interface is provided, the exported type library will list the interface as a source interface for the coclass that sources the events.   The event interface is typically defined in a COM type library.  In that case, the generated type library will also import the type library containing the definition of the source interface (see example below).

importlib("ButtonEvents.tlb");

[uuid(…), …]
coclass Button {
   [default] interface _Button;
   [default, source] interface ButtonEvents;
   interface _Object;
};

By providing an event source interface, unmanaged clients can more easily sink to the events sourced by managed classes with COM connection points.   For details on this technique, see the section 4.9.10 below.

COM Visibility of members

Parameter and Field level conversion

In the process of converting methods, properties and fields, various data type conversions take place.  For example, the managed String field may be converted to a BSTR field when exported to COM.  For details on these type conversions see Appendix A.


Registration

In order to be able to create managed types from COM applications the appropriate entries must be made in the Windows registry. The runtime ships with a registration utility called RegAsm for just this purpose. RegAsm reads the metadata within an assembly and adds the necessary entries to the registry so classic COM clients can create the NGWS classes transparently.

Once a class is registered, any classic COM client can use it as though the class were a COM class. The registration process is only done one time when the assembly is installed. The classes within the assembly are not COM creatable until they are actually registered.

Creatable classes

Not all classes within an assembly are creatable from COM. In fact; an assembly does not need to have any creatable classes. Managed classes are considered co-creatable if the class is public and also has public default constructor (the default constructor has no arguments). Only creatable classes are registered when the RegAsm tool is run on an assembly. Public classes without a public default constructor are visible but not creatable. Abstract classes are also not creatable.

The ability to create a type from COM can be overridden with the ComRegisterAttribute. The attribute can be applied to an individual class or to an entire assembly to control whether creatable types within the assembly are registered. To allow the creatable types to be registered, set the attribute to true. To prevent creatable type from being registered, set the attribute to false. Setting the attribute to false on the assembly prevents the registrations of all creatable types within the assembly. Setting the attribute to false on a specific class prevents the registration of that specific class. Settings applied to an individual class override any assembly settings. Therefore, setting the attribute to false on the assembly and true on an individual class will cause only that class to be registered. Only classes that are COM creatable can be registered. The attribute cannot be used to make an otherwise non-creatable class creatable.

Having creatable classes has several implications:

  1. Creatable classes must be registered before the class can actually be created.
  2. Running the RegAsm utility on a NGWS runtime assembly will cause all classes with public default constructors to registered.
  3. Once a class is registered it can be created with the COM CoCreateInstance API.
  4. If a class does not have an explicit CLSID when it is registered, one will be generated. The CLSID generated will be the same as the one generated by TlbExp

Under some circumstances, classes that are not COM creatable may still be creatable from managed code. For example, a class may not be COM creatable because it has no default constructor but there be another constructor that would make it possible for another managed class to create the type. Under those circumstances, it’s possible for one type to return an instance of another non-creatable type as a return value or an out parameter provided that the non-creatable type is COM visible (see section 4.1.4.4.5).

The RegAsm utility

RegAsm has the following command line syntax:

RegAsm AssemblyFile [/u] [/reg: RegFile] [/silent][/tlb: TlbFile]

The RegAsm utility registers or un-registers the creatable classes within the NGWS runtime assembly named AssemblyFile. RegAsm can also be used to generate a .reg file with the needed entries instead of actually making the changes to the registry.

Options:

/u - Causes RegAsm to unregister the public classes found in AssemblyFile as opposed to registering them. Omitting this option causes the public classes within the assembly to be registered.

/silent – Suppress all a console output.

/reg: RegFile - Generates a .reg file named RegFile for the assembly containing the registry entries that would have been added if this option had not been supplied. The registry is not changed when this option is used.

The .reg file can be used to update the registry on any machine by importing the file with the RegEdit tool. Note: The .reg does not contain any registry updates that may be made by any user defined register functions (see section 4.2.3 below).

/tlb: TlbFile - Generates and registers a type library with the name TlbFile describing the types found in the assembly. The type library can be used to provide type information to development tools like Visual Studio. Using the /tlb option has the same effect as running TlbExp then RegAsm.

Examples

RegAsm mytest.dll

Registers all public classes contained in mytest.dll.

RegAsm mytest.dll /reg: mytest.reg

Generates the file MyReg.reg that contains all the entries that would have been made to the registry if the /reg option had not been supplied. The registry is not updated.

RegAsm mytest.dll /tlb: mytest.tlb

Registers all public classes contained in mytest.dll. Generates and registers the type library mytest.tlb that contains definitions of all the public types defined in mytest.dll.

RegAsm mytest.dll /tlb: mytest.tlb /reg mytest.reg

Generates the type library mytest.tlb that contains definitions of all the public types defined in mytest.dll. Also generates the file MyReg.reg that contains all the entries that would have been made to the registry if the /reg option had not been supplied. The registry is not updated.

Registry entries

The RegAsm tool updates the registry with the following entries:

For each public, creatable class defined in the assembly the following entries are made

HKEY_CLASSES_ROOT\progid\CLSID
   {default} = {clsid of class}

HKEY_CLASSES_ROOT\{clsid}\InProcServer32
   {default} = mscoree.dll
   Class = name of class
   ThreadingModel = Both
   Assembly = Stringized assembly reference

HKEY_CLASSES_ROOT\{clsid}\ProgId
   (default) = name of class

HKEY_CLASSES_ROOT\{clsid}\TypeLib
   (default) = {tlbid}

If the /tlb is used, a type library is generated and the following additional entries are created in the registry.

HKEY_CLASSES_ROOT\Typelib\{tlbid}\x.y\   lcid\win32
   (default) = location of type library file

HKEY_CLASSES_ROOT\Typelib\{tlbid}\x.y\FLAGS
   (default) = 0

Where:

progid is the programmatic identifier assigned to the class.

clsid is the guid assigned to the class.

tlbid is the guid that identifies the type library.

x.y is the major and minor versions of the type library.

lcid is the locale id of the type library.

See section 4.1.4.3 above for details on how the progid, clsid and tlbid are generated.

The stringized assembly reference found under the InProcServe32 key is the string form of the name of the assembly containing the class. The string is obtained from AssemblyName.NameAsString and is used to locate and load the assembly when the class is being created. An example of a stringized assembly name is “FunnyFarm.BarnYard.Mammal, Ver=1.2.5.1”

Register/Unregister Functions

Any COM creatable class (as defined above) can also have a registration and un-registration functions. The register function is called during the registration process so that application specific registration work can be done. The un-registration function is called when the type is unregistered. The method must be static and must take a single String parameter for the name of the registry key being updated. During registration and un-registration, the appropriate method is called with the registry key being updated (i.e., “HKEY_CLASSES_ROOT\CLSID\<clsid>”).

A class should have, at most, one register function and one unregister function. A method is identified as the register function with ComRegisterFunctionAttribute. A method is identified as the un-register function with ComUnregisterFunctionAttribute.

Register and Unregister functions are provided for backward compatibility only. Users are strongly discouraged from using these functions. In situations where they are necessary, the unregister function should be sure to “undo” whatever the register function did.


Activating NGWS Classes

The NGWS runtime provides a standard class factory implementation that is capable of creating instances of any NGWS object. The runtime DLL (mscoree.dll) has an entry point for DllGetClassObject that creates can class factories for managed types as necessary. When NGWS classes are registered with RegAsm, mscoree.dll is supplied as the InProcServer32. This makes it possible for a COM clients to get a class factory for a NGWS class in exactly the same way they would get one for a COM class.

In order to actually create an instance of the NGWS class, the runtime must first locate the assembly that contains the class implementation. The runtime uses the information placed in the registry by RegAsm to locate the assembly that contains the NGWS class.

Given the CLSID passed to DllGetClassObject, the runtime opens the CLSID key in the registry and reads the class value from the key to determine the name of the class being requested. The runtime first checks with the loader to determine if an implementation of the named class is already loaded. If so, the runtime returns a reference to the object’s existing class factory to the caller of DllGetClassObject.

If a class with the specified name is not already loaded, the runtime reads the Assembly value from the registry and passes in on to the runtimes assembly resolver. The assembly resolver attempts to locate the assembly based on the assembly name. In order for the assembly resolver to locate the assembly, the assembly must be signed and installed in the runtimes assembly cache or it must be found along the application path. Assemblies loaded from the application path are only accessible from that application. Shared assemblies must be signed and installed in the assembly cache.

For complete details on how managed assemblies are located, see the spec on How the NGWS runtime Locates Assemblies

Once an assembly is located, the runtime loads the assembly and tries to locate the class’s metadata within the assembly. If the assembly contains a description of the class being requested, the runtime return a class factory object for the given class from DllGetClassObject.

After getting a class factory for a NGWS class from the runtime, the final step in activating an object is to call IClassFactory::CreateInstance. The same class factory object can be used to create any number of instances of the same class but each type of class requires a different class factory.


COM Callable Wrappers (CCW)

During the activation process, the runtime instantiates the object as well as a COM callable wrapper (CCW) for the object. The object is allocated from the garbage-collected heap while the CCW is allocated from a non-collected heap. This ensures that the runtime is free to move the object around in memory as necessary. The wrapper holds a single reference to the object in order to keep the object alive as long as the wrapper is in use. Since the wrapper is allocated in non-collected memory, the runtime is free to hand out interface pointers on the wrapper to the unmanaged client. The wrapper is reference counted like any other COM object. When the reference count reaches zero, the wrapper releases its sole reference on the object and the object is freed during the next garbage collection.

The CCW has the following responsibilities:

  • Preserving object identity
  • Maintaining object lifetime
  • Proxying custom interfaces
  • Marshaling method calls
  • Synthesizing selected interfaces


Preserving Object Identity

Activation is not the only way wrappers can be created. A CCW is also created as a side effect when one managed object returns a reference to another managed object from a method call (either as a return value or as an out parameter on a method call). In order to honor COM’s identity rules, it’s important that only one wrapper ever exists for a given managed object. This ensures that all interfaces on the same object have a common IUnknown. Therefore, when a CCW is needed, the runtime always checks for an existing wrapper on the object before creating a new one. This ensures that no more than one wrapper will ever exist for a given managed object.


Maintaining Managed Object lifetime

Each COM interface that is returned from the CCW is reference counted in the usual manner. The CCW in turn maintains a traced reference the managed object. That way the CCW can ensure that that the underlying managed object remains alive as long as unmanaged references exist on the wrapper. It also allows other managed objects to maintain a reference to the same managed object as the CCW.

However, unlike COM objects, managed object do not have deterministic lifetimes. That means that even though all reference to a CCW have been released, it may be some time before the managed object it wraps is actually freed. COM clients that make assumptions about when the objects they reference will be delete may experience unexpected behavior.

Process Shut down

Under some circumstance, leaks can also occur when an unmanaged process is shut down. Image an unmanaged process that holds a reference on a managed object implemented in another DLL. A CCW is created to wrap the managed object being referenced. Once the reference is released, the CCW becomes eligible for collection. If the process is shut down before the collection takes place, the object will leak. While you might think that the runtime could perform a collection before it shuts down (within DLLMain for example), there are several reasons why that isn’t possible.

  1. Performing a garbage collection during DllMain execution would surely result in a deadlock as object attempt to acquire the loader lock.
  2. The order in which DLL’s are notified of shut down is unpredictable and therefore, during shut down, objects may inadvertently attempt to execute code in other DLLs that have already been unloaded.

To ensure a leak-free shut down under these circumstances the unmanaged process must explicitly call CoEEShutDown. By explicitly calling CoEEShutDown, the runtime is given an opportunity to perform one last collection before the process is destroyed. However, CoEEShutDown collects all outstanding objects in any order. There can be no assumptions about the order in which objects are destroyed.

To ensure a leak-free shut down under these circumstances the unmanaged process must be sure to call CoUninitialize(). By explicitly calling CoUninitialize(), the runtime is given an opportunity to perform one last collection before the process is destroyed. However, CoUninitialize() collects all outstanding objects in any order. There can be no assumptions about the order in which objects are destroyed. This technique applies only on Win2000.


Proxying Custom COM Interfaces

Wrappers are created in order to expose a COM interface on a managed class. The COM interface that is exposed from the wrapper is really just a proxy for the actual interface that the object implements. The interfaces generated by the wrapper are always generated dynamically. As interfaces are requested, the wrapper uses the class metadata to determine if the class implements the requested interface. If it does, the wrapper creates a new vtable for the interface and populates the vtable with pointers to stub functions that delegate the calls to the real methods on the object. These tear-off interfaces are only created if, and when, they are requested. This reduces the object overhead considerably for objects that implement many interfaces as the vtable for the interfaces are only allocated if the interface is used.

At any given time, a CCW may be proxying several different interfaces on the same managed object. The CCW handles the details of generating the vtable and stubs as necessary.

All interfaces explicitly implemented by a NGWS class can be exposed through the CCW as a vtable-based COM interface. For backward compatibility with classic COM, the actual layout of the vtable for the COM interface can be controlled through the class metadata although most COM interfaces would typically have the same layout as the managed interface.

As mentioned, the function pointers in the vtable exposed by the CCW don’t actually point to the managed implementation of the method. Instead they point to stubs that marshal the call to the correct managed method. The stubs are responsible for the marshaling the call from the unmanaged code to managed code.


Marshaling Method Calls

As described in section 4.1.4.5, all methods have a managed and an unmanaged method signature. The CCW is then responsible for marshaling method calls from the unmanaged signature to the managed signature. It’s the stub that acts as the facilitator between these two programming models. The stub performs 4 important tasks during marshaling:

  1. It manages the code transition
  2. It catches exceptions and returns HResults
  3. It packs the return values into an out-parameter
  4. It marshals data types from one format to another

Managing Code Transitions

Handling Managed Exceptions

COM interfaces return HResults to convey error information to the caller. NGWS runtime introduces the ability to throw exceptions across object interfaces but only from one managed object to another. When classic COM clients are using NGWS objects, the stub must to maintain HResult semantics that these clients are expecting. Therefore, exceptions that occur during calls made by COM clients must be mapped to the equivalent COM HResults and returned to the COM client.

For example, given the following description of the IFoo interface:

interface IFoo : IDispatch
{
   HRESULT StartWork();
   HRESULT GetCount([out, retval] long *lCount)
}

A NGWS developer may implement the interface as follows

class CFoo : public IFoo
{
   void StartWork()
   {
      if (…) throw RuntimeException
   }

   long GetCount()
   {
      if (…) throw RuntimeExceptions
   }
}

In the NGWS runtime, the methods that return HResults are replaced by methods that throw NGWS runtime exceptions. In this case, if a NGWS runtime client uses the CFoo class, it can catch the exceptions thrown in CFoo::StartWork and CFoo::GetCount in the usual way. But, when the same class is used by a classic COM client, the stub catches the exception thrown by the class and returns an HResult to the caller. If the method executes without throwing an exception, the stub returns S_OK.

In order to do this, the runtime needs a mapping between Exceptions and HResults. Each NGWS runtime exception class defined in the NGWS frameworks has a static member called ErrorCode for storing a classic COM HResult. The class library exception classes are automatically initialized with the proper HResult. Some NGWS runtime exceptions map to existing HResults. For example, OutOfMemoryException maps to the HResult COR_E_OUTOFMEMORY, which is equivalent to E_OUTOFMEMORY. Most NGWS runtime Exceptions map to new HResult values.

User defined classes that extend System.Exception must also set the static ErrorCode field during construction. The ErrorCode field of the base exception class called Exception is initialized to E_FAIL. If the HResult field is not reinitialized in the sub class then the E_FAIL HResult is used as the default.

New exception classes can be created as required by application developers. Recoverable exceptions that are application related should be derived from the ApplicationException class. New exception class can be mapped to an existing HRESULT by supplying the HRESULT value in the exception’s constructor. For example, a new exception class called NoAccessException can be created and mapped to the HRESULT E_ACCESSDENIED with the following code:

Class NoAccessException : public ApplicationException
{
   ErrorCode = E_ACCESSDENIED;
}

CMyClass::MethodThatThrows
{
   NoAccessException *e;

   e = new NoAccessException();

   throw e;
}

Additional information about the exception is provided to the client through the IErrorInfo interface that it also exposed by the wrapper (See section 4.9.5 below).

Success Results - COM also supports the notion of success HResults that can be returned from any COM method to convey information to the caller rather than indicate and error condition. Examples include S_FALSE and DB_S_ENDOFROWSET. The mechanism described above (mapping exceptions to HResults) does not support success HResults mainly because the runtime does not advocate the use of exceptions for reporting non-exceptional situations (like a FalseException or EndOfRowsetException). None exceptional information should instead be reported to NGWS runtime client through a return value or an out parameter.

Converting Return Values

Return values from the NGWS runtime methods are handled through [out, retval] parameters added to the COM signature of the method. A NGWS runtime method that is defined with a void return simply has no [out, retval] parameter. A NGWS runtime method with an int return type would have an [out, retval] int * parameter added to the COM method signature.

Return values may also need to be converted from a NGWS runtime type to a COM type. For example, the following method returns a System.String that is converted to a BSTR in the COM Signature.

NGWS runtime signature:

System.String GetName();

COM signature:

HRESULT GetName([out, retval] BSTR *rv);

For information on type conversions see Error! Reference source not found. below.

Marshaling Data Types

As the SetDate example below shows, type conversion is also applied to method parameters. For the most part, integer and floating point data types are marshaled without conversion but other types like string, and date are converted to the NGWS classes that represent those types.

NGWS runtime signature:

void SetDate(System.DateTime d);

COM signature:

HRESULT SetDate(DATE d);

For complete information on type conversions see section Error! Reference source not found. below.


Synthesizing Selected Interfaces

In addition to the interfaces that are explicitly implemented by a NGWS class, the CCW can synthesize an implementation of the interfaces listed below on demand. The implementations of these interfaces are based on the metadata provided by the class and are completely implemented by the CCW on behalf of the object. While it is possible to override or completely replace the runtimes implementation of any interface, a NGWS class is never required to implement any of the following interfaces.

The IUnknown Interface

Even though NGWS objects are garbage collected, a standard implementation of IUnknown is provided to the classic COM client so that it can continue to use the AddRef, Release and QueryInterface. The reference counting is done on the CCW, which, in turn maintains a traced reference to the NGWS object. The NGWS runtime object is not reference counted directly

There are issues relating to determinism lifetime with this approach. The NGWS class is not destroyed until it is actually garbage collected. This differs from the classic COM model where objects are destroyed immediately upon the final call to Release.

As already mentioned, the wrapper’s implementation of QueryInterface creates tear-off for any interfaces explicitly implemented by the class as the client requests them. Given a class Foo that implements interfaces IFoo1 and IFoo2 and extends class Bar that implements IBar1, QueryInterface will respond to requests for IFoo, IFoo2 and IBar with interfaces that delegate calls to the classes implementation. In addition, the CCW will also synthesize a vtable and implementation of any of the interfaces shown Table 1 on demand. The sections that follow describe each of these interfaces in more detail.

Table 1. Synthesized Interfaces Table

Interface ID | Interface Description
_Foo The class interface for Foo | The class interface for Foo. Contains the methods of the class Foo, the class Bar, and of the NGWS frameworks class Object since all NGWS classes extend object. In addition, this interface has all methods of IFoo1, IFoo2 and IBar1.
_Bar The class interface for Bar | The class interface for Bar. Contains the methods of the class Bar, and of the NGWS frameworks class Object since all NGWS classes extend object. In addition, this interface has all methods of IBar1.
_Object The class interface for Object | The class interface for Object. Contains the methods of the class Object. This interface is defined in runtime class library
IDispatch | A standard dispinterface that contains all the same methods of _Foo.
IProvideClassInfo | A standard IProvideClassInfo interface that provides the CLSID of Foo
ISupportErrorInfo |
IErrorInfo | A standard interface for conveying detailed error information from callee to caller
IDispatchEx | A standard interface that allows callers to dynamically add and remove methods from a dispinterface among other things.
IConnectionPointContainer IConnectionPoint | A standard interface for dealing with events
IObjectSafety | A standard interface that allows caller to determine if a particular object is safe for scripting with.

The wrapper may, for some NGWS classes, also provide implementation of the following interfaces:

Interface ID | Interface Description
IEnumVariant: | For collection type classes, an interface for enumerating the objects in the collection.
IPersistStream: | For persistent objects an interface for reading and writing the object to a stream.

IDispatch Interface

The runtime often needs to provide an implementation of the IDispatch interface for COM clients to use. This can either be in response to a direct request for an IDispatch (through QueryInterface) or by calling through the IDispatch methods in a dual interface.

There are two separate IDispatch implementations that the runtime can use.

  • Ole Automation compatible Implementation
  • Internal Runtime Implementation

For the highest level of backward compatibility, the Ole Automation compatible implementation should be used. This is the same IDispatch implementation that earlier versions of Visual Basic used and it is the one that most COM applications are designed to use (see Appendix F for known compatibility issues). The downside of using this implementation is that it requires static type information (even for managed types). The runtime can, and will, automatically generate the type information for managed types as necessary, but the process is time consuming. See the section below on on-the-fly typelib generation.

As an alternative, the runtime provides its own internal implementation of IDispatch. This implementation does not require static type information unless it is specifically asked for by calling IDispatch.GetTypeInfo. Other methods in the IDispatch interface can be called without requiring any form of static type information. If IDispatch.GetTypeInfo or IDispatch.GetTypeInfoCount is called, the same on-the-fly typelib generation process described below is used.

The IDispatch implementation can be controlled with the custom attribute System.InteropServices.IDispatchImplAttribute. The attribute can be set at the assembly level or on an individual class. When applied to an individual class, the attribute overrides any assembly level setting for that class. If the attribute is missing, the internal implementation is used. For a description of the differences between the Internal and Ole Automation implementation see Error! Reference source not found..

Individual interfaces can also use the System.InteropServices.InterfaceTypeAttribute to control whether the interface is exposed to COM as IUnknown derived or IDispatch derived. The table below shows how the interfaces InterfaceTypeAttribute attribute and the classes IDispatchImplAttribute attributes can be combined to create the desired effect.

InterfaceTypeAttribute on Interface | IDispatchImplAttribute on Class | Effect on Interface
ComInterfaceType.InterfaceIsDual or ComInterfaceType.InterfaceIsIDispatch | IDispatchImplType.CompatibleImpl | Dual interface using compatible implementation.
ComInterfaceType.InterfaceIsDual or ComInterfaceType.InterfaceIsIDispatch | IDispatchImplType.InternalImpl | Dual interface using internal implementation.
ComInterfaceType.InterfaceIsDual or ComInterfaceType.InterfaceIsIDispatch | IDispatchImplType.SystemDefinedImpl or Missing | Dual interface using internal implementation.
ComInterfaceType.InterfaceIsUnknown | IDispatchImplType.CompatibleImpl | IUnknown derived interface
ComInterfaceType.InterfaceIsUnknown | IDispatchImplType.InternalImpl | IUnknown derived interface
ComInterfaceType.InterfaceIsUnknown | IDispatchImplType.SystemDefinedImpl or Missing | IUnknown derived interface.
Missing | IDispatchImplType.SystemDefinedImpl or Missing | Dual interface using internal implementation.

Responding to Queries for IDispatch

The attribute NoIDispatchAttribute is used to control how the class responds to queries for IID_IDispatch. By default all managed classes implement the IDispatch interface because the runtime can provide an implementation of the interface on behalf of the class. This behavior can be overridden with then NoIDispatchAttribute attribute. Attributing a class with this attribute causes the class to always respond to such queries for IID_IDispatch with E_NOINTERFACE.

This attribute has no effect on other interfaces that are dual (like the class interface). It only effects how the class responds to QueryInterface. Clients may still use the dispatch portion of other interfaces to late bind to the class. The class interface in particular is always dual and would still be available if a client queried for that particular interface.

On-The-Fly Type Library Generation

The downside of exposing IDispatch is that, depending on how the interface is used; static type information may be needed. For example, using the Ole Automation compatible IDispatch always requires COM type library for the managed type that’s being exposed. The internal implementation of IDispatch also requires a type library but only if the IDispatch.GetTypeInfo or IDispatch.GetTypeInfoCount methods are called.

The type information can be generated in advance and packaged with the application or it can be generated on the fly by the runtime when the IDispatch is utilized. There are several things to note about this automatic type library generation:

  1. Type libraries are only generated if the client of a managed class actually uses the IDispatch methods of an interface. If the client never calls the IDispatch methods, the type library is not be generated.

If a type library is needed, the runtime will first check for an existing type library for the given type. The typelib can be located in 3 ways:

  • If a registered type library is found, it will use the registered library rather than generating a new one. The type library must be registered in order for the runtime to locate it.
  • If the manage DLL contains a type library stored as a resource, the type library will be extracted from the DLL.
  • If a file with a .tlb extension is found in the same location as the managed assembly, that type library will be used.

If the runtime does need to generate a type library on the fly, the type library will be generated in its entirety. Partial type libraries cannot be generated. In other words, if type information is needed for one type that is packaged in a assembly along with 99 other types, type information is generated for all 100 types at that time. The type library is then registered so that it can be used again later.

Generated type libraries are named with the same name as the managed assembly but with a .tlb extension.

To avoid type library generation all together, there are three things you must do when defining new types:

  • Use the NoIDispatchAttribute an all classes in the assembly to ensure that the class does not respond to queries for IID_IDispatch.
  • Set the InterfaceTypeAttribute attribute to COMInterfaceTypeFlag.InterfaceIsUnknown on all interfaces exposed in the assembly to ensure that the interfaces are not IDispatch derived.
  • Use the HasDefaultInterface attribute to specify an explicit interface as the default. Doing so will suppress the automatic generation of the class interface. The interface should have the InterfaceTypeAttribute attribute set to COMInterfaceTypeFlag.InterfaceIsUnknown.

The Class Interface

In order for COM client to be able to access the members of a managed class, a new interface is defined when a class is exported to a type library with TlbExp. This interface is commonly referred to as the “class interface” and carries the same name as the class itself but is prefixed with an underscore. The class interface contains all the members of the managed class so that unmanaged users have access to the class members directly. The interface is automatically generated by TlbExp and is exposed to COM in addition to any other interfaces that the class implements explicitly. The class interface is always dual so COM clients can either early bind or late bind to the interface.

Managed clients never use (or even know about) the class interface. Instead, managed clients can simply access the class methods directly.

The example below shows the class interface for a simple managed class called Mammal.

In Managed C++:

__managed class Mammal
{
public:
   short Weight;
   Mammal (){   };
   __property short Height{
      get {return m_height;};
      set {m_height = value;};
   };
   void  Eat(){;};
   void  Breathe(){;};
   void  Sleep(){;};

protected:
   short m_height;
private:
   void Walk(){;};
};

In the COM type library produced by TlbExp:

    [odl,
    uuid(890A3E4A-B640-36A4-9F67-194D4B5BDA8A),
     hidden, dual,
     nonextensible,
     oleautomation ]
    interface _Mammal : IDispatch {
        [id(0x60020000)] HRESULT GetType([out, retval] _Type** pRetVal);
        [id(0x00000000)] HRESULT ToString([out, retval] BSTR* pRetVal);
        [id(0x60020002)] HRESULT Equals([in] IUnknown* obj,
                        [out, retval] VARIANT_BOOL* pRetVal);
        [id(0x60020003)] HRESULT GetHashCode([out, retval] long* pRetVal);
        [id(0x60020004)] HRESULT Eat();
        [id(0x60020005)] HRESULT Breathe();
        [id(0x60020006)] HRESULT Sleep();
        [id(0x60020007), propget] HRESULT Weight([out, retval] short* pRetVal);
        [id(0x60020007), propput] HRESULT Weight([in] short pRetVal);
        [id(0x60020008), propget] HRESULT Height([out, retval] short* pRetVal);
        [id(0x60020008), propput] HRESULT Height([in] short pRetVal);
};

    [
      uuid(C77C754F-EAC8-3092-82F3-3EDC33A77755)
    ]
    coclass Mammal {
        [default] interface _Mammal;
        interface _Object;
   };

There are several things to notice about the class interface:

  • It has the same name as the class but is prepended with an underscore. Had the interface _Mammal been defined elsewhere, the class interface would be named _Mammal_2.
  • A unique IID was generated for the interface which is unrelated to the CLSID generated for the Mammal class. There is not way of explicitly setting the IID of the class interface
  • It derives from IDispatch. The class interface is always dual.
  • It has the odl, dual, hidden, nonextensible and oleautomation attributes.
  • It contains all the public members of it’s base class (in this case System.Object)
  • It does not contain the private or protected members of the class (such as Walk).
  • Each member is automatically assigned a unique dispid. The DispIds can be set explicitly with the DispIdAttribute.
  • Like other interface methods, the method signatures are transformed to return HResults and have [out, retval] parameters.
  • The properties and fields are transformed into [propget] and [propput] and [propputref] as described in section 4.1.4.5.2.

There are several things to notice about the Mammal coclass:

  • It does no contain the methods of the managed class. Coclasses can not have members.
  • The class interface is identified as its default interface.
  • A unique CLSID was generated for the class automatically.
  • It also implements the class interfaces of all its base classes (in this case _Object)

Methods

Methods on the class interface are treated the same as methods on any other interface. They undergo the same translation described in section 4.1.4.5.1 above. The class interface includes public method of the managed class as well as any methods on the managed base class. The methods can be accessed through the dispatch portion of the interface or through the v-table directly. When late binding to the interface with IDispatch, it’s also possible to call methods found on the derived class as well. For example, if the Monkey class was created that extend the class Mammal in the example above, methods of the Monkey class would be callable through the _Mammal.Invoke().

DispIds

The CCW will generate unique dispids on demand for the methods and properties of the class. The dispids will be guaranteed to always be the same for any instance of the same version of the same class. In order to explicitly set the dispid of a member to a specific value, the DispIdAttribute can be set on an individual member (see the Interop Metadata Spec). If a classic COM type library with predefined dispids is converted into NGWS runtime metadata with the type library importer, the importer will retain the dispids of the imported dispinterface within the metadata. The DispIdAttribute is set for each member that has an explicit DispId in the type library. The runtime’s IDispatch implementation will honor the DISPIDs stored as custom attribute in the objects metadata. These dispids are preserved only to ensure that the interface can be exposed to COM in a compatible way.

Special Dispids – There are some Dispids that have special meaning associated with them. For example:

DISPID_VALUE (0) - The default member for the dispinterface—that is, the property or method invoked if the object name is specified by itself without a property or a method in a controller script.

DISPID_PROPERTYPUT(-3) - Indicates the parameter that receives the value of an assignment in a property put

DISPID_NEWENUM(-4) - The _NewEnum method of a collection.

DISPID_EVALUATE (-5) - A method named Evaluate that a controller implicitly invokes when it encounters arguments in square brackets. For example, the following two lines are equivalent:

x.[A1:C1].value = 10
x.Evaluate("A1:C1").value = 10

Managed classes that need to expose methods with any of these “special” dispids should that use the DispIdAttribute to explicitly assign the correct dispid to the member. The runtime makes no association between the members and these special Dispids.

In situations where multiple members of the same type end up with the same dispid, the member defined first will respond to the assigned dispid. Members defined subsequently will essentially be inaccessible. Duplicate dispids can result from collisions with the dispids that are automatically generated by the typelib exporter (in the range of 0x6002000 –0x6003000) or by inadvertently assigning the same Dispid with the DispIdAttribute attribute. Both situations should be avoided.

EXCEPINFO

Dispinterfaces return error information in a separate structure passed in on every call to Invoke. In a late bound call, a pointer to an EXCEPINFO structure is passed in to the call to IDispatch::Invoke. When an exception is encounter, the CCW packs additional details about the exception, including the HRESULT, into the EXCEPINFO structure and returns DISP_E_EXCEPTION from the call to Invoke. In order to find out more information about the error that occur during the call to Invoke, the classic COM client checks the EXCEPINFO structure passed in on the call.

Default Interface

COM has the notion of a default interface. Since classes cannot contain members, the members of the default interface are used as the members of the class by late bound languages like Visual Basic. In the NGWS runtime there is no need for a default interface because classes themselves can have members. However, when exposing managed classes to COM, they are much easier to use if they have a default interface.

By default, the runtime creates a class interface for each class and marks that interface as the default when generating type information for the class. This process can be overridden with the HasDefaultInterfaceAttribute attribute.

If the HasDefaultInterfaceAttribute attribute is specified for a class, the normal class interface is not created. Instead the first implemented interface supplied when the type was created is assumed to be the default. That interface will be marked as the default when the type information for the class is generated. For example, the following class does not have an explicit default interface.

interface IFoo {…}

public class Foo : public IFoo {
…
}

And therefore would a class interface would be generated when the class is exported and marked as the default interface.

interface IFoo {…}
interface _Foo {…}
coclass Foo {
   [default] interface _Foo;
   interface IFoo;
}

By specifying an explicit default interface with the HasDefaultInterfaceAttribute the class interface generation will be omitted:

interface IFoo {…}

[HasDefaultInterface] public class Foo : public IFoo {
…
}

The class would be exported as:

interface IFoo {…}
coclass Foo {
   [default] interface IFoo;
}

Default interfaces are defined separately at each level of an objects inheritance hierarchy. In other words, an object with an explicit default interface may derive from an object that uses the class interface as its default. In that case each object has a distinct default interface.

Default interface are not required to be dual interface when exposed to COM.

IEnumVariant

A custom marshaler is provided by the runtime that provides a bridge between the NGWS frameworks interface IEnumerator and the COM interface IEnumVariant. The custom marshaler causes method parameters and properties of type IEnumerator to be exposed to COM as IEnumVariant interface. So for example, the GetEnumerator method of the System.Collections.SortedList class whose managed signature is:

IEnumerator GetEnumerator(bool allowRemove);

would have the following signature when exported to a COM type library.

HResult GetEnumerator(bool allowRemove, IEnumVariant** ppEnum);

The marshaler automatically delegates the calls from the COM interface to the NGWS runtime interface with the following behavior:

HResult IEnumVariant.Next(ulong celt, Variant *rgVar, ulong, *pCeltFetched)
Calls IEnumerator.MoveNext and IEnumerator.Current celt times and marshals each object returned from GetObject into an element of the rgVar array. Sets pceltFetched to the number of elements loaded in rgVar. Return S_OK if celt elements were copied into rgVar otherwise returns S_FALSE.
HResult IEnumVariant.Skip(ulong celt)
Calls IEnumerator.MoveNext() celt times. Return S_OK if celt elements were skipped otherwise returns S_FALSE.
HResult IEnumVariant.Reset()
Calls IEnumerator.Reset()
HResult IEnumVariant.Clone(IEnumVARIANT **ppEnum)
Attempts to case the enumerator to the ICloneable interface. If successful calls IClonable.Clone to create a new enumerator which is returned in ppEnums

Locating Enumerators

In COM, dispinterfaces that provide enumerators do so through a property with a DISPID of -4. There are no rules defining the name or signature of this property. There is no standard way of identifying enumerators provided by interfaces that are not dispinterfaces. For example, in the following type library, the interface _Collection provides and enumerator through its _NewEnum member:

interface _Collection : IDispatch
{
   [id(0xfffffffc)] HRESULT _NewEnum([out, retval] IUnknown** ppvObj);
}

In the NGWS runtime, classes or interfaces that provide enumerators do so by implementing the IEnumerable interface. The recommended way of determining whether a class provides an enumerator is to cast the type to IEnumerable. If the cast succeeds, the enumerator is provided by the GetEnumerator method on the interface.

In order for a managed class to expose an enumerator to COM clients it must still expose a method with a dispid of 0xfffffffc. The Enumerator method on the IEnumerable interface will always have the proper DISPID of 0xfffffffc. However, DISPID’s on interfaces are not propagated to the implementation of the member on the class. The author of the class is therefore responsible for explicitly setting the DISPID on the method that returns the enumerator to the proper value of 0xfffffffc.

Consider the following example. The interface Fields extends the interface IEnumerable. The Collection class then implements the Fields interface.

In Managed C++

public interface Fields : public System.Collections.IEnumerable {
public void SomeOtherMethod();
}

public class Collection : public Fields {
   public IEnumerator GetEnumerator();
public void SomeOtherMethod();
}

When exported to a type library:

interface Fields : public IDispatch {
   [id(-4),  propget] HResult GetEnumerator([out retval] Unknown**ppv);
   HResult SomeOtherMethod();
}

interface _Collection : public IDispatch {
   [id(-4),  propget] HResult GetEnumerator([out retval] Unknown**ppv);
   HResult SomeOtherMethod();
}

coclass Collection {
   interface _Collection;
   interface Fields;
}

ISupportErrorInfo/IErrorInfo

Classic COM clients that use interfaces other than IDispatch typically use the ISupportErrorInfo and IErrorInfo interfaces in combination for finding out the details of an exception. When an errant HRESULT is returned from a method call on an interface, the client can query for an ISupportErrorInfo interface on the object. ISupportErrorInfo is used to determine if the current object supports the IErrorInfo interface. If the object does support the IErrorInfo interface, it returns S_OK from ISupportErrorInfo::InterfaceSupportsErrorInfo. The S_OK return value indicates that the client can reliably call the GetErrorInfo API to obtain an IErrorInfo interface on the latest exception object. ISupportErrorInfo::InterfaceSupportsErrorInfo only needs to be called once for the lifetime of the object. The classic COM client will then use the methods of the IErrorInfo interface to get additional details about the nature of the error. Through the IErrorInfo interface, a textual description of the error can be obtained as well as the source or the error, help file, help context, and GUID of class that generated the error.

interface IErrorInfo: IUnknown {
   HRESULT GetGUID([out] GUID *pGUID);
   HRESULT GetSource([out] BSTR *pBstrSource);
   HRESULT GetDescription([out] BSTR *pBstrDescription);
   HRESULT GetHelpFile([out] BSTR *pBstrHelpFile);
   HRESULT GetHelpContext([out] DWORD pdwHelpContext);
}

Within the COM callable wrapper, NGWS runtime exceptions thrown during any method call are caught so they can be returned to the client through IErrorInfo. The wrapper implements the ISupportErrorInfo interface and returns S_OK form any calls to ISupportErrorInfo::InterfaceSupportsErrorInfo indicating that the wrapper will call the SetErrorInfo API providing a valid IErrorInfo interface on the last thrown exception.

With the IErrorInfo interface, the client can get to all the information available about the Exception object. The following table describes how each of the IErrorInfo methods obtains it from the exception.

IErrorInfo Method | Source of Information
GetGuid | Always returns NULL.
GetSource | Exception.Source
GetDescription | Exception.Message or, if empty, the string returned from Exception.ToString().
GetHelpFile

And

GetHelpContext

 The Exception.HelpLink is parsed into the Helpfile and HelpContext As follows:

If Exception.HelpLink does contain a “#” character and, if the string to the right of the “#” character contains a valid numeric value, then GetHelpContext returns the numeric string converted to a DWORD and GetHelpFile returns the string to the left of the “#” character.

Otherwise, GetHelpFile returns the complete Exception.HelpLink and GetHelpContext returns zero. Note that Exception.HelpLink is intended to be a URL as opposed to a file name.

For example:

HelpLink: http://www.microsoft.com/msdn/22k343.htm

GetHelpFile: http://www.microsoft.com/msdn/22k343.htm

GetHelpContext: 0

HelpLink: http://www.microsoft.com/msdn/22k343.htm#top

GetHelpFile: http://www.microsoft.com/msdn/22k343.htm#top

GetHelpContext: 0

HelpLink: http://www.microsoft.com/msdn/22k343.htm#5534

GetHelpFile: http://www.microsoft.com/msdn/22k343.htm

GetHelpContext: 5534

HelpLink: c:\winnt\system32\corhelp.hlp#5534

GetHelpFile: c:\winnt\system32\corhelp.hlp

GetHelpContext: 5534

In order to get a reference to the actual exception object, the client can query through the IErrorInfo interface for the class interface of a specific exception class. For example, the client could query for the _SystemException interface or the _ArithmeticException interface. Members that are unique to a specific exception class are only available through the exceptions class interface.

IObjectSafety

All managed objects implement the IObjectSafety interface to indicate that they are safe for scripting. Container that host managed types can query the managed object for the IObjectSafety interface. The runtime implementation has the following behavior:

GetInterfaceSafetyOptions()
   Supported : INTERFACESAFE_FOR_UNTRUSTED_DATA | INTERFACESAFE_FOR_UNTRUSTED_CALLER
   Enabled : INTERFACESAFE_FOR_UNTRUSTED_DATA | INTERFACESAFE_FOR_UNTRUSTED_CALLER
SetInterfaceSafetyOptions
   Error if any other option is specified.

ITypeInfo Interface

All managed objects that are exposed to COM through a CCW provide an implementation of the ITypeInfo interface. The interface can be obtained through the Object’s IDispatch interface, through the objects IProvideClassInfo interface or by calling the GetType method on the objects class interface. The type information provided through the interface is exactly the same as the information found for that class in the type library produced by the TlbExp utility. In fact, the interface is produced by loading that exact type library. A side effect of requesting this interface is that the type library may be generated as a result of the request. Generating this type library can be time consuming and should be done sparingly as the type library is never saved and never registered. In order to avoid generating the type library, consider shipping the tlb along with you assembly.

In addition to providing type information on request as described above, the runtime also provides a custom marshaler for marshaling the System.Type class to the ITypeInfo interface. Any managed method that uses the System.Type class will automatically use the marshaler when the method is exposed to COM. The TlbExp utility will convert all references to System.Type to ITypeInfo on export.

For example, the following managed type:

interface TypeProvider {
   void SetType(System.Type t);
   void SetTypeRef(ref System.Type t);
   System.Type GetType();
}

Is converted to:

interface TypeProvider {
   HRESULT SetType([in] ITypeInfo *t);
   HRESULT SetTypeRef([in, out] ITypeInfo **t);
   HRESULT GetType([out, retval] ITypeInfo **t);
}

IProvideClassInfo Interface

The IProvideClassInfo interface is provided by the CCW for all managed objects in order to gain access to the individual ITypeInfo interface. See the section 4.9.7 for details.

IDispatchEx Interface

IConnectionPointContainer and IConnectionPoint Interface

Managed types that source events do so using the NGWS frameworks and runtime delegate based event model. The delegate model is quite different from the connection point model traditionally used by COM but the CCW is able to bridge the two models.

In order to sink managed events with COM connection points the managed class must identify an event interface. The event interface is only required if the events are to be sink from unmanaged code using connection points. The event interface can be defined in either managed code or in a COM type library (or IDL which is used to produce a type library). The code below shows the Button class along with the ButtonEvents event interface defined in IDL.

using System.Runtime.InteropServices

delegate void ClickDelegate(int x, int y);
delegate int ResizeDelegate();

[ComSourceInterfaces(“ButtonEvents”)]
public class Button
{
   public event ClickDelegate Click;
   public event ResizeDelegate Resize;
   public void OnClick(int x, int y) { Click(x, y); }
   public int OnResize() { return Resize(); }
   public void Init() {;}
}
library ButtonEventsLib
{
    [uuid(…)]
    interface ButtonEvents : IDispatch {
        [id(0x60020000)] HRESULT Click([in] int x, [in] int y);
        [id(0x60020001)] HRESULT Resize([out, retval] int *pRetVal);
    };

};

The event interface must meet the following requirements:

  • The event interface must have a method for each event the client wants to sink (which does not necessarily have to be every event). The methods must have the same name and signature as the events members for the class sourcing the events. In the example above, the ButtonEvents interface has a Click method with no return value that takes 2 integer arguments. The name and signature match the Click member of the Button class
  • The event interface must be identified as a source interface on the class sourcing the events with the ComSourceInterfaces attribute.

By identifying an event interface for a managed class, the CCW for that class will automatically implement IConnectionPointContainer interface. A connection point is provided for each source interface identified with the ComSourceInterfaces attribute. The managed object does not explicitly implement the interface.

Unmanaged clients can call IConnectionPointContainer.FindConnectionPoint passing the IID of the source interface to obtain an IConnectionPoint interface. A connection point interface can be obtained for any interface identified in the ComSourceInterfaces attribute for a given class.

Unmanaged clients wanting to sink with managed events should then provide an implementation of the event interface. As events are fired from managed code, the CCW will automatically call the corresponding methods of the event interface on any client that has connected to the connection points provided by the CCW. Clients can connect to the CCW by calling IConnectionPoint.Advise passing the unmanaged implementation of the source interface. The event interface is actually connected to the managed delegates on a per-event basis. In other words, the event interface may contain only a subset of the actual events sourced by the managed class. Events and methods are matched by name and signature when IConnectionPoint.Advise is called. If an event is fired that has no matching method on the source interface, the source will not receive the event.


Aggregation, Containment and Inheritance

The COM reusability is accomplished through containment and aggregation. Managed objects can participate in both models as the inner object in the relationship as described below. When it’s desirable for the managed object to be the outer object, the NGWS runtime provides inheritance (see section 0).

The only requirement for managed object to support aggregation or containment is that the type must be public and must have a public default constructor. Meeting both these requirements simply makes the object COM creatable, which is necessary for both containment and aggregation of the object.

Containing Managed Objects

Containment is the simpler of the two reuse models and is supported by managed objects. Containment is typically used when the outer object needs to modify the behavior of the inner object. To do so, the outer object simply creates an instance of the inner object during construction and delegate calls to the inner object as necessary. The outer object can choose which calls to delegate and which calls to handle it self. There are no special requirements of the runtime in order for objects to support containment.

Aggregating Managed Objects

Aggregation is slightly more complex. It is typically used when the outer object wants to expose another objects implementation of an interface without modification. All managed objects automatically support COM style aggregation with the managed object being used as the inner object (or aggregatee). In order to aggregate a managed object, the unmanaged outer object creates the managed inner object by calling CoCreateInstance passing the outer object’s IUnknown as a OuterUnknown parameter. When an outer IUnknown is passed to a managed object during construction, the managed object caches the interface and uses it as follows:

  • The outer IUnknown is not reference counted since the inner objects life is scoped within the lifetime of the outer object.
  • If the inner object is queried for an interface that it does not support, the inner object delegates the call to the outer objects IUnknown interface.
  • All calls to the inner objects AddRef and Release methods are delegated to the outer objects IUnknown.

These three behaviors make it possible to aggregate any managed object. With this type of aggregation relationship, it’s possible to have a single COM object that is partly implemented in managed code (the inner portion) and partly in unmanaged code (the outer portion).

Exposing Inheritance to COM

When managed interfaces are exposed to COM, they always extend IUnknown or IDispatch (See the class System.InteropServices.InterfaceTypeAttribute) even when the interface is inherited from another interface on the managed side. The same applies for the class interface that’s generated for managed classes. For example:

In Managed C++:

Interface IBase {
    void  m1();
}

Interface IDerived : implements IBase {
    void  m2();{
}

class CDerivedImpl : implements IDerived {
    void  m1();
    void  m2();
}

In an Exported Type Library:

Interface IBase : public IDispatch {
    void  m1();
}

Interface IDerived : public IDispatch {
    void  m2();
}

Interface _CDerivedImpl : public IDispatch {
    Boolean Equals();
    int GetHashCode()
    // … plus all other methods of object

    void  m1();
    void  m2();
}

coclass CDerivedImpl {
    interface IBase;
    interface IDerived;
    interface _CDerivedImpl;
    interface _Object;
}

Notice how IBase and IDerived both extend IUnknown in the type library. The inheritance relationship between the two interfaces is intentionally removed. This prevents a COM client from calling methods of the IBase interface through the IDerived interface. In order for a COM client to use either interface, it must explicitly query the coclass for the proper interface before making the call.

Also notice that the class interface contains all the methods on the derived class as well as all the methods on its base class. (System.Object in this case). This is different from the way other (non-class) interfaces are exposed. When late binding through either of these interfaces, only the methods visible on the interface are callable. For example, you could not late bind to method m1 through the IDerived interface. One exception to this rule is with the _Object. The through _Object interface, any method supported by any class in the objects hierarchy can be called. For example, you could safely call m2 late bound through _Object.

An alternative approach would have been to preserve the interface inheritance within the type library. In other words, have the IDerived interface actually extend the IBase interface rather than IUnknown. But, by organizing the interfaces as shown, the IDerived interface becomes more version resilient. Consider what happens to existing users of the IDerived interface when a new method m3 is added to the IBase Interface in the example above.

If the interface inheritance were preserved, the user of the IDerived interface would assume the m2 method was in the 5th slot of the IDerived vtable (after the 3 methods of IUnknown and method m1 from IBase). But, when the new method is added to IBase, the m2 method would actually be relocated to the 6th slot making room for m3 in IBase. When the client tried to make a call on the 5th slot, the call would either go to the implementation of m3 or cause an access violation. Notice that this would happen even if the author of the IBase interface created a new IID for IBase when the new method was added. The reason for the failure is because the definition of the IBase interface is embedded within the definition of IDerived and yet IBase can be changed without the user of IDerived ever know it.

Notice that the client never queried for IBase. The only interface that that client queried for was IDerived, which may have been written by a completely different developer. The only way to prevent this problem would be to change the IID of IBase and every interface that extends IBase.

Now lets consider what happens in the same scenario if the interface inheritance is not exposed to COM. The client that uses IDerived can only calls methods on the IDerived interface. The client assumes that method m1 is in slot 4 of the IBase vtable and the method m2 is in slot 4 of the IDerived v-table. In order to call methods of IBase it must explicitly query for the IID of IBase. When a new method is added to IBase a new IID is generated for the interface automatically (add reference to section that describes automatic IID generation when its added). Now when clients call method m2, the call succeeds because slot of the IDerived v-table still points to method m2 even though a new method was added to IBase. Also notice that if the client queries for the IID of the original IBase interface, the query may succeed or fail depending on whether an implementation of that version of the interface is provided by the class. The author of the class chooses whether to support only the new version of the interface or continue to support the original version. Either way, the client can respond without failing.

Of course managed clients are unaffected by the change in the IBase interface because binding within the runtime is done dynamically.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Context

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Versioning

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Remoting

Accessing managed objects remotely


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Threading

© 2000 Microsoft Corporation. All rights reserved.


Exposing Classic COM objects to the NGWS Runtime

This section describes the mechanism that makes it possible to use COM objects from managed code. This is the mechanism that Runtime clients use to access the services provided by COM objects. When a NGWS runtime client creates an instance of a NGWS object, the object is exposed through a Runtime Callable Wrapper (RCW) that acts as a proxy for the real unmanaged object. These wrappers appear to be just like any other managed class to the NGWS runtime client but they really just marshal calls between managed (NGWS runtime) and unmanaged (classic COM) code. Each COM object being referenced outside the runtime will have a single RCW regardless of the number of references that exist on that object. This ensures that a single object identity is shared between the RCW and the COM object.

Another more advanced techniques for accessing COM object is described in the COM Interop Custom Marshaling specification.

When a NGWS runtime client loads a classic COM object, a Runtime Callable Wrapper (RCW) is created, which looks like a NGWS object to that client. This RCW takes on one of two forms based on how the object was created.

When created as an early bound object, the RCW will be of a specific type and will implement the interfaces the classic COM object implements, and expose as fields any properties available from the coclasses default IDispatch interface. This new NGWS runtime object can be used just like any other managed type. You can cast the object to any of the interface that the object implements and call methods on those interface directly. You can even reflect on the class using the Reflection APIs

Some interfaces on the classic COM object will not be exposed on the NGWS runtime class. These include IUnknown, IDispatch, and IDispatchEx (more to follow). These interfaces are automatically consumed by the RCW and are factored into a different form. For details on how the RCW uses these interfaces see section 5.3 below.

If the object is created in a late bound manner, the RCW will of type System.Object. Users will still be able to reflect on the type and dynamically invoke its members but will only be able to do so through reflection.


Binding to COM classes

NGWS applications can bind to COM classes in an early or late bound fashion. Early binding requires complete type information for the COM class at compile-time and requires the underlying COM class to support early binding as well. Early binding is a much more natural way of using a class. Early bound instances are typically created with the new operator and members are accessed directly through the object instance.

When early binding isn’t possible (or desirable) COM classes can also be created in a late bound fashion. Late bound references do not require a type library for the COM class although you can still late bind to classes that do have metadata. Late bound instances are created and their member are accessed through the reflection API.

COM class that only support late binding through IDispatch are only accessible through reflection from within managed code.

NGWS Metadata Generation

In order to early bind to a COM class or interface a NGWS runtime metadata definition of that class or interface must be available. It’s this NGWS runtime metadata definition of the type that managed clients bind against at compile. The runtime also uses this metadata definition to generate the Runtime Callable Wrapper that will wrap an instance of the COM object at execution time. The metadata must be available at both compile time and at execution time.

The metadata for a COM type is no different than the metadata for a other managed type. The same API’s are used to generate it (IMetadataEmit or ReflectionEmit). In fact, a compiler using the either API could generate the exact same metadata. The types defined in the metadata are no different then types that would be found in any managed program other than a few flags that are set to indicate that the type is a COM type.

The metadata can be generated in various ways. Usually it’s generated from an existing type library that describes the types although it doesn’t have to be. Once generated, the metadata may be persisted to a file or it may be used once and discarded. If persisted, the metadata can be examined with tools such as metainfo.exe or ildasm.exe. The next section explains the various options for creating metadata type definitions.

A NGWS runtime metadata definition of a COM type can be generated by:

  1. Converting an existing type library.
  2. Compiling a source code definition of the type using a capable compiler.

The simplest and most common technique is to start with a COM type library and the utility called TlbImp.exe to convert it the NGWS runtime metadata. There is also an API exposed by the runtime that does the same thing but is more is suitable for use from within a development environment. The other option of describing the type in source does not require a type library to start with but is a more advanced technique.

Generating Metadata From a Type Library

Given an existing type library there are two techniques for converting the tlb file to metadata. The first choice is to use the TlbImp utility to do the conversion. This technique is described in the following section. The other option is to use the TypeLibConverter class in the System.Runtime.InteropServices namespace to do the conversion. Both techniques are essentially the same; the output is identical. However, since the TypeLibConverter works with an ITypeLib interface, it can be use to convert an in-memory type library whereas tlbimp.exe must have a persisted tlb library. The TypeLibConverter class is described below and in the reference pages for the type.

For details on the actual conversion that TlbImp and ITypeLibConverter perform see section 5.13.

Using TlbImp (Type Library Importer)

The type library importer is responsible for converting the type definitions found within a COM type library into equivalent definitions in a NGWS runtime assembly. Conversions are done for an entire type library at one time. You cannot use TlbImp to generate type information for a subset of the types defined within a single type library. The output of TlbImp is an assembly that contains NGWS runtime metadata for the types defined within the original type library. This file can be examined with tools such as metainfo.exe or ildasm.exe.

It's often useful to be able to strong name NGWS runtime assemblies (it's impossible to make references to weak named assemblies from strong named assemblies, for example). Therefore, TlbImp also include options for supplying enough information to generate strongly named assemblies.

TlbImp has the following command line syntax:

TlbImp TlbFile [/out: file] [/silent][/unsafe][/originator: file] [/keyfile: file][/keycontainer: name]

Where TlbFile can be the name of any file containing a COM type library. This file can be a .tlb, .dll, .odl, .ocx, .exe file or any other file format with a type library embedded as a resource. The file must be able to be opened with using LoadTypeLib() API.

TlbImp only imports one type library at a time. On rare occasions, a single file may contain multiple type libraries. In such cases, the specific type library to be converted can be specified by append \res to the end of the file name where res is the number of the resource containing the library (i.e. “foo.exe\3”)

Options:

/out: file - The name of the output file that the metadata definitions will be written to. If the option is omitted, the metadata is written to a file with the same name as the type library (not the file but the actual library name within the file) defined within the input file with the .DLL extension. If the output file is the same name as the input file and error is generated to prevent overwriting the typelib.

/silent – Suppress all a console output.

/unsafe – Generates metadata that has runtime security checks suppressed. Call any method defined exposed in this way may pose security risks. This option should not be used unless the author is aware of the risk and responsibilities of expose such code.

/keyfile: file – Signs the resulting assembly with a strong name using the public/private key pair found in file. The key file should be in the formatted generated by the SN utility.

/keycontainer: name - Signs the resulting assembly with a strong name using the public/private key pair found in the key container called name.

/originator: file – Uses the originator key found in file to identity the resulting assembly. The originator key is generated from the public/private key pair supplied with the /keyfile or /keycontainer options if this option is not supplied. This allows an alternate originator key to be supplied to support developer key pair scenarios. The file is in the format generated by SN the utility that is also part of the SDK.

The /keyfile and /keycontainer options both allow assemblies to be strongly named. It only makes sense to supply one of them at a time.

Examples

TlbImp mytest.tlb

Generate a metadata DLL with the same name as the typelib found in mytest.dll and with the .DLL extension.

TlbImp mytest.tlb /out: Mytest.dll

Generate a NGWS runtime metadata DLL with the name MyTest.dll.

Using TypeLibConverter to import a Type Library

The TypeLibConverter class in the System.Runtime.InteropServices namespace implements the ITypelibConverter interface that can be used by both managed and unmanaged applications that need to convert metadata to type information or vice-versa.

For a complete description of the TypeLibConverter class, see the reference pages for the specific type.

A complete description of the actual conversions made during type library importing is contained in the reference pages for the TypeLibConverter class..

Using a Compiler to Create Metadata

When a type library describing a COM class or interface is not available, the only other option is to create a duplicate definition of the class or interface in source code. Using this approach requires 3 things:

  1. A compiler that is capable of generating the proper NGWS class definitions.
  2. A precise description of the interface or coclass being defined.
  3. Knowledge of how to convert from the COM type system to the NGWS runtime type system.

Type library to Assembly Conversion

Common conversions

Library level conversion

When the TlbImp imports a type library, the types defined within the library are automatically placed in a namespace with the same name as the type library the classes came from. For example:

Library AcmeLib {
   interface Widget {};
   coclass Slingshot {};
}

The AcmeLib type library is imported into the AcmeLib namespace. The Slingshot class would be referred to as AcmeLib.Slingshot.

namespace AcmeLib {
   interface Widget {};
   class Slingshot {};
}

Module level conversion

A type library can have one or more modules containing the definitions of constants and methods as shown in the type library below.

Constants defined within modules are imported as public constant static members of a class with the same name as the module they were imported from. Constants defined outside of a module are not imported.

Methods are also imported as public static members of a class with the same name as the module they were imported from. The members are decorated with the necessary attributes to allow them to directly be called from managed code.

COM Type library

[
   uuid(12345678-1234-1234-1234-123456789ABC),
]
Library TestConstants
{
   [
      uuid(12345678-1234-1234-1234-123456789ABC),
      dllname(“test.dll”),
   ]
   Module Constants
   {
      const long FRAME_COLOR = 0x10;
      const long WINDOW_COLOR = 0x20;
      const long BUTTON_COLOR = 0x40;
      …

      [entry(10) pascal double Square([in] double x);
      [entry(“MessageBox”) pascal int MsgBox([in] BSTR s);

   }
}

Converted to

public class Constants
{
   public const static long FRAME_COLOR = 0x10;
   public const static long WINDOW_COLOR = 0x20;
   public const static long BUTTON_COLOR = 0x40;

   [DllImport(“test.dll”,
EntryPoint = “10”)]
   public static extern double Square(double d);

   [DllImport(“test.dll”,
EntryPoint = “MessageBox”,
CallingConvention = CallingConvention.StdCall)]
   public static extern int MsgBox(String s)
}

Type level conversion

Interfaces

When an interface is imported into an NGWS runtime assembly, the IUnknown methods and the IDispatch methods are stripped from the interface. The imported interface is attributed with the GuidAttribute to retain the IID assigned in the type library and with the InterfaceTypeAttribute if the interface extends IDispatch as opposed to IUnknown.

Unmanaged Interface

[uuid(…), ]
interface IWidget : IUNknown {
   HRESULT Foo()
   HRESULT Bar()
};

[uuid(…), ]
interface IGadget : IWidget {
   HRESULT Baz()
};

Also notice that the methods of the base interface are also added to the derived interface. In this case the methods Foo and Bar are added to the IGadget interface.

Managed Interface

[Guid(…), InterfaceType(ComInerfaceType.InterfaceIsIUnknown)]
interface IWidget {
   void Foo();
   void Bar();
};

[Guid(…), InterfaceType(ComInerfaceType.InterfaceIsIUnknown)]
interface IGadget : implements IWidget {
   void Foo();
   void Bar();
   void Baz();
};

Classes

Add member of default interface

Imported to managed class

How do we handle duplicate names

Structures

Enumerations

Constants

Constants will not be imported from type library in the PDC.

TypeDefs

Type definitions (typedefs) within a type library are not imported. Instead, parameters and fields are imported as the underlying types. For example, a parameter of type BUTTON_COLOR is imported as type integer since BUTTON_COLOR is merely an alias for integer. However, once imported, parameters and fields do carry information that associates them with their original type in the ComAliasNameAttribute. The ComAliasNameAttribute is used to attribute a field, parameter or return value with the name of the type library and the type within the library that was used as an alias. For example, in the following type library, the cl parameter is typed as a BUTTON_COLOR, which is an alias for an integer.

In a type library

Library MyLib {
   typedef [public] int BUTTON_COLOR;

   interface IBar {
      HResult SetColor([in] BUTTON_COLOR cl);
      HResult GetColor([out, retval] BUTTON_COLOR *cl);
   }

   coclass Bar {
      [default] interface IBar
   }
}

Converted to

interface IBar {
   void SetColor([ComAliasName(“MyLib.BUTTON_COLOR”)]] int cl);
   [ComAliasName(“MyLib.BUTTON_COLOR”)] in GetColor();
}

class Bar {
   void SetColor([ComAliasName(“MyLib.BUTTON_COLOR”)]] int cl);
   [ComAliasName(“MyLib.BUTTON_COLOR”)] in GetColor();
}

Notice that the actual type BUTTON_COLOR is not defined in the resulting metadata. The parameters that were typed as BUTTON_COLOR in the type library are typed as the underlying type (int in this case) and are attributed with the ComAliasNameAttribute. Arguments on class methods are also decorated with the attribute. Tools could then use the attribute in whatever way is appropriate.

Member Level conversion

Method Conversions

Property Conversions

Event Conversions

COM type libraries can define interface to be used for eventing. Coclasses that source events identify the event interface as with the [source] attribute. The event interface is implemented by the event sink and consumed by the event source. The COM connection point interface (which are not described in the type library) are used to wire the event source to the event sink. The Button class shown below implements the IButton interface and sources the IButtonEvents interface.

interface IButton {
   HRESULT Init();
}

interface IButtonEvents {
   HRESULT Click([in] int x, [in] int y);
   HRESULT Resize([out, retval] int *pRetval};
}

coclass Button {
   [default] interface IButton;
   [source, default] interface IButtonEvents;
}

The NGWS runtime event model differs considerable from COM connection point model. Managed classes that wish to sink events do so by passing a delegate to the event source. They do not use the COM connection points.

The runtime’s interop service can be used to seamlessly bridge the two models. During import, several types are created that allow managed applications to sink events sourced by unmanaged classes using the NGWS runtime event model.

  1. A delegate type is created for each event in the event interface. The delegate name is created by concatenating the name of the event interface, an underscore, the name of the event, and the word “Delegate”. For example, in the type library above, the Click event produces the delegate called IButtonEvents_ClickDelegate. The signature of the delegate is a direct translation of the unmanaged method signature. See method conversions above.
  2. The default interface is import in the usual way. The interface name remains the same. In this example, the interface is called IButton interface.
  3. The event interface is import in the usual way. The interface name remains the same. In this example, the interface is called IButtonEvent.
  4. In addition, a second event interface with “_Event” append to its name is created that has the events as members along with an “add_event” and “remove_ event” method for each event. The class sinking to the event source uses this interface to add itself to the sources sink list. In the example above, IButtonEvents_Event
  5. The class sourcing the events is imported as any other class would be except the class name is prepended with two underscores. The class implements any interface that is explicitly implemented by the COM coclass. It is also decorated with the ComSourceInterfaces attribute which contains the name of the interface that the class sources events for.
  6. A second class is also created for each coclass that sources events. The second class has the same name as the COM coclass, extends the class described in 3 above and implements the managed event interface described in 2 above.

Other types that are defined during the importing process are for internal use by the runtime.

The following types demonstrate the classes and interfaces that what would be generated for the Button class shown above:

// 1 - A delegate for each event
delegate void IButtonEvents_Click(int x, int y);
delegate int IButtonEvents_Resize();

// 2 - Direct import of original default interface
public interface IButton {
   void Init(};
}
// 3 - Direct import of original event interface - Not of interest to managed sinks
public interface IButtonEvents {
   void Click(int x, int y);
   int Resize(};
}

// 4 - Modified version of the event interface with NGWS runtime events for managed sinks
public interface IButtonEvents_Event {
   IButtonEvents_Click Click;
   IButtonEvents_Resize Resize;
   void Click(IButtonEvents_ClickDelegate};
   void Click(IButtonEvents_ClickDelegate);
   void Resize(IButtonEvents_ResizeDelegate};
   void Resize(IButtonEvents_ResizeDelegate);
}

// 5 - Direct import of original coclass
[ComSourceInterface(“IButtonEvents_Events”)]
public class __Button : IButton {
   void Init(};
}

// 6 - Extension of the original coclass with events added
public class Button : __Button, IButtonEvents_Event
{
   void Init(};
   event IButtonEvents_ClickDelegate Click;
   event IButtonEvents_ResizeDelegate Resize;

   add_Click(OnClickDelegate d);
   remove_Click(OnClickDelegate d);
   add_Resize(OnResizeDelegate d);
   remove_Resize(OnResizeDelegate d);
}

Parameter and Field conversion

The parameter and field conversion rules used by the type library exported are documented in the Data Marshaling Spec.


Activating COM classes

Early Bound Activation

Instantiating an instance of an object in NGWS is simply a matter of creating the object in whatever syntax is appropriate for your language. The same is true for COM objects being used from the NGWS runtime. Here are some examples of how to create an instance of the AcmeLib.Slingshot object in various languages:

In Visual Basic:

Dim ss as AcmeLib.Slingshot
set ss = new AcmeLib.Slingshot.

In Managed C++:

AcmeLib.Slingshot ss;
ss = new AcmeLib.Slingshot();

In C#:

AcmeLib.Slingshot ss;
ss = new AcmeLib.Slingshot();

At compile time, a metadata definition of the AcmeLib.Slingshot object must be available. Again, the technique used to do this depends on your language. In Visual Basic, adding a reference to the metadata DLL makes the type available. In Managed C++, you import the metadata DLL with the #import statement.

At runtime, when a managed application tries to instantiate the class, the runtime again needs to locate the metadata for the class. The situation is the same regardless of whether the class is a NGWS class or a COM class. This metadata must be available at runtime in order to activate a class. Since the metadata for COM classes is always packaged in a module other than the module containing the COM code, activating a COM class usually requires two separate modules.

Once the metadata for the class is located, the runtime loads the metadata assembly and begins to activate the class. The metadata contains various flags to indicate how the class should be constructed. See the Interop Metadata Spec for details on the specific metadata used. If the runtime recognizes that the class is a COM class (metadata for COM classes has the tdImport flag set), it fabricates an RCW for the class and delegates the actual construction to COM’s CoCreateInstance API using the CLSID specified in the metadata. If the metadata was generated with the TlbImp tool, the CLSID was carried forward from the original type library.

The underlying COM class must be co-creatable in order to activate the class in this manner. If the COM class is not creatable the construction will fail. However, RCW’s can be created in other ways. For example, a creatable object may return an instance of another object or interface from a method call. Upon return from the call, an RCW for the returned object would be created. Objects created in this way do not have to be co-creatable but they are required to have metadata.

The next section describes another technique for creating COM objects using reflection. Reflection is very useful for creating COM objects that have no metadata however, it can still be used on objects that do have metadata. Keep that in mind when reading the following section. All the techniques described below also apply to object created in an early bound fashion.

Late Bound Activation Thru Reflection

Another method of instantiating a classic COM server is through reflection. The process is the same regardless of whether the class is a COM class or a NGWS class. Using Reflection to create COM objects is particularly useful when an object does not have accompanying metadata.

A Type object for a specific class can be obtained through the static method Type.GetTypeFromProgID that returns a System.Type object that represents the type specified by a specific ProgID. With the type object you can create an instance of the COM class by passing the type to one of the CreateInstance methods of the Activator class.

You can also inspect the COM type by calling method of the type object. If the COM type has accompanying metadata (produced by importing the type library with TlbImp) , complete information is available about the type through the Type object. If the COM type does not have accompanying metadata, the Type object will provide relatively little information. COM types that have no metadata are wrapped with a generic RCW with the name __ComObject. __ComObject is a private type that is used solely for wrapping COM objects that don’t have associated metadata.

You can pass the object instance to one of the InvokeMember functions on the Type object to actually call methods and access properties of the object through the objects IDispatch interface (if the object supports IDispatch). Arguments are packed into an object array, which is then passed to InvokeMember along with the method name, a set of binding flags, a Binder, and the object instance. When using InvokeMember with a COM object, the following rules apply:

  • The name passed to InvokeMember is case insensitive. This is a limitation of the underlying IDispatch interface.
  • The binding flags passed to InvokeMember are translated into flags passed to IDispatch.Invoke as shown in the table below. Notice that the SetProperty binder flag causes both the DISPATCH_PROPERTYPUT and the DISPATCH_PROPERTYPUTREF flags to be set when IDispatch.Invoke is called. To do an explicit PropertyPut or PropertyPutRef, use the PutDispProperty or PutRefDispProperty binding flags respectively. These flags are only for use with COM objects.
    Binder Flag | Dispatch Flag
    BinderFlags.InvokeMethod | DISPATCH_METHOD
    BinderFlags.GetProperty | DISPATCH_PROPERTYGET
    BinderFlags.SetProperty | DISPATCH_PROPERTYPUT | DISPATCH_PROPERTYPUTREF
    BinderFlags.PutDispProperty | DISPATCH_PROPERTYPUT
    BinderFlags.PutRefDispProperty | DISPATCH_PROPERTYPUTREF
  • The Binder argument is not used.
  • The arguments packed in the object array are converted to COM variants and passed to IDispatch.Invoke. The Object to Variant conversions rules are described in the Data Marshal Spec.
  • In order to pass one or more arguments by reference (in a variant with the VT_BYREF flag set), an array of ParameterModifiers must be passed to InvokeMember. The ParameterModifier array should contain a single ParameterModifier element. The ParameterModifier has a default indexer property. The nth index of the default property should be set to true to indicate that the nth parameter should be passed by reference. For example, the following code passes the second argument in a variant with the VT_BYREF flag set.
    ParameterModifier[] pms = new ParameterModifier[1];
ParameterModifier pm;

pm = new ParameterModifier(3);  // create a param mod that describes 3 params
pm[2] = true;                 // pass the 2nd argument by reference
pms[0] = pm;                 // add pm to the array of modifiers

obj.InvokeMember("m1", BindingFlags.InvokeMethod,
                     null, obj, null, pms, null, null);

The following code samples show how to use InvokeMember on COM types without metadata:

In Managed C++:

#using <mscorlib.dll>

using namespace System;
using namespace Reflection;

Type typ;
Object obj;

typ = Type::GetTypeFromProgID(“Acme.Slingshot”);
obj = Activator.CreateInstance(typ);
typ.InvokeMember("Aim", BindingFlags::InvokeMethod, NULL, obj, NULL);

In C#:

using System;
using System.Reflection;

Type typ;
Object obj;
Object[] prms = new Object[2];

typ = Type.GetTypeFromProgID("Acme.Slingshot");
obj = Activator.CreateInstance(typ);

prms[0] = 10;
prms[1] = 150;

typ.InvokeMember("aim", BindingFlags.InvokeMethod, null, obj, prms);
typ.InvokeMember("SHOOT", BindingFlags.InvokeMethod, null, obj, null);

If you do have metadata for the type, you can use reflection to create a late bound object reference by passing the name of the type to the static method Type.GetType().

The following code sample shows how this can be done:

In C#:

using System;
using System.Reflection;

Type typ;
AcmeLib.Slingshot ss;
Object[] prms = new Object[2];

typ = Type.GetTypeFromProgID("Acme.Slingshot");
ss = (AcmeLib.Slingshot)Activator.CreateInstance(typ);
prms[0] = 10;
prms[1] = 150;

typ.InvokeMember("aim", BindingFlags.InvokeMethod, null, ss, prms);
typ.InvokeMember("SHOOT", BindingFlags.InvokeMethod, null, ss, null);

To invoke a method on a COM object by DISPID rather than by name, you can pass the name “[DISPID=n]” to Type.InvokeMember where n is the DispId of the member. For example, the following code can be used to invoke the method with a DispId of 5 without knowing the name of the method

typ.InvokeMember ("[DISPID=5]", BindingFlags.InvokeMethod, nothing, obj, Params)

It’s important to note that the __ComObject type is a private type used exclusively to wrap COM types that have no metadata. Consequently, reflecting on this type does not reveal information about the underlying COM object. Instead it reveals information about the __ComObject type. For example, the FullName property of an __ComObject type is always “System.__ComObject” as opposed to having the name of the underlying COM object. Other members of the Type object return similar information. Type.GetMembers() for example, returns the members of __ComObject rather than the members of the underlying COM object that you might expect.

For more details on the use of the Type object or the InvokeMember method, see the class library documentation for each.


Runtime Callable Wrappers (RCW's)

The RCW has the following responsibilities:

  • Preserving object identity
  • Maintaining object lifetime
  • Proxying custom interfaces
  • Marshaling method calls
  • Consuming selected interfaces


Preserving Object Identity

An individual RCW may appear to implement several interfaces. When the RCW is cast from one interface to another, internally the wrapper makes a matching QueryInterface call to the classic COM server. If the Query succeeds, the returned interface is cached within the wrapper. All calls on the wrapper are delegated to the underlying object through interfaces it exposes. Regardless of whether the RCW is created early bound, late bound, or even in response to a method call, there will never be more than one instance of the RCW for each instance of the underlying COM object. The RCW preserves the identity of the underlying COM object by comparing the objects IUnknown interface to determine whether the same object is exposing two different interfaces. Therefore, each RCW wraps a unique IUnknown interface.


Maintaining Object lifetime

The RCW is also responsible for keeping the COM object alive as long as the RCW is being used. The RCW itself is garbage collected like any other managed object but internally it holds a reference on the COM object it wraps. When the RCW is garbage collected, the wrappers finalizer is called which, in turn, calls Release() on any interface pointers it’s caching. Garbage collection for a RCW is triggered under the same circumstances as collection for an NGWS object thereby allowing managed clients to treat a COM object like an NGWS object without worrying about COM reference counting.

Deterministic Finalization

Under most circumstance, the scenario described above would work just fine but in some cases it does present a problem. The RCW is automatically garbage collected when the last reference to the wrappers is destroyed. However, there are no guarantees about how long it will take for the RCW to be collected. In fact, it may be some time before the RCW is actually garbage collected. If the underlying COM object is holding resources opened (like a Conn connection) this can be a serious problem as the resources are left opened for prolonged periods. This problem is often referred to as the none-deterministic finalization problem.

Developers using RCW’s should be aware of none-deterministic problem and be prepared to guard against it, especially when the RCW wraps an object that has a large amount of resource overhead. An object that holds open a Conn connection would be a prime example.

Explicit Release

One partial solution to this problem is to take on responsibility for explicitly release references to the underlying COM object. In the normal scenario, the RCW releases all references to the COM object when the wrapper is garbage collected, but you can explicitly release the references yourself rather than waiting for the garbage collector to do it. By passing the RCW to System.InteropServices.ReleaseComObject, you force the wrapper to call Release (one time) on any interfaces it’s caching. The code sample below shows how to explicitly release all references held by an RCW.

In Managed C++:

Baz() {
   Acme.Slingshot ss;         // an RCW for a COM class
   ss = new Acme.Slingshot();   // create the COM object
   ss.shootRoadRunner ();      // use it
   ReleaseComObject(ss);      // explicitly release it
}

This solution is does not working every case however. Consider the following managed code where the object co is a COM object:

class Baz() {
   void M1(Foo f1) {
   }
   void M2(Foo f2) {
      ReleaseComObject(f2)      // ineffective release
   }
}

In this case, the managed class Baz has two methods that take objects as parameters. If the Foo object passed to each of these methods is the same COM object, the call to ReleaseComObject may not be effective. When M1 is called, an RCW is created that wraps the COM object. When f1 goes out of scope at the end of M1, the RCW becomes ready for garbage collectible. The references held by the RCW on the COM object are not released until the collection takes place. If M2 is called before the collection happens, the RCW adds another reference to the underlying COM object to account for f2. When ReleaseComObject is called in M2 the RCW release one reference on the COM object but the second reference keeps the underlying object alive until the RCW is finally collected. The only way to be assured the ReleaseComObject will actually release the underlying object is to make the call before each RCW goes out of scope.


Proxying Custom Interfaces

The RCW also acts as a proxy for most the interfaces exposed by the underlying COM object. Some interfaces are consumed by the RCW but not exposed through the RCW (see Consuming Selected interfaces below). The RCW class itself has no members, but it does implement the same interfaces as the COM coclass that it wraps.

Some languages (like Visual Basic) may choose to show the methods of the default interface as methods of the class itself (see the Interop Metadata Spec for details on identifying the default interface). Those languages would then be responsible for directing calls on the class to calls on the default interface of that class. The RCW does nothing to facilitate this.

A user of an RCW can naturally cast from one interface implemented by the RCW to another. The interface definition needs to be available in order for the cast to succeed. For example:

In Managed C++:

#import “AcmeLib.dll”      // provides defintion of Slingshot and IWeapon
Acme.Slingshot ss;
Acme.IWeapon aw;
ss = new Acme.Slingshot();   // create the COM object
aw = (IWeapon) ss;         // may throw an InvalidCastException
aw.FireWeapon();

The caller can check to see if the RCW supports a particular interface before casting with the following code:

In Managed C++:

Acme.Slingshot ss;
Acme.IWeapon aw;
ss = new Acme.Slingshot();
if (aw.GetType().IsInstanceOfType(ss))
{
   aw = (IWeapon) ss;
   aw.FireWeapon();
}


Marshalling Method Calls

The methods defined on the interfaces the RCW implements look like NGWS runtime methods and follow NGWS runtime conventions. This implies that the NGWS runtime view of these methods differ from the COM view. The static conversion of the interfaces can be done automatically with TlbImp as described above but RCW is responsible for marshaling calls from one view to another at runtime.

The marshaling process involves the following steps:

  • Managing the transition to unmanaged code.
  • Handling error that occur within managed code and converting HResults to exceptions
  • Converting out parameters to return values
  • Marshaling data types from one format to another

Managing Code Transitions

Handling Errors

Converting Parameters and Return Values

Marshaling Data Types

© 2000 Microsoft Corporation. All rights reserved.


Consuming Selected interfaces

IUnknown

The RCW will hold a reference to the classic COM Server, and will release that reference when it is garbage collected. At that point the classic COM Server will be Released. When the NGWS runtime client requests a new interface the wrapper will handle the call through QueryInterface(). When a new interface is passed to a NGWS runtime client via any means the wrapper will be responsible for releasing the reference on that interface when the wrapper is garbage collected.

The NGWS runtime only maintains one RCW per object. It distinguishes between objects by comparing the values of the IUnknown pointers it receives. When an interface is being passed into the NGWS runtime, the Runtime queries on that interface for IUnknown, then checks that interface pointer against the ones already being wrapped.

ISupportErrorInfo

If the classic COM Server being wrapped returns an error HRESULT from a method call, the RCW will throw an exception to the NGWS runtime client. If the classic COM Server provides additional error information (via ISupportErrorInfo and GetLastError) the extended error information will also be propogated into the exception object.

The failure HRESULT returned from a call is converted to an exception and thrown by the RCW on return from the call. The NGWS runtime client can then catch the exception just like any other exception. The type of exception thrown depends on the HRESULT returned. The table below contains a complete list all NGWS runtime exceptions along with the HRESULTs return values that will cause those exceptions to be thrown. If a failure HRESULT that is not listed in the table is returned, the runtime throws an ApplicationException. The client can determine the HRESULT that caused the exception by examining the ErrorCode field of the Exception object.

HResult Returned | Exception thrown by RCW
COR_E_ACCESS | AccessException
COR_E_AMBIGUOUSMATCH | AmbiguousMatchException
MSEE_E_APPDOMAINUNLOADED | AppDomainUnloadedException
MSEE_E_APPDOMAINUNLOADINPROGRESS | AppDomainUnloadInProgressException
COR_E_APPLICATION | ApplicationException
COR_E_ARGUMENT | ArgumentException
E_POINTER | ArgumentNullException
COR_E_ARGUMENTOUTOFRANGE | ArgumentOutOfRangeException
COR_E_ARITHMETIC | ArithmeticException
COR_E_ARRAYTYPEMISMATCH | ArrayTypeMismatchException
MSEE_E_CLASSUNLOADED | ClassUnloadedException
COR_E_COMEMULATE_ERROR | COMEmulateException
COR_E_CONTEXTMARSHAL | ContextMarshalException
COR_E_CORE | CoreException
NTE_FAIL | CryptographicException
COR_E_DIRECTORYNOTFOUND | DirectoryNotFoundException
COR_E_DIVIDEBYZERO | DivideByZeroException
COR_E_DUPLICATEWAITOBJECT | DuplicateWaitObjectException
COR_E_ENDOFSTREAM | EndOfStreamException
COR_E_TYPELOAD | EntryPointNotFoundException
COR_E_EXCEPTION | Exception
COR_E_EXECUTIONENGINE | ExecutionEngineException
COR_E_FIELDACCESS | FieldAccessException
COR_E_FILENOTFOUND | FileNotFoundException
COR_E_FORMAT | FormatException
COR_E_INDEXOUTOFRANGE | IndexOutOfRangeException
COR_E_INVALIDCAST | InvalidCastException
COR_E_INVALIDCOMOBJECT | InvalidComObjectException
COR_E_INVALIDFILTERCRITERIA | InvalidFilterCriteriaException
COR_E_INVALIDOLEVARIANTTYPE | InvalidOleVariantTypeException
COR_E_INVALIDOPERATION | InvalidOperationException
COR_E_IO | IOException
COR_E_METHODACCESS | MethodAccessException
COR_E_MISSINGFIELD | MissingFieldException
COR_E_MISSINGMANIFESTRESOURCE | MissingManifestResourceException
COR_E_MISSINGMEMBER | MissingMemberException
COR_E_MISSINGMETHOD | MissingMethodException
COR_E_MULTICASTNOTSUPPORTED | MulticastNotSupportedException
COR_E_NOTFINITENUMBER | NotFiniteNumberException
E_NOTIMPL | NotImplementedException
COR_E_NOTSUPPORTED | NotSupportedException
COR_E_NULLREFERENCE | NullReferenceException
E_OUTOFMEMORY | OutOfMemoryException
COR_E_OVERFLOW | OverflowException
COR_E_PATHTOOLONG | PathTooLongException
COR_E_RANK | RankException
COR_E_REFLECTIONTYPELOAD | ReflectionTypeLoadException
COR_E_REMOTING | RemotingException
COR_E_SAFEARRAYTYPEMISMATCH | SafeArrayTypeMismatchException
COR_E_SECURITY | SecurityException
COR_E_SERIALIZATION | SerializationException
COR_E_STACKOVERFLOW | StackOverflowException
COR_E_SINKHRONIZATIONLOCK | SinkhronizationLockException
COR_E_SYSTEM | SystemException
COR_E_TARGET | TargetException
COR_E_TARGETINVOCATION | TargetInvocationException
COR_E_TARGETPARAMCOUNT | TargetParameterCountException
COR_E_THREADABORTED | ThreadAbortException
COR_E_THREADINTERRUPTED | ThreadInterruptedException
COR_E_THREADSTATE | ThreadStateException
COR_E_THREADSTOP | ThreadStopException
COR_E_TYPELOAD | TypeLoadException
COR_E_TYPEINITIALIZATION | TypeInitializationException
COR_E_VERIFIER | VerifierException
COR_E_WEAKREFERENCE | WeakReferenceException
All other HResults | ComException

Only failure HRESULTs (those with the severity bit set) will cause exceptions to be thrown by the RCW. Success HRESULTs do not cause exceptions to be thrown.

To get additional information about the exception, the NGWS runtime client would typically examine the fields of the Exception object that’s thrown. In order for the runtime to provide meaningful information about the error, the COM server must implement the IErrorInfo interface. If the COM server supports IErrorInfo, the runtime will use the information provided through IErrorInfo to initialize the Exception object. If the classic COM server does not support IErrorInfo, the Exception object is initialized with default values.

Exception Field | COM Source of information
ErrorCode | HResult returned from call
HelpLink | If IErrorInfo->HelpContext() is non-zero, the String formed by concatenating IErrorInfo->GetHelpFile() and “#” and IErrorInfo->GetHelpContext(). Otherwise the string returned from IErrorInfo->GetHelpFile().
InnerException | Always Null
Message | String returned from IErrorInfo->GetDescription()
Source | String returned from IErrorInfo->GetSource()
StackTrace | The stack trace
TargetSite | The name of the method that returned the failing HResult.

IProvideClassInfo

If the classic COM Server being wrapped implements IProvideClassInfo, the RCW will use this type information to provide support for Reflection as well as interrogating the type information for it’s own purposes. Type information may also be obtained from the Registry, or from an IDispatch implementation. As mentioned in Section 2, Class information is required before an object can be wrapped.

IDispatch

Dispinterfaces exposed by COM servers can only be accessed from the NGWS runtime through reflection. Reflection is the mechanism for doing object inspection and dynamic invocation. You cannot make a direct method call on an RCW if the RCW is only wrapping a dispinterface.

Properties

A dual interface or dispinterface may declare properties as well as methods. All properties have corresponding accessor methods for setting or getting the property values. When converting a type library description of an interface with properties to metadata, a property as well as one or more accessor methods for that property is created. The type library conversion process applies the following transformations to property accessor methods.

  • Properties with the [propget] attribute are converted to managed properties of the same type along with a corresponding a get method called get_propertyname.
  • Properties with either a [propput] or [propputref] attribute are converted to managed properties of the same type along with a corresponding a set method called set_propertyname.
  • Properties with both a [propput] and [propputref] attribute are converted to managed properties of the same type as the [propputref] along and a corresponding a set method called set_propertyname and another accessor method with the same type as the [propput] with the name “let_propertyname” prefix.

For example:

In a Type Library:

interface ISample : IDispatch {
   [propget,    HRESULT prop1([out, retval] short *pVal);
   [propput,    HRESULT prop1([in] short newVal);

   [propget,    HRESULT prop2([out, retval] IFoo **pVal);
   [propputref, HRESULT prop2([in] IFoo *newVal);

   [propget,    HRESULT prop3([out, retval] IFoo **ppIFoo);
   [propput,    HRESULT prop3([in] BSTR *text);
   [propputref, HRESULT prop3([in] IFoo *pIFoo);
}

In Managed C++:

interface ISample {
   __property short prop1 {get {}; set {};};   // propget, propset
   __property IFoo prop2 {get {}; set {};};    // propget, propset
   __property IFoo prop3 {get {}; set()};   // propget, propputref
   Letprop3(String text);               // propput

}

In Visual Basic:

Public Property Get Prop1() As Integer … End Property
Public Property Let Prop1(val as Integer) … End Property

Public Property Get Prop2() As IFoo … End Property
Public Property Let Prop2(val as IFoo) … End Property

Public Property Get Prop3() As IFoo … End Property
Public Property Let Prop3(val as IFoo) … End Property
Public Setprop3Value(String as Text) … End Property

Methods

Events

IDispatchEx

ITypeInfo

The runtime also provides a custom marshaler for marshaling the ITypeInfo interface to the System.Type class. Any unmanaged method that uses the ITypeInfo interface will automatically use the marshaler when the method is exposed to the NGWS runtime. The TlbImp utility will convert all references to ITypeInfo to System.Type on import.

For example, the following unmanaged type:

interface TypeProvider {
   HRESULT SetType([in] ITypeInfo *t);
   HRESULT SetTypeRef([in, out] ITypeInfo **t);
   HRESULT GetType([out, retval] ITypeInfo **t);
}

Is converted to:

interface TypeProvider {
   void SetType(System.Type t);
   void SetTypeRef(ref System.Type t);
   System.Type GetType();
}

IConnectionPointContainer

IPersistFile, IPersistStream

IEnumVariant

A custom marshaler is provided by the runtime that provides a bridge between the COM interface IEnumVariant and the the NGWS frameworks interface IEnumerator. The custom marshaler causes method parameters and properties of type IEnumVariant to be exposed to the NGWS runtime as the IEnumerator interface. So for example, a COM collection object with a property of type IEnumVariant whose unmanaged signature is:

HResult _NewEnum(IEnumVariant *pEnum);

Would have the following signature when exported to a COM type library.

IEnumerator _NewEnum ();

The marshaler automatically delegates the calls from the NGWS runtime interface to the COM interface in the following way:

bool IEnumerator.MoveNext()

Calls IEnumVariant.Next(1, …) and caches the Variant returned as an Object. If Next returns S_FALSE, the method returns False otherwise the method returns true.

Object IEnumerator.Current()

Returns the object cached when MoveNext was called.

For additional details on custom marshaling see the Custom Marshaling spec.

Identifying Enumerators

In the NGWS frameworks, enumerators are provided by the System.Collections.IEnumerable interface.

public interface IEnumerable
{
   IEnumerator GetEnumerator();
}

The interface has a method called GetEnumerator that returns an enumerator. When imported by typelib importer, any COM coclass that provides an enumerator will also appear to implement the IEnumerable interface. For example, the Fields interface defined in a typelib as follows:

interface Fields : {
   …
   [id(0xfffffffc)] HRESULT _NewEnum([out, retval] IUnknown** ppvObj);
   …
};

Would be imported by the typelib importer as:

interface Fields : implements IEnumerable
{
   Object _NewEnum();
   IEnumerator GetEnumerator();
}

Notice that the GetEnumerator method is automatically added to the interface and that the interface implements IEnumerable. The proper way of obtaining the enumerator for any managed type (including RCW’s) is to cast the type to IEnumerable and then call IEnumerable.GetEnumerator.


Inheritance

The COM model for reusability is containment and aggregation. The NGWS runtime adds the ability to use implementation inheritance. You can even have managed type that derives from a COM objects RCW. The managed type can then expose all the method and properties of the COM object as well as methods and properties implement in managed code. The result is an object that is partly implemented in managed code and partly implemented in unmanaged code sometime called a mixed-mode object.

Inheriting from COM coclasses

To accomplish this, there are three requirements that the base, unmanaged type must satisfy:

  • There must be metadata that defines the unmanaged type.
  • The type must be creatable.
  • The type must be aggregatable (in the COM sense).

If the requirements are met, a managed type can then extend the RCW for that unmanaged type and override the methods provided by the base object. The inheritance is done exactly the same as it would if the base object were managed. For example

In Managed C++:

#import “AcmeLib.dll”      // provides defintion of Slingshot and

class Catapult : public AcmeLib.Slingshot  // extend COM type
{
   Load() {…};
   Fire() {…};           // delegates to base implementation
   {
      Slingshot::Fire();
   }
}

Catapult *cp = new Catapult();

cp.Load();         // calls derived implementation
cp.Aim();         // calls base implementation
cp.Fire();         // calls derived which delegates to base

[discuss RegisterObjectCreationCallback here]

Interface Inheritance

When importing and interface from a COM type library, all inheritance relationships are preserved with the exception of IUnknown and IDispatch. For example, The following interfaces defined in a type library would be converted to metadata as shown.

In a Type Library:

Interface IUnknown {…};
Interface IDispatch : public IUnknown  {…};

Interface IBase : public IDispatch {
   void m1();
}

Interface IDerived : public IBase{
   void m2();
}

coclass CDerivedImpl {
   interface IDerived;
}

In a Managed C++:

Interface IBase {
   void m1();
}

Interface IDerived : implements IBase {
   void m1();
   void m2();
}

class CDerivedImpl implements IDerived {
}

Notice that the IBase interface does not derive from IDispatch once it’s imported. The inheritance from IUnknown and IDispatch are automatically removed during import because equivalent functionality is provided thru other mechanisms. Also notice that the methods of IBase (m1) are also added to the definition of the IDerived interface. This is ensures backward compatibility with existing COM types. For example, consider what happens when the interfaces defined in the above type library are imported, implemented in managed code and then exported. COM clients that were using the original unmanaged implementation expect to be able to call methods of IBase directly from through IDerived interface when the implementation is move to managed code. By adding the methods of IBase to IDerived, we ensure that this behavior is still supported.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Metadata

© 2000 Microsoft Corporation. All rights reserved.


Threads and Context

COM Threading Models

Apartments – COM apartments are used to group objects with similar sinkhronization requirements so that calls to objects within those apartments can be dispatched correctly. COM supports 3 apartment types.

 | STA-Main | STA | MTA | NTA
Is access to the objects sinkhronized? | Yes | Yes | No | No
Are objects always accessed by the same thread? | Yes | Yes | No | No
Are threads that leave the apartment allowed to re-enter apartment? | No | No | Yes | Yes
Are objects isolated from objects in other apartments? | Yes | No | Yes | Yes
Number of apartments per process | 1 | n | 1 | 1
Number of threads per apartment | 1 | 1 | n | n
Requires message pumping | Yes | Yes | No | No

STA – The COM single threaded apartment is used to ensure sinkhronized access to objects that are not explicitly thread safe. COM uses the windows message pump to sinkhronize all calls to objects living within the same STA. An individual process may have one or more STA apartments. COM automatically marshals all calls from objects outside the STA. An STA is explicitly created by calling CoInitializeEx(COINIT_APARTMENTTHREADED). STA’s are also created implicitly when objects mark with ThreadingModel=Apartment are created from non-STA threads.

MTA – The COM multi-threaded apartment is used to host objects that are built in a thread safe manner. Calls made to objects within this type of apartment are handled by the callers thread. COM makes no effort to ensure sinkhronization to objects within the MTA. Each individual process has at most one MTA. An MTA is explicitly created when a thread calls CoInitializeEx(COINIT_MULTITHREADED). All threads that call CoInitializeEx(COINIT_MULTITHREADED) reside within the same multithreaded apartment. An MTA can also be created implicitly when objects mark with ThreadingModel=Free are created from a non-MTA thread.

NTA – The COM neutral threaded apartment is used to host objects that have no thread preference. Calls to objects within the NTA are always made by the callers thread regardless of the type of apartment the thread is calling from.

COM+ 1.0 Contexts

Contexts – COM+ 1.0 contexts provide a way of grouping objects with similar characteristics so that the policies applied to that context can be enforced as calls are dispatched in and out of the context.

COM+ 1.0 Context Attribute | Setting
Activate in activators context | On/Off
Transaction | Non-supported, Supported, Required, Requires New
Sinkhronization | Non-supported, Supported, Required, Requires New
Object Pooling | On/Off, Max Instances, Min Instances, Timeout
Declarative Construction | Arbitrary string
JIT Activation | On/Off
Load Balancing | On/Off
Instrumentation (tracker?) | On/Off
Declarative Authorization | Role Names
Auto Deactivated | On/Off
SPM? | ?

NGWS Runtime Contexts

NGWS Runtime Context Attribute | Setting
Transaction | Non-supported, Supported, Required, Requires New
Sinkhronization | Non-supported, Supported, Required, Requires New
Reentrancy | Non-supported, Supported, Required, Requires New
Thread Affinity | Non-supported, Supported, Required, Requires New

See Jonathan’s context doc

Unified NGWS Runtime Context Model

Application Domains

Apartment to Context Mapping

© 2000 Microsoft Corporation. All rights reserved.


Restrictions

Type Library Requirements

The techniques for using COM class described in this document require that the class be described in NGWS runtime metadata. This metadata can be generated in a variety of ways the simplest of which is by converting an existing COM type library. Most COM objects already have a type library either in the form of a separate .TLB file or embedded within the .DLL or .EXE file that contains the object. If type library is not available, one can be created from an existing .IDL file using the MIDL compiler that is part of the Win32 SDK If neither form of type information exists then you have 3 options:

  1. Create a type library from scratch for the object using IDL and the MIDL compiler.
  2. Restrict your use of the object to late bound access only. This requires that the COM object support late bound access by exposing an IDispatch interface.
  3. Use the alternative approach described in the COM Interop Custom Marshaling specification for accessing the COM object.

Other Restrictions

There are other situations in which custom marshaling may be required. These include:

  • Interfaces with methods that use type unsafe parameters such as void * or unions
  • Interfaces with methods that return success codes
  • Interfaces with methods that pass buffers of data where one parameter contains the buffer and another parameter specifies the size of the buffer.
  • If a type library contains an interface that use one of the restricted techniques described above, other interfaces described within the type library are still usable.


Type Library to Metadata conversions

The COM method signatures differ from the NGWS runtime signatures in the following ways:

  • COM data types are converted to NGWS runtime types. For example BSTR parameters are converted to parameters of type System.String. See the Data Marshaling Spec for a complete list of the data type conversions and how they can be overridden.
  • Parameters marked as [out retval] in a type library are converted to method return values and removed from the parameter list.
  • HResults are not returned from methods. Instead the methods return the [out retval] parameter and throw exceptions when a failure HResult is returned. Any method that returns an HRESULT and does not have an out parameter marked as retval returns void.

The class defined in the metadata derives from System.Object, and implements all of the interfaces which the classic COM object implements. While the method definitions appear on the class, they are flagged as MethodImpl. This means that when you create an instance of this class, you can call the methods inherited from Object but you cannot call the interface method implementations directly. Instead, you can call them through an interface pointer acquired from the class. This is analogous to the way classes and interfaces work in classic COM.

This implies that the following code will NOT work (where CFoo wraps a classic COM server implementing IFoo, which has a Bar method)

CFoo MyFoo = new CFoo;
MyFoo->Bar();

This does work:

IFoo MyIFoo = (IFoo) new CFoo;

The wrapper classes that are created (like CFoo in the above example) do have a simple constructor which is implemented by the wrapper. We may call CoCreateInstance() during this constructor or defer creation of the classic COM object to another time (like the first method call). You should be prepared for ClassLoadException (or related exceptions) at any time.

Appendices


Appendix A - Type Library Importer Conversions

During the course of import information from a type library, the importer makes various data type conversions. The conversion rules used by TlbImp are captured below. Any type not explicitly listed below is converted to an I4. These types can then be manipulated with the helper routines found in the System.Runtime.InteropServices.Marshal class.

The following type library type | Is imported as the following Element Type | With the following MarshalAs Attribute | System Type | Built-in Language Type
 |  |  |  | VB | Mgd C++ | C#
char, boolean, small | I1 |  | System.SByte | n/a | byte | sbyte
wchar_t, short | I2 |  | System.Int16 | short | short | short
long, int | I4 |  | System.Int32 | integer | long | int
hyper | I8 |  | System.Int64 | long | __int64 | long
unsigned char, byte | UI1 |  | System.Byte | byte | byte | byte
unsigned short | UI2 |  | System.UInt16 | N/A | unsigned short | ushort
unsigned long, unsigned int | UI4 |  | System.UInt32 | N/A | unsigned long | uint
unsigned hyper | UI8 |  | System.UInt64 | N/A | unsigned __int64 | ulong
single | R4 |  | System.Single | single | float | float
double | R8 |  | System.Double | double | double | double
VARIANT_BOOL | Boolean | VariantBool |  | boolean | bool | bool
void * | I4 |  |  | n/a | n/a | n/a
HResult | UI4 | Error |  | n/a | unsigned long | uint
SCODE | UI4 | Error |  | n/a | unsigned long | uint
BSTR | STRING | BStr | System.String | string | n/a | string
LPSTR or [string, …] char * | STRING | LPStr | System.String | string | n/a | string
LPWSTR or [string, …] wchar_t * | STRING | LPWStr | System.String | string | n/a | string
VARIANT | OBJECT |  | System.Object |  |  |
DECIMAL | VALUETYPE <System.Decimal> |  | System.Decimal |  |  |
DATE | VALUETYPE <System.DateTime> |  | System.DateTime |  |  |
GUID | VALUETYPE <System.Guid> |  | System.Guid |  |  |
CURRENCY | VALUETYPE <System.Currency> |  | System.Currency |  |  |
IUnknown * | OBJECT | Interface | System.Object |  |  |
IDispatch * | OBJECT | Interface | System.Object |  |  |
SAFEARRAY(type) | SZARRAY(type) | SAFEARRAY | type[] |  |  |
typedef BaseType MyType | BaseType |  |  |  |  |
MyStruct | VALUETYPE <MyStruct> |  |  |  |  |
MyEnum | VALUETYPE <MyEnum> |  |  |  |  |
MyInterface * | CLASS <MyInterface> | Interface |  |  |  |
MyCoClass | CLASS <_Class> | Interface |  |  |  |
Pointer Types
char *, boolean *, small * | ByRef I1 |  | System.SByte | n/a | byte | sbyte
wchar_t *, short * | ByRef I2 |  | System.Int16 | short | short | short
long *, int * | ByRef I4 |  | System.Int32 | integer | long | int
hyper * | ByRef I8 |  | System.Int64 | long | __int64 | long
unsigned char *, byte * | ByRef UI1 |  | System.Byte | byte | byte | byte
unsigned short * | ByRef UI2 |  | System.UInt16 | N/A | unsigned short | ushort
unsigned long *, unsigned int * | ByRef UI4 |  | System.UInt32 | N/A | unsigned long | uint
unsigned hyper * | ByRef UI8 |  | System.UInt64 | N/A | unsigned __int64 | ulong
single * | ByRef R4 |  | System.Single | single | float | float
double * | ByRef R8 |  | System.Double | double | double | double
VARIANT_BOOL * | ByRef Boolean | VariantBool |  | boolean | bool | bool
void ** | ByRef I4 | Add MashalAs attrib post PDC |  | n/a | n/a | n/a
HResult * | ByRef I4 | Error |  | n/a | unsigned short | ushort
SCODE * | ByRef I4 | Error |  | n/a | unsigned short | ushort
BSTR * | ByRef STRING | BStr | System.String | string | n/a | string
LPSTR * | ByRef STRING | LPStr | System.String | string | n/a | string
LPWSTR * | ByRef STRING | LPWStr | System.String | string | n/a | string
VARIANT * | ByRef OBJECT |  | System.Object |  |  |
DECIMAL * | ByRef VALUETYPE <System.Decimal> |  | System.Decimal |  |  |
DATE * | ByRef VALUETYPE <System.DateTime> |  | System.DateTime |  |  |
GUID * | ByRef VALUETYPE <System.Guid> |  | System.Guid |  |  |
CURRENCY * | ByRef VALUETYPE <System.Currency> |  | System.Currency |  |  |
IUnknown ** | ByRef OBJECT | Interface | System.Object |  |  |
IDispatch ** | ByRef OBJECT | Interface | System.Object |  |  |
SAFEARRAY(type) * | ByRef SZARRAY(type) | SAFEARRAY | type[] |  |  |
typedef BaseType MyType | ByRef BaseType |  |  |  |  |
MyStruct * | ByRef VALUETYPE <MyStruct> |  |  |  |  |
MyEnum * | ByRef VALUETYPE <MyEnum> |  |  |  |  |
MyInterface ** | ByRef CLASS <MyInterface> | Interface |  |  |  |
MyCoClass * | ByRef CLASS <_Class> | Interface |  |  |  |


Appendix B - Type Library Exporter Conversions

The following Element Type | With the following MarshalAs Attribute | Is converted to the following Type library type | System Type | Built-in Language Type
 |  |  |  | VB | Mgd C++ | C#
Boolean |  | VARIANT_BOOL | System.Boolean | boolean | bool | bool
I1 |  | char | System.SByte | n/a | byte | byte
I2 |  | short | System.Int16 | short | short | short
I4 |  | long | System.Int32 | integer | long | int
I8 |  | hyper | System.Int64 | long | __int64 | long
UI1 |  | unsigned char | System.Byte | byte | byte | byte
UI2, CHAR |  | unsigned short | System.UInt16 System.Char | n/a | unsigned short, | ushort, char
CHAR | Char | char | System.Char | n/a | char | char
UI2 | Error | SCODE | System.UInt16 | n/a | unsigned short | ushort
UI4 |  | unsigned long, unsigned int | System.UInt32 | N/A | unsigned long | uint
UI4 | Error | SCODE |  | n/a | n/a | n/a
UI8 |  | unsigned hyper | System.UInt64 | N/A | unsigned __int64 | ulong
R4 |  | single | System.Single | single | float | float
R8 |  | double | System.Double | double | double | double
STRING | BStr | BSTR | System.String | string | n/a | string
STRING | LPStr | LPSTR | System.String | string | n/a | string
STRING | LPWStr | LPWSTR | System.String | string | n/a | string
OBJECT |  | VARIANT | System.Object |  |  |
VALUETYPE <System.Decimal> |  | DECIMAL | System.Decimal |  |  |
VALUETYPE <System.DateTime> |  | DATE | System.DateTime |  |  |
VALUETYPE <System.Guid> |  | GUID | System.Guid |  |  |
VALUETYPE <System.Currency> |  | CURRENCY | System.Currency |  |  |
OBJECT | Interface | IUnknown * | System.Object |  |  |
OBJECT | Interface | IDispatch * | System.Object |  |  |
SZARRAY(type) | SAFEARRAY | SAFEARRAY(type) | type[] |  |  |
BaseType |  | typedef BaseType MyType |  |  |  |
VALUETYPE <MyStruct> |  | MyStruct |  |  |  |
VALUETYPE <MyEnum> |  | MyEnum |  |  |  |
CLASS <MyInterface> | Interface | MyInterface |  |  |  |
CLASS <_Class> | Interface | MyCoClass |  |  |  |
Reference Types
ByRef Boolean |  | VARIANT_BOOL * | System.Boolean | boolean | bool | bool
ByRef I1 |  | char * | System.SByte | boolean | byte | sbyte
ByRef I2 |  | short * | System.Int16 | short | short | short
ByRef I4 |  | long * | System.Int32 | integer | long | int
ByRef I8 |  | hyper * | System.Int64 | long | __int64 | long
ByRef UI1 |  | unsigned char * | System.Byte | byte | byte | byte
ByRef UI2, ByRef CHAR |  | unsigned short * | System.UInt16 | N/A | unsigned short | ushort
ByRef CHAR | Char | char * | System.Char | N/A | char | char
ByRef UI4 |  | unsigned long * | System.UInt32 | N/A | unsigned long | uint
ByRef I4 | Error | HRESULT * | System.UInt32 | N/A | unsigned long | uint
ByRef R4 |  | single * | System.Single | single | float | float
ByRef R8 |  | double * | System.Double | double | double | double
ByRef STRING | BStr | BSTR * | System.String | string | n/a | string
ByRef STRING | LPStr | LPSTR * | System.String | string | n/a | string
ByRef STRING | LPWStr | LPWSTR * | System.String | string | n/a | string
ByRef OBJECT |  | VARIANT * | System.Object |  |  |
ByRef VALUETYPE <System.Decimal> |  | DECIMAL * | System.Decimal |  |  |
ByRef VALUETYPE <System.DateTime> |  | DATE * | System.DateTime |  |  |
ByRef VALUETYPE <System.Guid> |  | GUID * | System.Guid |  |  |
ByRef VALUETYPE <System.Currency> |  | CURRENCY * | System.Currency |  |  |
ByRef OBJECT | Interface | IUnknown ** | System.Object |  |  |
ByRef OBJECT | Interface | IDispatch ** | System.Object |  |  |
ByRef SZARRAY(type) | SAFEARRAY | SAFEARRAY(type) * | type[] |  |  |
ByRef BaseType |  | typedef BaseType MyType |  |  |  |
ByRef VALUETYPE <MyStruct> |  | MyStruct * |  |  |  |
ByRef VALUETYPE <MyEnum> |  | MyEnum * |  |  |  |
ByRef CLASS <MyInterface> | Interface | MyInterface * |  |  |  |
ByRef CLASS <_Class> | Interface | MyCoClass * |  |  |  |


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Appendix C - Metadata/Type library Conversion Samples

© 2000 Microsoft Corporation. All rights reserved.


Sample 1

NGWS Runtime Metadata COM Type library
namespace AnimalLib {
   class Mammal
   {
      void Eat();
      void Breathe():
      void Sleep();
   }
}

[uuid(0000…0000), version(1.0)]
library AnimalLib
{
   [odl, uuid(0000…0000), hidden, dual, nonextensible, oleautomation]
   interface _Mammal : IDispatch
   {
      [id(0x00000000), propget] HRESULT ToString([out, retval] BSTR* pRetVal);
      [id(0x60020001)] HRESULT Equals([in] VARIANT obj, [out, retval] VARIANT_BOOL* pRetVal);
      [id(0x60020002)] HRESULT GetHashCode([out, retval] long* pRetVal);
      [id(0x60020003)] HRESULT GetType([out, retval] _Type** pRetVal);
      [id(0x6002000d)] HRESULT Eat();
      [id(0x6002000e)] HRESULT Breathe([out, retval] long* p1):
      [id(0x6002000f)] HRESULT Sleep([out, retval] double* p1);
   }
   [uuid(0000…0000)]
   coclass Mammal
   {
      [default] interface _Mammal;
   }
}


Sample 2

NGWS Runtime Metadata COM Type library
namespace AnimalLib {
   Interface IAnimal
   {
      void Eat();
      void Sleep():
   }

   class Mammal : IAnimal
   {
      void  Eat();
      void  Breathe():
      void  Sleep();
   }
}

[uuid(0000…0000), version(1.0)]
library AnimalLib
{
   [uuid(0000…0000)]
   interface IAnimal : IUnknown
   {
      [id(0x60020000)] HRESULT Eat();
      [id(0x60020001)] HRESULT Sleep[out, retval] long* p1):
   }

   [odl, uuid(0000…0000), hidden, dual, nonextensible, oleautomation]
   interface _Mammal : IDispatch
   {
      [id(0x00000000), propget] HRESULT ToString([out, retval] BSTR* pRetVal);
      [id(0x60020001)] HRESULT Equals([in] VARIANT obj, [out, retval] VARIANT_BOOL* pRetVal);
      [id(0x60020002)] HRESULT GetHashCode([out, retval] long* pRetVal);
      [id(0x60020003)] HRESULT GetType([out, retval] _Type** pRetVal);
      [id(0x6002000d)] HRESULT Eat();
      [id(0x6002000e)] HRESULT Breathe([out, retval] long* p1):
      [id(0x6002000f)] HRESULT Sleep([out, retval] double* p1);
   }

   [uuid(0000…0000)]
   coclass Mammal
   {
      [default] interface _Mammal;
      interface _Object;
      interface IAnimal;
   }
}


Sample 3

NGWS Runtime Metadata COM Type library
namespace AnimalLib {
   Interface IAnimal
   {
      void  Eat();
      void  Sleep();
   }

   Interface IHuman
   extends IAnimal
   {
      void  Walk():
      void  Talk():
   }
   Class Person
   implements IHuman
   {
      void  Eat();
      void  Sleep();
      void  Walk():
      void  Talk():
      String GetName();
      short GetAge();
   }
}

[uuid(0000…0000), version(1.0)]
library c:\winnt\system32\mamsvr.dll
{
   [uuid(0000…0000)]
   interface IAnimal : IUnknown
   {
      HRESULT Eat();
      HRESULT Sleep();
   }

   [uuid(0000…0000)]
   interface IHuman : IUnknown  // does not have the methods of IAnimal
   {
      HRESULT Walk();
      HRESULT Talk();
   }

   [uuid(0000…0000), dual, odl, oleautomation]
   interface _Person : IDispatch
   {
      [id(0x00000000)] HRESULT ToString([out, retval] BSTR* pRetVal);
      [id(0x60020000)] HRESULT Init();
      [id(0x60020001)] HRESULT GetClass([out, retval] _Class** pRetVal);
      [id(0x60020003)] HRESULT Equals([in] _Object* obj, [out, retval] VARIANT_BOOL*                                        pRetVal);
      [id(0x60020004)] HRESULT GetHashCode([out, retval] long* pRetVal);
      [id(0x60020005)] HRESULT Clone([out, retval] _Object** pRetVal);
      [id(0x60020006)] HRESULT Dispatch([in] BSTR name,[in] _Variant** parameters,
                        [out, retval] _Variant** pRetVal);
      [id(0x60020007)] HRESULT Wait();
      [id(0x60020008)] HRESULT Wait_2([in] long p1);
      [id(0x60020009)] HRESULT Notify()
      [id(0x6002000a)] HRESULT NotifyAll();
      [id(0x6002000b)] HRESULT Finalize();
      [id(0x6002000c)] HRESULT GetObjectContext([out, retval] long* pRetVal);
      [id(0x6002000d)] HRESULT Eat();
      [id(0x6002000e)] HRESULT Sleep();
      [id(0x6002000f)] HRESULT Walk();
      [id(0x60020010)] HRESULT Talk();
   }

   [uuid(0000…0000)]
   coclass Person
   {
      [default] interface _Person;
   }
}


Appendix D- Differences between Internal and OleAut implementation of IDispatch

  1. Invoking a method with some required & some optional parameters but not giving any parameters
    • Compatible: "Invalid number of parameters." (0x8002000e),
    • Internal: "Not enough storage is available to complete this operation." (0x8007000e)
  2. The error being returned indicates that the typelib that OLEAUT uses to do the invoke call is invalid. The argument type of the method being invoked is invalid and this is what is causing the "Bad Variable Type" to be returned. Since the custom implementation is not driven by a typelib the error code being returned is different.
    • Compatible: "Bad Variable Type"
    • Internal: "Catastrophic failure”
  3. When a COM client passes a VT_* | VT_BYREF but sets the member to NULL,
    • Compatible: Returns E_POINTER: "A null reference pointer was passed to the stub."
    • Internal: Returns DISP_E_EXCEPTION: "Exception occurred.”
  4. When a COM client passes a variant of type VT_ERROR to a method taking integer arguments
    • Compatible: fails with DISP_E_TYPEMISMATCH
    • Internal: Succeeds
  5. Late-bound COM clients can invoke managed methods with int or uint parameters passing VT_I4 or VT_INT and VT_UI4 or VT_UINT if the server is using the Internal implementation. If the server is using the Compatible implementation, the client can only use VT_I4 and VT_UI4. VT_INT and VT_UINT will fail with a type mismatch. (HRESULT 0x80020005)
  6. The internal implementation of IDispatch requires that named arguments appear after positional arguments. The named arguments can appear in any order must follow a method position arguments.


Appendix E - Known compatibility Issues

While the goal of the runtimes “compatible” IDispatch implementation is to be completely compatible with VB6, there are know areas of incompatibility that were made for performance reasons. This section documents areas where the runtimes compatible IDispatch differs from VB6.

  1. Passing ByRef parameters with uninitialize pointers.
    • When a COM client passes a VT_BSTR | VT_BYREF to a COM server, but sets the pbstrVal to NULL, OleAut gives HRESULT 0x800706f4 (E_INVALIDARG): "A null reference pointer was passed to the stub."
    • When a COM client passes a VT_BSTR | VT_BYREF to a NGWS runtime server (using the OleAut-Compatible IDispatch implementation), but sets the pbstrVal to NULL, a NullReferenceException is thrown.
  2. Date precision
    • The precision of the OLE Automation DATE type is limited to seconds. The precision of System.DateTime is 100 nanoseconds. Because of this, COM clients should avoid setting or comparing DATEs to literal numbers such as 123.456 if passing these DATEs to managed code. The numbers are not guaranteed to roundtrip exactly, so users should be using OLE Automation functions to create and compare DATE types.


Isolated Storage

NGWS Runtime Security Specification

Synopsis: Isolated storage is the new model for applications to handle persistent data. Developers can now access data in an platform-independent way such as, “the current_orders file for this application for the current user”, without having to concoct an ad hoc path in a file system such as “C:\app13\JoeUser\current_orders”. Such file system paths are awkward to handle in deployment across a wide range of systems including administration of the data and access rights associated with these applications: by providing an identity-based storage abstraction, isolated storage handles these problems in a generalized way for all developers to take advantage of.


Introduction to Isolated Storage

Applications often need to store data – traditionally file systems have satisfied this requirement for both client and server based systems. However, web applications present a new set of requirements for data storage by applications because they often consist of several components from various sources, and because these new applications often exist “on the web” rather than being installed into systems as has been the rule in the past.

So what’s wrong with conventional file systems for application storage? Data is stored in a hierarchical file system as referenced by a unique path (e.g. c:\files\my-app\file1.txt) so applications must use various ad hoc methods to locate their own area on any given system, and to avoid interfering with the data of another application that might happen to have the same name or use the same area. The design of these systems provides no help for applications as to how they should parcel up space, since the architecture dates back to the days of static pre-installed applications on single systems – often where most or all applications were made by the same people, in which case it is easy to avoid these pitfalls. However, in a world where applications and components come from many different sources and need to work together smoothly (without cross-interference) in all combinations, these ad hoc techniques (i.e. hope you have chosen a unique path for your application’s data) break down quickly.

What are the security implications of using file systems for application data? From a security standpoint, file systems have an inherent weakness in that data written by one application for one purpose is potentially exposed to other applications, including possibly malicious use of the data. Certainly with security mechanisms such as ACLs (access control lists that say who can use what files) some protections exist, however: (a) current implementations restrict what users can use what files – leaving unaddressed the problem of unwanted access by code even running within the context of a trustworthy user; (b) security is only possible to the extent there is active and thorough administration. For example, for any new application and administrator needs to assign it a unique path for its files, configure the application to only use files in that area (which can’t be enforced normally), and then ensure that security is setup properly on all systems before the application is allowed to run there. With web applications this represents a large expenditure of effort, if it is really feasible at all.

Isolated storage provides a configuration-less data store for applications. Data compartments are defined based on the identity of the application or component code – there is no need to conjure unique paths to ensure that data storage of different applications does not overlap so long as the identities are unique. Administrators can limit how much storage applications may have based on trust, plus set an overall size limit to prevent disk-full errors when many pieces of code compete for available resources.


Feature requirements

Isolated storage uses evidence to specify a unique storage area for use in an application or component. Filesystems refer to files by explicit pathnames (e.g. c:\temp\myapp\temp.txt), however, there is nothing to prevent other code from accessing “myapp”’s files. Isolated storage provides true isolation in that the identity of “myapp” (detailed below) uniquely determines the root of a virtual file system that only “myapp” can access. Thus, rather than many applications and components sharing a common resource like a file system (or the registry), instead each has its own file area inherently assigned to it, and fully isolated from other applications making it essentially private.

This approach provides benefits to both programmers and administrators, and should result in better applications for users. By using application identity information to define a unique data compartment for the application and/or component, IsolatedStorage makes administration easier in two distinct ways. First, applications have a standard place to store their data – programmers don’t have to use ad hoc techniques to take a corner of the file system for their use, this also lets administrators remove data for an application when uninstalling it. Also, from a security point of view, by isolating each application’s data into compartments, this approach protects against other code getting at another’s data.


Types of isolation

Basic kinds of identity used in scoping isolated storage are as follows (with details following):

  1. user – authenticated user identity, or a unique “anonymous” user if none.
  2. domain – domain isolation is based on evidence associated with the application domain:
    • web applications identity is taken from their URL site (e.g. “http://www.microsoft.com”);
    • shell hosted code identity is based on the application directory path (e.g. “C:\office”).
  3. assembly – code is identified cryptographically by strong name or publisher
    • strong name – strong name identity (e.g. Microsoft.Office.* or Microsoft.Office.Word)
    • software publisher – software publisher identity (based on public key, like identity perm.)

NOTE: The terms ‘domain’ and ‘assembly’ are used to denote these specific identities in the remainder of the document.

First-time readers should not be daunted by the variety of identities used to scope data compartments. The “80%” case anticipated for all V1 isolated storage use is just going to be “user+domain”, that is: by application for a particular user. So when you use “app1” it has its store, and “app2” has its; other users of these same applications even on the same machine will get separate places for their data.

Machine containment

All isolated storage is contained by machine – that is, storage is not shared between machines, nor is data accessible across machines with this storage feature. (Inter-machine storage can be constructed on top of the basic functionality provided, but is not a goal of this feature itself.)

Machine containment is always implicit, by virtue of using storage facilities local to a given machine. The identity of a particular machine is never exposed by use of this feature.

Clarification: machine containment is described here in the sense that all user containment is implicitly scoped to the local machine code runs on.
For example, if Alice and Bob use an app on the same machine that app would use separate isolated stores for them. Then if Alice runs the same app on another machine she will get a different store there that is different from that on the first machine. This is the sense in which machine containment encompasses user.

<PUNT!> This spec uses the term “machine” compartment to refer to only machine containment irrespective of user, as opposed to what is actually machine+user containment which is denoted by the less cumbersome term “user”.

User containment

Isolated storage is contained by user in all but the most highly trusted level (level 3, below) of usage. User containment is important to ensure that information is never leaked from one user to another by means of this feature. The location of storage for user isolated storage is determined by the platform OS notion of user identity for the process the code is running in at the time the store is opened. Impersonation changes the notion of ‘current OS user’ dynamically.

User containment is always implicit, by virtue of using storage associated with a given user. The identity of a particular user is never exposed by use of this feature. On the Windows platform, user’s profile directory is where IsolatedStorage data is stored. (See data storage section following for details.)

On platforms that do not support different user login identities there is only one common user identity: effectively user containment is synonymous with machine containment in this case. Nevertheless, from a permission enforcement point of view this single user identity is still a user identity – in other words on these platforms, level 1 (below) isolated storage permission will allow use of this single (same as machine) user container.

Domain containment

Domain containment is based on evidence associated with the application domain the code is running in. As noted above, it is important to provide this IsolatedStorage containment even if based on a weak identity such as DNS name or installed location so as not to require digital signing in order to use the feature.

For web applications (a.k.a. “IE host”) hosted by a web browser, the URL determines the identity of the domain. Hosts such as IE associate URL evidence with the application domain that runs code from a web site. The domain identity for purposes of IsolatedStorage data compartmentalization is the full URL truncated at the rightmost slash. For example, if the URL evidence is “http://www.contoso.com/apps/myapp/home.htm” then the domain identity is derived as “http://www.contoso.com/apps/myapp”. Using the truncated URL rather than just the site allows a number of independent applications to coexist within a single site without conflicting storage use. While this does allow a malicious site to more easily use up storage, an intent attacker could still use different DNS site names rather easily in any case in which case there is no reliable way of knowing that it is really one entity consuming overly much storage. (To be blunt, we aren’t going to be able to deter malicious storage consumption very effectively in any case, so we should maximize benefit to real applications in the design.)

For shell hosted (a.k.a. “EXE host”) applications – typically, applications installed in a machine and launched – the domain identity is similarly derives from the application directory path. For example, if the main exe is run from the path “C:\office\winword.exe”, then truncation at the rightmost ‘\’ yields the domain identity “C:\office”.

Define app directory as: AddData(“AppDomainConfig.APPLICATION_BASE”)

The “friendly name” of the domain (AppDomain.GetFriendlyName) is associated with stores to help users identify the application that uses a certain store for purposes of administrating or clean-up of old information no longer needed. (We cannot expect users to recognize applications by their base URL.)

Assembly containment

In addition or instead of containment by domain, signed code assemblies may also have their own data storage compartment. Assembly identity is based on cryptographic digital signature, either by strong name or by software publisher. Generally speaking, strong name identity is most flexible, allowing both compartments unique to a particular component or broader namespaces that allow a suite of components to use a common share provided all are based on the same signing key. Software publisher is a simpler model where only the key identity defines the containment.

Strong name identity (e.g. Microsoft.Office.* or Microsoft.Office.Word) is valid for code that is signed with the private key corresponding to the public key forming the root of the strong name namespace. Separate data compartments can be used based on full or partial forms of the strong name of a component. For example, a assembly signed with strong name “first.second.third.fourth” can used any or all of these IsolatedStorage compartments: “first.second.third.fourth”, “first.second.third.*”, “first.second.*”, “first.*”, or “*” – all of which are rooted to the key associated with the digital signature of the code.

Software publisher containment identity is based on the public key corresponding to the private key used to sign the code (like the publisher identity permission). Only the key is used to determine identity: the signing certificate and whether or not there is a trust relationship (a la WinVerifyTrust) established is not relevant to this feature.

If an assembly has both strong name and software publisher identity, then that composite identity is used to define assembly containment. In other words, only assemblies with the same combination of both kinds of identity can access the same store.

Usages allowed by the permission

These basic identities enable the following types of isolation, listed in increasing level of trust (strong name and software publisher are basically same level of trust):

Level | Isolated storage data containment | Security impact
0 | No isolated storage use allowed | None
1 | user + domain | Data contained by domain Potential denial-of-service
 | user + domain + assembly |
2 | user + assembly | Potential app-to-app leakage
3 | ALL users, domains, assemblies | Potential for total compromise

Notes:
(1) all isolated storage containment implicitly includes “by machine” isolation

It is important to note that site identity is very weak (subject to spoofing), while cryptographically based code identities are very strong. Application code is expected to use appropriately strong identity to provide sufficient protection for the data being stored: always use strong identities for valuable or private data.

Here are a few scenario sketches to illustrate and motivate the need for these various types of storage:

  1. (user + domain) Typical use includes storing user parameters such as preferences of user data for use by a specific application. Data can only be accessed within the context of the same application and only when run by the same user. For example, an application might store user’s identifying information, save the state from the last time the application was run, use the store to record an audit trail of past application usage, and save preferences for options of the application.
  2. (user + domain + assembly) Similar to scenario above, but with a third party assembly that wants to keep a private data store. The assembly is private in the sense that only code in that assembly may access the data, yet the data store is also scoped by the application it runs in so that the assembly does not represent a potential leak exposing data to other applications. In particular, this addresses the web site isolation requirement of a web browser, where (without special trust) data of one site should never be leaked to another at the client. The scenario for this feature is similar to the above, with the third party code having its own container within the scope of the application in which it is being used.
  3. (user + assembly) In contrast to 2 above, this provides a data store for the assembly that is accessible in any domain context. The per-assembly data compartment requires additional trust as it potentially provides a “tunnel” between applications that could compromise the data isolation of applications, in particular web sites. Typical usage would be for user data that applies across multiple applications – for example, license information, user’s personal information (name, authentication credentials, etc.) that is independent of application.

Security risks of IsolatedStorage use

This section describes the security risks associated with the varying types of usage of the IsolatedStorage feature as described. Default policy will be assigned according to the risk / utility balance of each kind of usage.

Level 0: the feature is disabled so trivially there is no associated risk.

Level 1: data containment is scoped by domain identity

Stored information is only available within the same domain, as determined by the domain identity associated with the application domain. Signed assemblies within the application still have data compartment scoped by the domain identity so can’t bridge outside of the application via this feature. The remaining risk is denial of service whereby excessive amounts of storage are consumed, preventing use of storage for other use. For V1 we don’t plan to implement any kind of protection for this sort of denial of service attack; users should be able to handle this inconvenience by reclaiming the storage via normal or possibly manual means.

Note that strong protection against this sort of attack is extremely difficult without severely limiting the utility of the basic feature. For example, web browser “cookies” attempt to prevent denial of service attacks by restricting the size of each cookie (4kb) and the number of cookies (20) a single web site can store. Unfortunately it is easy to circumvent this protection by simply using multiple web sites in one attack.

There is a potential denial-of-service attack with this level of feature usage in that malicious code could intentionally consume large quantities of storage. By using numerous domain identities (such as by using several web site addresses, all owned by the attacker) an arbitrary amount of storage can be consumed, beyond the limits set for one domain identity. The only defense against these nuisance attacks are: restrict usage of level 1 to somewhat trustworthy applications, explicit user action to clean out storage, or possibly by an administrative tool that uses some heuristic to detect maximum storage usage and delete it.

Level 2: data containment is scoped by user (and assembly identity)
Information store is scoped by the assembly identity and is independent of what domain it is used in. An application using a third party assembly that has this permission potentially risks information leaking from the application.

This is an issue of the application trusting the assembly it uses. Of course security policy must also trust the code to allow this kind of usage, but there is a separate issue of applications trusting their assemblies independent of that. Ultimately, by choice of the components they call applications implicitly make these trust decisions.

Level 3: unrestricted (any user, any store)
This level potentially allows access to any store, even another user. A high level of trust by policy and by applications, as described in the preceding paragraph, is necessary to ensure that no security weaknesses are exposed by use of this feature.

OS level permissions to access store

Code must have all necessary native platform OS rights to access any stores. For example on Windows 2000, user permissions to read/write the files underlying isolated storage must satisfy the Access Control List (ACL) restrictions of the file system.

Within the NGWS platform, applications may expect to always have OS rights to access isolated storage within the current user scope. However, applications that do (platform-specific) impersonation need to be responsible for ensuring that they have the proper rights in the new user identity to use storage.

Administrative code that uses the highest “Level 3” permission also needs to satisfy OS restrictions, which may require OS-dependent code or setup.


Quotas

Quotas are enforced to address potential denial of service problem of isolated storage from a malicious or buggy site wasting or filling up the system disk space. All quotas are in bytes.

Quotas apply to bytes of file space as well as a “byte cost” (may exceed actual storage consumption of the implementation, but should never underestimate) for directory and other information retained within the store.

Quotas will be conservatively rounded off to a block size. Block size is used to speed up allocation and quota management. The block size is implementation dependent.

Administrative quotas

These quotas apply to all uses of isolated storage; an attempt to write data that exceeds any applicable quota will result in an exception. Permissions may further limit sizes for certain code, but can never allow code to exceed established administrative quotas.

  1. <PUNT!> Quota on the overall size of the machine store
  2. Quotas on the overall size of each user’s total store
  3. <PUNT!> Minimum remaining disk space

See section 3.1 on how the machine administrator can configure these.

Permission quotas

Quota granted in permissions further restrict specific code, up to be not exceeding administrative quota.

Note: a single entity may use any number of distinct site or signature identities so these quota are merely a guideline for well-behaved use of the feature; the global store limits are what truly enforce against malicious denial of service attacks.


Lifetime

Lifetime management features help keep actual consumption of isolated storage within bounds set by quotas with a minimum user/admin intervention, working as automatically as feasible (given a simple implementation) to efficiently utilize storage. From time to time the system reclaims expired or expendable data.

Isolated storage data lifetime is managed by the system in units of domain (application) identity scopes.

Data can have an expiration date – or not – and can optionally be marked for guaranteed retention as detailed below. The following table shows how these features combine in use.

Expiration | Retention | Description | Reclaimable?
in future | Not guaranteed | keep around unless space runs out | second choice
expired | (any) | exceed expiration date | first choice
in future | Guaranteed | must be kept until expiration | no
none (permanent) | Guaranteed | must be kept indefinitely | no
none | Not guaranteed | this is not possible |

Expiration

Expiration dates are set to allow the system to automatically reclaim old information no longer needed. There are two purposes for having expiration of data: applications may volunteer to have data removed after a given date, or restricted permission may force an application to only be able to keep persistent data in the isolated store for a certain maximum length of time. Either way, expiration is important to prevent a continual buildup of old data that may no longer be used.

Expiration is set by the application code in terms of days since last use. Any read or write access resets the expiration for another N days in the future, but the number of days since last use remains the same.

We can implement expiration without needing a background service to actively reclaim storage. Expiration is voluntary rather than mandatory, meaning that if storage expires on a certain day (say, a holiday) we don’t necessarily have to remove it then; rather we can expire the data when the store is next used in a more “lazy” fashion. Also a simple tool that passes over the stores deleting expired data can easily be fired off periodically from existing OS services, say at 3am every day.

Maximum expiration

The machine administrator can set the maximum number of days information can be marked for expiration on a machine-wide basis. Only data marked for retention (see next section) is exempt from this limitation. For example, if the parameter is set to 30 days then an attempt to set a longer expiration will result in the smaller value being used. See section 3.1.

Guaranteed retention of data

Given permission to do so, data in isolated storage may be created marked for guaranteed retention. Guaranteed retention can be used with an expiration date or without (meaning it never expires). Guaranteed retention data is not subject to administrative maximum expiration date restriction described above.

This storage can only be deleted by the code with this identity and necessary permission, or by explicit user/admin action. However, this data will be deleted when expiration date is exceeded.

All other storage is “expendable” and subject to reclamation before expiration if quotas are exceeded.

Reclamation

Reclamation is the process of deleting data in a store to make room for more, both: old expired data can be automatically deleted to make room, or as a next step any expendable (but not guaranteed retention) unexpired data.

Reclamation happens when:

  1. a store reaches total maximum quota and more room is needed
  2. automatically as a storage consumption level for a store is exceeded (details follow)
  3. as an explicit administrative operation (a simple utility)

When a write to isolated store exceeds some quota reclamation should be attempted before raising an exception indicating an error. Only after attempted reclamation fails to get back any storage should the exception be fired. This will potentially cause a significant performance hit to the unlucky application that triggers this, however until we can implement a more sophisticated background reclamation service the slow-down is preferable to an error. Implementation should remember when reclamation has been done and minimize cost of repeated attempts (for instance, if no new expendable storage is created since the last reclamation any subsequent attempt is sure to not find anything and can be completely skipped).

We should build in a simple heuristic to do automatic reclamation (otherwise, clean up of expired data only happens when store is full – resulting in often-near-full stores which is not a desirable common state).

Here’s one simple algorithm (with experience we can adjust the parameters):

Used = bytes used; Trigger, Step = variables; Max = store quota in bytes

  1. For a new store set Step = min ( Max*0.25, 10MB )
  2. For a new store set Trigger = Step
  3. On close of each substore, check if Used > Trigger, continue if so
  4. Reclaim expired storage in the entire store
  5. If Trigger –Used< Step/4 (i.e. not much below Trigger) then
    set Trigger = Trigger + Step, else leave Trigger as is
  6. Go to 3 to wait for another chance

Any store that is currently in use is always exempt from reclamation.


IsolatedStoragePermission Overview

We define a permission object to control what code is permitted to use IsolatedStorage, and if so to what degree. Future versions may include other restrictions in this permission, however for Beta1 we only:

  1. Allowed usage (what identities can be used to scope data compartment)
  2. storage quota size
  3. maximum allowed expiration of data
  4. Ability to retain data indefinitely

This is a code access permission and all callers (subject to overrides) must have the appropriate permission for the applicable usage level. Note that all callers do not need to have the signature evidence, as described in detail – see section 4.1.

The granted permission for Quota of the assembly or domain is all that applies to a given isolated storage use. It is not necessary for a full stack walk to have every permission satisfy the quota.

Other aspects of the isolated storage permission are stack walked and all callers must have necessary permission to do operation.

Usage allowed

IsolatedStoragePermission is a standard permission object in System.Security.Permissions namespace that defines allowed usage of the IsolatedStorage feature.

The permission state consists of these level (following) corresponding to the permission granted to code to use this feature, and a Boolean flag designating if the code is allowed to mark data to be retained indefinitely or not (see section 2.3.2).

  1. (levels 0, 1, 2, 3, per section 2.1.5)
  2. Administrative access: allows browsing & deletion of entire user store, but not read access other than own domain/assembly identity
  3. Super access: full access to any store of the user store, regardless of identity of the domain/assembly itself (includes ability to enumerate contents of the IsolatedStorage data store)

Data retention allowed

This Boolean flag in the permission corresponds to having permission to request guaranteed retention of data, as described in section 2.3.3.

IsolatedStoragePermission API

IsolatedStoragePermission defines the abstract class for any kind of isolated storage use: IsolatedFilePermission derives from this and defines permission to use isolated storage within the file system.

enum IsolatedStorageContainment {
// Issue: do we order these as below or allow combinations of
//        separate levels of user use?
None = 0,
DomainIsolationByUser=1,
AssemblyIsolationByUser=2,
AdministerIsolatedStorageByUser=5,
UnrestrictedIsolatedStorage=7
};
abstract class IsolatedStoragePermission
{
   // c’tor
   IsolatedStoragePermission (
            enum IsolatedStorageContainment UsageAllowed,
            Boolean PermanentData);
   // properties
   long UserQuota {set;get;}
   long ExpirationDays {set;get;}
   bool PermanentData {set;get;}
   enum IsolatedStorageContainment UsageAllowed {set;get;}
}
class IsolatedStorageFilePermission : IsolatedStoragePermission
{}

The permission object defines the highest level of usage allowed for the code. Thus, CheckDemand (IPermission d) checks that d.UsageAllowed <= this.UsageAllowed.

The Quota limit defines maximum size of isolated storage use by the code granted the permission. For files the quota is in bytes (for other types it may be in different units).


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Administration

© 2000 Microsoft Corporation. All rights reserved.


Configuration of Isolated Storage

The following parameters are configurable by the machine administrator: globally and on per application basis using the configuration system.

Parameter | Default | Description
UserRoot | (tbd/DEV) | Root of the user store in file system
UserQuota | 20MB | Maximum overall size of each user’s total store

Post-V1

MinRemaining 10MB Minimum remaining disk space


Administration of stores

Use the StoreAdm command line tool to managed isolated storage for the currently logged in user. The tool provides three simple functions that are typically used one at a time, however if two or more are specified they will be performed in the order they appear on the command line.

  • LIST: displays existing isolated storage in use
  • RECLAIM: explicitly checks for and frees expired isolated storage – this simply gives the system an opportunity to cleanup expired resources, and never deletes data unless it has expired
  • REMOVE: deletes all existing isolated storage – useful to free up disk storage if managed applications themselves do no properly delete storage no longer in use


Isolated storage API

Programs use isolated storage through a few classes that implement the feature.

The classes correspond to the following functional aspects of using isolated storage.

  1. Mapping from identity to the associated isolated store
  2. Operations on the isolated store
  3. Using files within an isolated store
  4. Managing isolated stores
  5. Handling out-of-space conditions

Isolated stores are referenced by modes and identities as described below – a new store is created the first time if one does not already exist, otherwise the existing store with all its content is returned.

Domain and assembly identities, as well as permissions, are only checked at the time the reference to the isolated store object itself is made (typically the constructor). Thereafter, it is possible for code to pass around the object reference to other code that may not itself have the corresponding identity or not have permission. Protecting references to these objects is the responsibility of the code. This design decision is consistent with the System.IO.File classes (which only check permissions at open-time), both choices resulting mainly for performance concerns.


Mapping from identity

The scope of an isolated storage data compartment is derived from the application domain identity, the code identity, and the mode of containment prescribed by the code. This mode of containment is the key to how the identities are combined, and deserves detailed description.

Data compartments are determined by identity, and the store that can be used for data is what constitutes the compartment. Data isolation is always per-machine, and can also be a combination of per-user, per-domain, per-publisher, per-strong-name (although not all possible mixes of these are supported). The value IsolatedStorageScope defines these isolation factors, and Boolean OR values represent combinations. Each valid combination is separately described below.

enum IsolatedStorageScope {
      User=1,
      Domain=2,
      Assembly=4,
      };

Access to an IsolatedStorage object via the constructors as described below constitutes “use of the store” for purposes of expiration accounting.

Mode: user + domain + assembly

Within an domain scope, signed assemblies can have their own isolated stores. First off, this gives assemblies a unique place to store data without having to worry about conflicting storage locations with other code. For third parties, the assembly has a separate store that is not accessible to other application code – ensuring clean software engineering isolation so that other code can’t conflict, plus a certain degree of privacy (see general discussion on limitations).

Note that signed assemblies by virtue of being included in the application domain can use the general domain scoped store as well. Assemblies may have some isolation from the rest of the application, but no “protection” in the other direction is possible.

The constructor API for the IsolatedStorage object based on assembly digitally signed identity within an domain, scoped per-user is as follows.

store = IsolatedStorageFile.GetUserStoreForDomain ();

The form above uses the public key identity of the signed code invoking the constructor. Only the code identity of the immediate caller is relevant. If the code is not signed (or signature is not valid) an ArgumentException results. (We could make it work like User|App but if code expects a signature identity we need to let it know.)

Successfully constructed object represents that assembly’s store – per domain – within which it can store its own data. Permission required to use this form of isolated storage is DomainIsolationByUser (or greater).

Mode: user + assembly

Independent of domain scope, signed assemblies can have their own isolated stores. This store allows code to access the same storage from any number of domains, and as such carries an increased security requirement for use.

The constructor API for the IsolatedStorage object based on assembly digitally signed identity only, scoped per-user is as follows.

store = IsolatedStorageFile.GetUserStoreForAssembly();

The form above uses the public key identity of the signed code invoking the constructor. Only the code identity of the immediate caller is relevant.

Successfully constructed object represents that assembly’s store – usable in any domain – within which it can store its own data. Permission required to use this form of isolated storage is AssemblyIsolationByUser (or greater).


Operations on the isolated store

This section describes operations on the IsolatedStorage object once constructed as described above, through which data can be stored in these compartments.

The expiration time (in days, subject to admin restriction) can be set by the property ExpirationDays. Use the constant NoExpiration to designate data that should not expire (also subject to restriction).

Guaranteed retention of the store can be requested by setting the flag so named. Code must have the corresponding IsolatedStoragePermission granted to set this.

class IsolatedStorageFile
{
// C’tor
   IsolatedStorageFile (IsolatedStorageScope Mode);
// Members
   ulong MaximumSize {get;}   // Maximum allowed quota
   ulong CurrentSize {get;}   // used quota
   Object DomainIdentity {get;)   // domain isolation
   Object AssemblyIdentity {get;)// assembly isolation

// Methods
   Remove();   // removes this store
   Close();   // closes the store & all files in it
}


Using files within an isolated store

IsolatedStorageFileStream, a subclass of IsolatedStorage, handles isolated storage for the file system. Constructed based on domain/code identity, this class exposes a virtual file system rooted in the associated data compartment store.

IsolatedStorageFileStream provides methods such as Open, CreateDirectory, and so forth that provide exactly the same interface as their System.IO.File counterparts. Specifically, nearly all File methods that take path names as arguments are supported (see list below); only relative paths are supported – interpreted as a hierarchy of files and directories within the corresponding store.

class IsolatedStorageFileStream : FileStream
{
// C’tor
IsolatedStorageFileStream ( String path, FileAccess mode );
// Methods
      // mirrors File methods that take paths as args:
      //   Copy, Create, CreateDirectories, CreateDirectory,
      //   Delete, Exists, GetDirectoryNames, GetFileNames,
      //   GetFiles, OpenRead, RemoveDirectory, Rename

      }

Thus, to open a file called “options” in an domain’s own isolated store and then write options information into it with 60 day expiration, the code sequence is:

IsolatedStorageFileStream stream;
stream = new IsolatedStorageFileStream(“filename”, FileMode.OpenOrCreate);
stream.Write(data);
stream.Close();


Handling out-of-space conditions

Isolated storage can only be used within the limitations established by administrators and policy, and within the constraints of available backing storage. When an attempt to write data exceeds any of these limitations (after all available space has been reclaimed) an exception is raised and the operation fails.

Generally there will be no way for applications to recover when this happens. For guaranteed retention stores this is identical to the classic full-disk problem; for other uses isolated storage is always subject to expiration so the application needs to be able to continue even if all storage is lost.

To aid applications handling these potential space problems, some simple aids are provided. The current and maximum size of a store can be determined with the get-only properties IsolatedStorage.CurrentSize and IsolatedStorage.MaximumSize. The maximum is not a guarantee, but simply a measure of the space available at the time: concurrent use of isolated storage means it is always possible that space may be consumed between checking maximum size and actually attempting to use it. However, the maximum size can reasonably be used as an upper limit on available storage.


Managing isolated stores

Unlike the forgoing sections that cover normal usage of the isolated storage feature by typical applications, this section describes the APIs used by typically by tools that enable users and administrators to managed isolated storage.

Management functions are separated into two levels: the first is intended to enable normal management of isolated storage data, the second is completely unrestricted access to any data in isolated storage. By normal management we mean common operations that cover a broad set of data across domains and assemblies:

  1. browse all isolated data store to see what domains are using how much storage
  2. tools to explicitly delete some or all of isolated storage
  3. tools to do more sophisticated storage management than the basic reclamation service implemented by the system
  4. NOT ability to read/write data owned by domains / assemblies

Enumeration of stores

Special APIs allow a program to enumerate all stores within isolated storage, for the current user, for the machine, or even for all users on the machine. From the enumeration of the stores, expiration dates and so forth can be accessed or changed. Use of stores via these APIs does not constitute “use” for purposes of expiration.

The following APIs provide these functions:

IEnumerator IsolatedStorageFile.GetEnumerator(IsolatedStorageScope );

The above APIs respectively require the following permissions (or greater): AdministerIsolatedStorageByUser, UnrestrictedIsolatedStorage. Access to other users isolated storage may be blocked by native operating system security independent of the security mechanisms described here (just as even with FilePermission some files may not be accessed due to ACL settings).

When enumerating stores across all users, an API is required to tell for a given store what user it is associated with. Code that has UnrestrictedIsolatedStorage permission may use IsolatedStorage.User property to get a string that represents the user.

Trusted code can use these APIs to look through the stores of many domains by enumeration, or use the API in 4.5.3 below to access specific stores directly.

It is important that opening stores within an enumeration by a fast operation – specifically, it should involve minimal I/O to open successive stores. (In the unrestricted case, enumeration across different users will involve different files; this comment is meant to apply to enumeration of files within a single user scope.)

Deletion of stores

(Note: contrary to intro above, this may be used by apps in some cases)

The following API deletes an entire store (including deletion of all contents):

static IsolatedStorageFile.Remove(IsolatedStorageScope)

The parameter specifies whether it is important to delete all the data immediately (true), or if the storage can simply be freed up for later reclamation (false). (If the intent is to remove the data from the system then the former is appropriate; if merely to make space then the latter is most efficient.)

There is no permission required for this operation – permission checks when the store is opened protect the store. (Without this operation, any code accessing the store would be able to delete all data inside it anyway.)

Accessing isolated storage by evidence

The last API to enable complete use of isolated storage by highly trusted code is the creation of new stores based on evidence. Use of this API requires both IsolatedStoragePermission.UnrestrictedIsolatedStorage and SecurityPermission.ControlEvidence permission.

This API enables a tool to be written to migrate isolated storage from machine to machine. Stage one enumerates all stores and copies their state and contents; stage two reads that information on another machine, creates corresponding stores and populates them with data.

class IsolatedStorageFile
{
   static IsolatedStorageFile GetStore(
      IsolatedStorageScope,
      Object DomainIdentity,
      Object AssemblyIdentity);
}


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Design notes

© 2000 Microsoft Corporation. All rights reserved.


Storage model

The basic storage model is that identity determines a unique virtual file system root, and the domain or assembly may create files and folders therein to store data.

Files and directories within the isolated store appear just like those of some directory of a conventional filesystem. The identity that determines the scope of the isolated storage is effectively the “path” of the root of this store, and below this virtual root the isolated storage APIs expose files and directories just like the file classes expose those of the filesystem.

All isolated storage exists in the filesystem under a single directory (configurable, default something like %WINDIR%\IsolatedStorage\). Under here, the storage is laid out (functionality, actual implementation may differ) as follows:

Machine store
Entity #1 scope substore (user or publisher or strong name)
files and folders
Entity #2 scope substore
files and folders

User ‘A’ store
Entity #1 scope substore
files and folders
Entity #2 scope substore
files and folders


Location of store

Administrator configuration can specify the base location for as detailed in section 3.

Default location of store for the per user store is the user profile directory. User login accounts that do not have an associated profile directory may not use isolated storage by default. However, since the location of isolated storage root in the file system is configurable, administrators may create a place in the file system and administratively specify it for isolated storage (ensuring that the ACLs are set right).


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

NGWS Runtime Performance Counters

© 2000 Microsoft Corporation. All rights reserved.


Performance Counter Overview

This document describes the set of performance counters that the runtime publishes. These counters can be read using APIs that will be provided in the future by the Class Library and can be viewed using the Performance Monitor tool on platforms on which the tool is available. The runtime performance counters will always be available in the client and the server builds.

The runtime performance counters are grouped into categories or performance counter objects. The following sections describe the categories and the counters available in each category.


Runtime Performance Counter Categories

Title | Definition | Default Counter
Context and Remoting | A context is an ordered sequence of properties that define an environment for the objects resident inside it. Contexts are created during the activation process for objects that are configured to require certain automatic services such as synchronization, transactions, just-in-time activation, security, etc. Remoting is the mechanism used to make method calls between object instances across a boundary. Boundaries include calling between contexts, application domains, processes, and machines. | Remote Calls/sec
Exception | Exceptions are violations of semantic constraints of an implementation language, the runtime, or an application. | Exceptions thrown
Interoperability | Interoperability deals with how the runtime interacts with all external entities, including classic COM, external libraries, and the operating system. | Managed to unmanaged transitions
JIT Compilation | Just-in-time (JIT) compilation is a compilation method used to compile methods to native machine language immediately before execution of the method. | Compiled method count
Loading | Loading is the process used to locate a binary form of a class and constructing from that binary form an object that represents the class. | Class count
Lock | A lock or mutex is data associated with each object in the runtime that a program can use to coordinate multithreaded access to the object. | Total contention
Memory | Memory is the set of storage locations that a program uses to store its variables. | Garbage collections
Networking | Networking is a set of services that allow a program to access remote resources over a network. | Connections active
Security | Security is the set of mechanisms used to provide controlled access to resources. | Runtime check count
Thread | A thread is an independent unit of execution with its own working memory that operates on values and objects that reside in a shared main memory. | Recognized by the runtime


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Runtime Performance Counters

© 2000 Microsoft Corporation. All rights reserved.


Context and Remoting Counters

Title | Definition
Channels | The current number of channels loaded in the current application domain.
Context Proxies | The number of context proxies. A call from an object in one context to an object in another context will go through a context proxy and will be affected by the policy that the combined context properties enforce. The number of context proxies should be small in a well behaved application.
Context-Bound Classes Loaded | The number of context-bound classes that are currently loaded. These classes have overhead associated with them. They are typically used for transaction isolation or propagation. Note: This counter is not yet implemented.
Context-Bound Objects Allocated/sec | The rate at which context-bound objects are allocated. Note: This counter is not yet implemented.
Contexts | The current number of contexts.
Remote Calls (Total) | The total number of remote calls. Remote calls are calls between processes, between application domains, or between machines.
Remote Calls/sec | The remote call rate. Note: This counter is not yet implemented.


Exception Counters

Title | Definition
Average Stack Depth | The average depth of the stack for exceptions. The depth of the stack is the distance between the throw point and the catch point. The calculation of the depth of the stack takes into account only the size of the managed frames. The closer the catch point is to the throw point, the better the performance.
Exceptions Thrown (per sec) | The rate at which exceptions are thrown by the runtime.
Exceptions Thrown (Total) | Total number of exceptions thrown by the runtime since the program began executing.
Filters Executed/sec | The rate at which exception filter blocks are executed by the runtime. A filter block determines contains code that is executed to determine if the catch block of an exception handler can handle a particular kind of exception.
Finally Blocks Executed/sec | The rate at which exception finally blocks are executed by the runtime. A finally block contains code that is always executed after a try block is executed.


Interoperability Counters

Title | Definition
Classes Exposed to COM | The number of classes that are currently exposed to COM through a COM Callable Wrapper (CCW) class.
%Execution Time Marshaling | Percentage of execution time spent in marshaling. Marshaling time includes time spent in marshaling arguments and return values. Time spent in custom marshaling is not counted.


JIT Compilation Counters

Title | Definition
%Execution Time Compiling | Percentage of execution time spent in JIT compilation.
Code Pitching/sec | The rate in bytes/sec at which JIT-compiled code is unloaded by the runtime.
JIT Compiled IL Code (Current) | Number of bytes of IL code that were compiled by the JIT compilers for methods that are currently loaded.
JIT Compiled IL Code (per sec) | The rate in bytes/sec at which IL code is compiled by the JIT compilers.
JIT Compiled IL Code (Total) | Total number of bytes of IL code that were compiled by the JIT compilers since the program began executing.
JIT Failures | The total number of methods that the standard JIT compiler failed to JIT. The standard JIT compiler normally fails to JIT a method when it is too complex.


Loading Counters

Title | Definition
%Execution Time Loading | Percentage of the execution time spent in loading assemblies, application domains, classes, etc. Note: This counter is not yet implemented.
Application Domains Loaded (Current) | The number of application domains that are currently loaded.
Application Domains Loaded (per sec) | The rate at which application domains are loaded.
Application Domains Loaded (Total) | The total number of application domains that were loaded since the program began executing.
Application Domains Unloaded (per sec) | The rate at which application domains are unloaded.
Assemblies Loaded (Current) | The number of assemblies that are currently loaded.
Assemblies Loaded (Total) | The total number of assemblies that were loaded since the program began executing.
Assembly Search Length | The average search length for assemblies.
Class Load Failures/sec | The rate at which classes failed to load.
Classes Loaded (Current) | The number of classes that are currently loaded.
Classes Loaded (per sec) | The rate at which classes are loaded.
Classes Loaded (Total) | The total number of classes that were loaded since the program began executing.
Loader Heap Size | The total size of the heap used by the loader.


Lock Counters

Title | Definition
Contention (per sec) | The rate at which threads in the runtime attempt to acquire a lock unsuccessfully.
Contention (Total) | The total number of times threads in the runtime have attempted to acquire a lock unsuccessfully since the program began executing.
Queue Length (Instantaneous) | The average number of threads that are currently waiting to acquire a lock.
Queue Length (Total) | The average number of threads that waited to acquire a lock or are waiting to acquire a lock. The average is computed over all waits on locks since the program began executing.


Memory Counters

Title | Definition
%Execution Time Garbage Collecting | Percentage of execution time that is spent garbage collecting.
%Free | Average ratio of unused to used space in allocation contexts at garbage collection time. Note: This counter is not yet implemented.
Collections (Gen0) | The number of garbage collections that happened in generation 0.
Collections (Gen1) | The number of garbage collections that happened in generation 1.
Collections (Gen2) | The number of garbage collections that happened in generation 2.
Finalization Survivors | Number of objects with executed finalizers. If this number is high, you can reduce the number of finalizers or call finalizers explicitly.
GCs (Large Object Heap Induced) | The number of garbage collections in response to large object heaps. The allocation of large objects is handled specially. These garbage collections always occur in generation 2. Note: This counter is not yet implemented.
GCs (%Program Induced) | The percentage of garbage collections that are programmed induced.
Handles | Number of handles in use. Handles are used to implement references.
Heap Size (Gen0) | The heap size in kilobytes for generation 0.
Heap Size (Gen1) | The heap size in kilobytes for generation 1.
Heap Size (Gen2) | The heap size in kilobytes for generation 2.
Heap Size (Large Object) | The heap size in kilobytes for large object heaps. The allocation of large objects is handled specially.
Memory Allocated/sec | The rate in bytes/sec at which memory is allocated by managed code.
Memory Survived (Gen0 to Gen1) | The average amount of memory that survived from generation 0 to generation 1.
Memory Survived (Gen1 to Gen2) | The average amount of memory that survived from generation 1 to generation 2.
Pinned Objects | Number of pinned objects caught by garbage collector. Pinned objects are objects that should not be a moved to a different location.
Sink Blocks | Number of sink blocks in use. A sink block is data that belongs to an object and is allocated on demand. Synchronization primitives use sink blocks.


Networking Counters

Title | Definition
Bytes Received | The total number of bytes that were received by the program over the network since the program began executing.
Bytes Sent | The total number of bytes that were sent by the program over the network since the program began executing.
Connections Active | The number of network connections that are currently active.
Datagrams Received | The total number of datagrams that were received by the program over the network since the program began executing. Datagrams are used to implement connectionless networking.
Datagrams Sent | The total number of datagrams that were sent by the program over the network since the program began executing. Datagrams are used to implement connectionless networking.


Security Counters

Title | Definition
%Time Signature Authenticating | The percentage of overall time spent in authenticating signatures.
Link-time Checks | The total number of link-time checks since the program began executing.
Remote Security Stack Depth | Average depth of security stack propagated via remoting. Note: This counter is not yet implemented.
Runtime Checks (%Execution Time) | Percentage of execution time spent in runtime checks.
Runtime Checks (Total) | The total number of runtime checks since the program began executing.
Stack Walk Depth | The instantaneous average depth of the stack traversed for security checks.


Thread Counters

Title | Definition
Threads (Logical) | The current number of logical threads.
Threads (Physical) | The current number of physical threads.
Threads Recognized (Current) | The number of threads that are alive and have executed code under the control of the runtime.
Threads Recognized (Total) | The number of threads that have executed code under the control of the runtime since the program began executing.


Performance Counter Security Considerations

A program needs to have a PerfMon Counter Permission to be able to connect to performance counters published by another application. The details are TBD.


Performance Counter Issues

  • Thresholds are useful. For example, an upper threshold for the total thread count might indicate a number which when exceeded will result in significant performance degradation. Thresholds need to be described in detail in the documentation.
  • Need to investigate WMI support on Win98 and Win2000.
  • The initial implementation will expose counters on a system-wide and per-process basis. In future, counters on a per application domain basis may be available.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

NGWS Runtime Security Permissions Specification

© 2000 Microsoft Corporation. All rights reserved.


Introduction to NGWS Runtime Security Permissions

This document is structured to answer the following questions:

  • what are permissions?
  • what are they used for?
  • how do you use them?

Permissions in the runtime are implemented as objects that describe a set of operations the can be secured for a given resource. A permission object describes operations or access subject to security control, it does not represent a right to do such operations or access in and of itself. Permissions are used by both application code and the runtime security system as the building blocks of the runtime security system in many ways.

  • code requests the permissions it needs in order to run
  • the security system policy grants permissions to code in order for it to run
  • code may demand granted permissions of calling code
  • code may override the security stack with assert/deny/permit-only

Standard permissions are defined by the runtime to secure resources that are exposed by the base class libraries that are a part of the runtime. These standard permissions should always be used to secure access to the resources corresponding to these permissions – for example, the FileIOPermission for file access, and so forth. Custom permissions may be defined to control access to other resources, and used in exactly the same manner standard permissions are used.

Most applications that use basic resources via the standard class libraries need only to request the right set of permissions which in turn need to be granted before the code can be safely run. (A separate policy spec describes the grant in detail.) Only code that exposes – and therefore must protect – resources that are controlled by permissions needs to use permissions other than simply request them.

In addition to resource protection there are identity permissions that provide a secure way to determine the identity of managed code, such as its software publisher by signature, web site of origin, and so forth. Identity permissions are granted based on evidence from the application code, and are never requested.


Standard permissions

The standard permission set supports the resource protection requirements of the class libraries that we ship in the NGWS runtime – the runtime base class libraries and NGWS frameworks classes. The following table summarizes the basic semantics of the standard permissions, followed by brief descriptions of the individual permissions (full details are in separate specs for each permission class):

EnvironmentPermission | Read/write access to environment variables
FileDialogPermission | Access to files and folders in the filesystem via open/save dialog only, ensuring user must OK the file access request for it to succeed.
FileIOPermission | Create/read/write/delete files and folders in the filesystem.
IsolatedStoragePermission | Provides storage space safely isolated by software identity and user.
ReflectionPermission | Access to names and type information about private class members via reflection
RegistryPermission | Create/read/write/delete access to Windows registry keys and values.
SecurityPermission Collection of simple permissions used by the security system
  • Ability to assert any permission that has been granted
  • Ability to P/Invoke to use unmanaged code
  • Ability to do advanced thread operations
  • Ability to execute (without this the code will not be allowed to run)
  • Skip verification (allows code to run without being verified)
  • For a complete list of permissions covered by SecurityPermission, refer to the sub-topic entitled ‘SecurityPermission’.
UIPermission
  • Use of unrestricted, “safe”, or embedded windows. A “safe” window is reliably labeled with title identifying application that is using it.
  • Use of clipboard: unrestricted, limited by web site, or disallowed.

Note: the net classes also define permissions for network access.


EnvironmentPermission

This permission defines access to environment variables. The permission distinguishes between the following two types of environment variable access.

  • READ: Read-only access to the value of the environment variable.
  • WRITE: Write access to the value of the environment variable. Also allows for the creation and deletion of environment variables.


FileDialogPermission

The FileDialogPermission controls access to system files based on end user selection within the system file dialog, ensuring that the user has knowledge and authorizes any files accessed by code. Code that does not have file permission can access files explicitly selected by the user via the standard open dialog. Having this permission allows code to get read-only access to file(s) selected by the end user. This allows the program to bring up the File Open dialog. The file that the end user selects is returned as a read-only file handle.

This permission allows code to request file access at run time via the standard file dialogs; when the user selects a file that action gives the code one-time access to the file. File access given via the file dialog is independent of the FileIOPermission. If code has both FileDialogPermission and FileIOPermission for a given file it may either use the file dialog UI or silently open the file such as through File class.


FileIOPermission

This permission controls the ability to access files and folders. The permission distinguishes between the following three different types of file I/O access. (Full spec is here.)

  • READ: Read-only access to the contents of the file or access to information about the file, such as its length or last modification time.
  • WRITE: Write access to the contents of the file or access to change information about the file, such as its name. Also allows for deletion and overwriting.
  • APPEND: Ability to create new files. Ability to write to the end of a file only, meaning you can not access bits before the end of the file when you opened it, only after.

All of these permissions are independent, meaning that rights to one don't imply rights to another. If more than one permission is desired, they can be or'd together as shown in the code sample at the end of this document. For instance, write permission does not imply read and append. File permission is defined in terms of canonical absolute path names; checks should always be made with canonical file path names.

The FileIOPermission describes protected operations on files and folders. A File class is implemented that provides secure access to files and folders. The security access check is done at the time the handle to the file is created. By doing the check at creation time, the cost of the security check is minimized. Opening a file happens once, while reading and writing happen multiple times. Once the file is opened no further checks are done. If the object is passed to an untrusted caller, then it can be misused. File permission access is only checked on the file open. This means for example that file handles should not be stored in global statics where code with less permission might get at them.

The parameter strings in the permission methods can be either case sensitive or case insensitive. This setting, which will be controlled by a flag in the admin tool, will allow for the model to extend to other platforms beyond Windows. The flag will set the case sensitivity for both the permissions and the file objects themselves so that their case sensitivity is consistent.

The flags specify the actions that can be performed on the file(s) or folder(s): Read, Write, and Append. In addition, the actions can be or'ed together to form more complicated access requests. See code sample at end of document for examples.

Folder access implies access to all files it contains and below it in sub-/subsub-/etc.-folders.


IsolatedStoragePermission

This permission controls what code is permitted to use IsolatedStorage and, if permitted, to what degree. The following restrictions on IsolatedStorage are supported.

  • Allowed usage (what identities can be used to scope data compartment)
  • Storage quota size
  • Maximum allowed expiration of data
  • Ability to retain data indefinitely

Please refer to the Isolated Storage specification for more detailed information about isolated storage.


ReflectionPermission

This permission is for the ability to read the name and type information of non-public members of types (but not actual access to contents of members). Also, ability to find subclasses and superclasses, what module and assembly the class is in. For more details, see ReflectionPermission API specification.

The ability to use Reflection.Emit is also controlled by this permission. Reflection.Emit has important security implications as it allows code to create dynamic assemblies. The ReflectionEmit permission should be granted with caution.


RegistryPermission

This permission defines what access is allowed for registry keys and values: reading, create, and writing. For each type of access (read, create, write/overwrite) a list of keys and values is specified designating that kind of access is permitted. Each type of access must be specified separately (for instance, write access does not imply read). Access for a key implies that access for all values underneath it and its sub-keys.


SecurityPermission

This permission defines a collection of simple permission flags used by the security system.

Execution: This permission must be granted in order for code to run – not granting this permission gives administrators a mechanism to prevent certain code from running at all.

Assertion: Ability to override the security check for any permission that has been granted.

UnmanagedCode: Ability to call unmanaged code, such as calling native code via P/Invoke or via COM interop.

SkipVerification: Allows code to run without passing verification. This permission should only be granted to code that is trustworthy as it circumvents the normal verification process that protects against attacks by breaking the rules of containment based on memory type safety. Use of this permission is justified when code is known to be safe on the basis of trust or other inspection, or known-verified code may skip repeated verification for performance reasons. Since without verification strict security enforcement cannot be ensured by the system, this permission should only sparingly be used when absolutely necessary. As this permission effects verification only, user code check for this permission is not a useful operation.

SerializationFormatter: Represents the right for code to be a serialization formatter—giving code the ability to get and set any non-transient data members of serializable types. This permission allows code to discover and alter the private state of an instance. Code with this permission can access all the non-transient fields of any instance of a serializable type it gets a hold of regardless of its accessibility.

ControlDomainPolicy: This permission is required for a domain host to be able to specify domain-specific policy. See Policy specification for application domain policy.

Operation | Description | Security implications | Remarks
AppDomain.SetAppDomainPolicy | Ability to set application domain policy for an existing domain | Affects security policy and therefore permission grants |

ControlEvidence: Allows domain hosts to provide evidence to code loaded in domains created by the host. Typically evidence provided by the host in this manner would be location of origin or digital signatures found on the assembly. If a host does not have this permission than any domain it creates will have the same evidence as the host by default.

Operation | Description | Security implications | Remarks
AppDomain.CreateDomain( … Evidence …) | Ability to set host evidence upon domain creation | Escalation of Permission (by setting evidence that would increase the permission set of the loaded code) | Applies only when Evidence is passed to method
AppDomain.Load( … Evidence …) | Ability to provide evidence upon loading an assembly into the domain | Escalation of Permission (by setting evidence that would increase the permission set of the loaded code) | Applies only when Evidence is passed to method

ControlPolicy: Allows code to access and modify policy, both machine and user policy configuration.

Operation | Description | Security implications | Remarks
SecurityManager.SavePolicy | Ability to change security policy | Changes to policy will affect the permissions granted to code |
SecurityManager.SecurityEnabled | Ability to turn off Runtime security enforcement | When security is turned off all code is, in effect, fully trusted |

ControlPrincipal: Allows code to replace the Principal object for a given thread. This function is important to applications using role-based security as the Principal object affects subsequent authorization actions based on identity/role information. For more information refer to the Role-based Security specification.

Operation | Description | Security implications | Remarks
AppDomain.SetPrincipalPolicy | Controls the type of Principal created | Allows code to change the behavior of role-based security |
Thread.CurrentPrincipal | Ability to replace the Principal object for the current thread | Spoofing of user identity (code could replace current role-based user info) |

ControlThread: Allows the following operations on threads that need protection against unrestricted use.

Operation | Description | Security implications | Remarks
AppDomain.GetThreads | Returns an array of the threads in the application domain | Information not needed by code that should not be able to manipulate threads. |
Thread.SetCurrentContext | Sets thread’s context | Allows context switching |
Thread.SetAppDomain | Sets thread’s domain | Allows app domain switching |
Thread.PushDomain | (similar to above) | (same as above) |
Thread.PopDomain | (similar to above) | (same as above) |
Thread.Suspend | Suspend execution | Interferes with execution |
Thread.Resume | Resume execution | (same as above) |
Thread.Interrupt | Interrupt a thread | Interferes with execution | Thread is able to protect self from
Thread.Sleep | Puts a thread to sleep | (same as above) | (same as above)
Thread.Stop | Stops a thread running | (same as above) | (same as above)
Thread.SetPriority | Set thread priority level | Take cycles away from others | Could allow up to a maximum
Thread.SetBackground | Set fore/background | Interferes with execution |


UIPermission

The UIPermission defines access to various aspects of the Windows user interface: use of display windows, access to events, and use of the clipboard.

The window use level controls access to the window graphic user interface programming model, specifically: use of top-level windows, drawing in windows on the screen, handling user input and other events within the windowing system.

Partially trusted code may only use a special kind of window known as a “safe window” that guarantees the window will be appropriately labeled so that the user is aware of the source of the content, in order to prevent malicious use of such windows as mock system dialogs that might attempt to fool the user into entering a system password or some other kind of Trojan Horse attack.

The window use level of permission is represented as levels, listed from least to most restrictive:

  1. AllWindows: all windows & all events may be used
  2. SafeTopLevelWindows: safe top-level window use only – details follow
  3. SafeSubwindows: Only allow safe subwindow use (for embedded components)
  4. NoWindows: No window use allowed at all

Window use level of “safe” top-level window (2 above) allows use (including destruction) of any “safe” top-level window created within the app domain, and implies subwindow use (3). The friendly name of the app domain is used to reliably identify the code using the window to prevent impersonation of system messages and login requests by partially trusted content. Application has access to all events in these “safe” windows with exception of events in the title bar itself: the system menu and close button events cannot be intercepted. The specifics of the “safe” restrictions are as follows:

  • Title bar with content identifying adornment required. The title bar will always have a close button; the minimize and maximize buttons are optional (on by default). The title bar’s icon and the first part of the title can be specified by the application. However, the title will always be appended with the following string: “ – (TBD) Application” which is always visible in the title bar, if necessary the application-specified title may be truncated to ensure this.
  • Maximum window size is limited to Windows maximized window dimensions. That is, the window cannot have a width greater than the current desktop width, and it cannot have a height greater than the current desktop height, minus the height of the taskbar if it is always on top and not auto-hidden.
  • The window cannot be programmatically positioned to have the title bar off-screen in the vertical.

Window level use 3 allows drawing and event access is limited to one or more clipping subwindows associated with the app domain. This permission level is useful for embedded applications such as a control on a web page. The host (IE in this case) associates subwindows it wants to allow code to use within an app domain; the host must have unrestricted window permission granted. In doing so the host is responsible for enforcing the “safe” restrictions on the unrestricted top-level window the subwindow is on. In effect this allows a trusted host to make a “safe” subwindow within an unrestricted top-level window such that it behaves as a “safe” window for purposes of security.

For sub-PC devices with small displays (too small to support a windowed interface) window restrictions apply in a different yet analogous way as follows.

  1. AllWindows: entire screen & all events may be used
  2. SafeTopLevelWindows: N/A
  3. SafeSubwindows: Only allow drawing to a prescribed portion of the screen (under host control)
  4. NoWindows: No drawing is allowed at all

The clipboard access permission has three states:

  1. AllClipboard: any clipboard access is allowed
  2. OwnClipboard: paste from same app domain only (pending implementation cost)
  3. NoClipboard: no clipboard access is allowed

The middle permission level restricts clipboard paste operations to only work on content copied from the same app domain; the other states are self-explanatory. When the clipboard data is written on a copy operation, security information uniquely identifying the app domain of origin is recorded and subsequently compared at paste time to control access. Even unrestricted clipboard permission code should attach app domain context information on copy (could suppress this, but no clear reason why). Interoperability with IE would be desirable, but if cost is high is not required.

Drag/drop is always an intentional user interface operation, so there is no attack accessing private information (as long as windows are properly marked per the UIPermission) – no permission restriction. However, the API should be sure to only expose private information on the actual drop itself and not merely by dragging content over a window area (which could result in unintentional leak of data). The window event message queue restrictions need to prevent programmable simulation of drag & drop.


Identity permissions

Identity permissions check code identity based on evidence about the application code and provide reliable information about the identity of managed code. Just as other permissions describe protected operations on resources that are used to make security decisions, identity permissions describe code identities upon which security decisions can be made. Supported forms of code identity are:

  • software publisher, based on evidence of Authenticode™ signature
  • strong name, based on cryptographically signed namespace
  • zone of origin of the code
  • web site of origin of the code
  • URL of origin of code (including protocol, e.g. http://site/app)

Identity permissions should only be used to infer additional trust when satisfied – it is not recommended to use an identity check as a basis to disallow an operation when the identity matches. Use of positive identity checks to reduce access to specific entities are easily defeated by hiding of evidence, which can be accomplished in a variety of ways. Examples include: checking for a certain software publisher (may hide identity by not signing), or a certain web site (may hide by use of another server address), and so forth.


Identity permissions in depth

Identity permissions are based on tests against granted identity permissions derived from evidence to determine the identity of caller or callers. Security policy makes decisions based on code evidence that controls what code runs and what permissions are to be granted. For the standard permissions, the grant determines the set of protected operations the code is allowed; identity permissions are granted to identify the code based on evidence. For example, if a given piece of code has evidence associated with it including a digital signature, a strong name, a zone and site of origin, then identity permissions corresponding to these will be granted by policy. Where policy uses code groups based on evidence to grant standard permissions, the identity permission grant is independent of any policy configuration and only a function of the evidence and identities it contains. (See policy specification for more detail on policy.)

It is worth noting the semantics of identity permissions as code access permissions. A stack walk check of an identity permission tests the identity of all callers and succeeds only if all callers have matching identity. Overrides work as with other code access permissions: asserting an identity will cause a subsequent check to succeed even if callers above do not have the necessary identity. For example, some component may require all its callers have a certain identity to use it, however, in some special case it may be desired to expose the functionality to any caller – to do this, a method with the necessary identity should assert its identity and then call the component (the assert allows the component check to succeed even though above the immediate caller is other code without the proper identity). Denying an identity effectively hides it so that subsequent checks will fail to match the identity (this allows code to hide its trusted identity and call code in an anonymous way).

The following sections describe the basic identity permissions: for complete details see class specifications.


ZoneIdentityPermission

This permission defines the zone from which code originates. Using this permission code can check that calling code is from a certain zone. Zones are defined and mapped from URL by IInternetSecurityManager and related APIs. Only exact zone matches are defined for the permission – a URL can only belong to exactly one zone. For an overview of zones and complete details, see URL Security Zones Overview.


StrongNameIdentityPermission

This permission defines the cryptographically strong name of the assembly. Identity consists of two parts: the strong name key, and the simple name part. The strong name key part is required and must match exactly to satisfy the permission. The identity permission is satisfied by an exact strong name key match plus simple name part match as described below.

The simple name part is specified as a string of the standard form runtime simple name format – for example, System.Security.Permissions – possibly with a terminal wildcard denoted by ‘*’ that matches additional subdivisions of the namespace. Some examples follow to define allowed forms more fully:

* | Wildcard matches all simple names (for strong name key)
Microsoft.Office.* | Wildcard matches names beginning Microsoft.Office
Microsoft.Office.Word | Specific simple name
Micro* | Invalid: wildcard must be a simple name component
*.Office | Invalid: wildcard must be last position only

Notes:

  • the form abc.def.* matches abc.def as well as abc.def.ghi, abc.def.ghi.jkl.xyz and so forth
  • the wildcard may only be used as the final simple name component as delineated by dots


PublisherIdentityPermission

An Authenticode™ signature on an assembly provides cryptographically strong proof of software publisher to grant this identity permission. The publisher identity is based on the public key as defined by an X509v3 certificate (i.e. not the name of the publisher embedded in the certificate) of the digital signature on an assembly. A single publisher identity applies to all code within the assembly. Other kinds of digital signatures as well as multiple signatures may be supported in future.


SiteIdentityPermission

This permission defines the web site from which code originates. Site identity is only defined for code from URL with protocol HTTP, HTTPS, FTP – the site is the string between the ‘//’ after the protocol and the following ‘/’ if present, for example “site.microsoft.com” in the URL “http://site.microsoft.com/x.htm”.

Sites may be matched exactly, or by wildcard ‘*’ prefix at the dot delimiter, as follows.

  • www.microsoft.com
  • *.microsoft.com
  • *.k12.wa.us
Note *.microsoft.com matches ‘microsoft.com’ (no prefix) as well as ‘www.microsoft.com’ etc. Without wildcard site name must be a precise match.

Use of HTTPS protocol provides authenticated web site name that results in a grant of authenticated site name, which is indicated by AuthenticatedSite Boolean flag. Matching an authenticated site permission requires the HTTPS (or in future other authenticated protocol); matching unauthenticated site (flag set off) will match any protocol (HTTP or HTTPS or etc.).


URLIdentityPermission

This permission defines the URL from which code originates. The complete URL is considered including the protocol (HTTP, HTTPS, FTP) and the file, for example “http://site.microsoft.com/x.htm”.

URLs may be matched exactly or by wildcard in the final position, for example “http://site.microsoft.com/app/*”.


Using permissions

The preceding sections describe what permissions are; this section talks about what permissions are used for.

Security policy decides on an appropriate set of permissions to grant any managed code that runs. Certain operations on protected resources – for example reading a file – require that the code by granted permission to read the specific file or else a security exception is raised. Thus, by properly configuring policy to assign permissions based on trust, security can be enforced to allow highly trusted code to do many things, and less trusted code is suitably restricted in what it can do.

Security permissions are enforced on managed code that is verifiably typesafe, which ensures that code can only access memory through well-defined interfaces and cannot use any objects outside of these bounds. The verification and execution within the EE combine to provide a security boundary as strong as ring-based security systems, without overhead of security boundary transitions. This allows different code (by assembly granularity) to be assigned potentially different permissions, all running together in the same app domain. Since the interfaces between classes of different assemblies are well-defined, code may call other code of different levels of trust in a controlled fashion.


Stack walk

Luring attacks represent an inherent security problem with partially trusted mobile code interacting with more and less trusted code within the system. The basic attack is for less trusted code to call more trusted code and get it to do something on its behalf since it does not have the permissions itself. The traditional solution to this threat is to make the more trusted code inaccessible, or to ensure that trusted code always checks its caller’s permissions or doesn’t expose dangerous functionality in the first place. System APIs that afford access to kernel services are an example of the former; COM “safe for scripting” object safety is an example of the latter approach.

The runtime provides protection against luring attack by demanding the permissions of all callers, rather than just the permission of the code attempting a protected operation. Since in a luring attack the less trusted code will always be somewhere in the causal sequence of calls, it will never be able to manipulate more trusted code to do things it does not have permissions for itself. Managed code type safety ensures that the chain of callers can always be reliably determined so attack code cannot hide its identity or raise its level of trust.

The runtime cost of demanding permissions of all callers is considerable, however without it all application code is exposed to potential luring attacks by less trusted code. Where necessary for performance code may be written to reduce the extent of stack walk permission demands, however, whenever this is done the programmer is taking responsibility for not exposing a security weakness.

Application code using the files via the base class libraries must have the corresponding file permission for the operation to be allowed. Implementations of the base class libraries for standard resources such as files or network perform permission demands for the corresponding permission. For example, before opening a file read-only the file class demands that all callers have read access for that file, and will only proceed with the operation if the demand succeeds.

Class libraries that expose resources via the underlying O/S API implementation such as file require high trust to be able to call unmanaged code. Again in the case of the file class, after demanding the permission of the callers the file class asserts its permission to call the native API (unmanaged code), and proceeds with the operation. The assertion of permission incurs an obligation on the class library code to only allow operations to be performed within the constraints of the permission demands it is responsible for making, and never more. Such a transition from managed to unmanaged code – which is only possible by highly trusted managed code – is analogous to the security checks that typically protect kernel services exposed to application APIs.

Stack walk overrides

A stack walk demand of permissions of all callers starts with the lower stack frames are works up to the top of the stack. The demand succeeds only if the code of all frames above the point of demand on the stack have the required permission. To allow certain trusted code to perform operations beyond the permissions of some of its callers, a set of overrides to the stack walk are provided. Overrides are defined for a certain permission or permissions and have the effect of stopping the stack walk in the case of a demand of those permissions.

Here is how the most commonly used override – assert – works. Suppose method A (top of the stack) calls B which calls C, and the code for these methods have different sets of permission granted. If C does a demand of some permission P1, then only if both B and A have P1 granted will the demand succeed. However, suppose B wants to “vouch for” A which does not have P1: in this case B may assert P1. Now when C does a demand the stack walk checks B for P1, succeeds, and then notes the assert of P1 which matches the demand, with the effect that the stack walk succeeds at this point without going further up.

Another override – deny – works similarly but instead fails the stack walk instead; deny is useful to allow trusted code to “decline” some of its permissions on a temporary basis. A third form – permit-only – is similar to deny in that it causes a premature stack walk failure; the difference is that with this override the permissions that should continue to succeed are named instead of those that should be failed.

This section defines the basic model for stack walk overrides; following sections describe coding situations where these overrides are useful. For complete details on how overrides work, see Permission object.


Requesting permissions

The most basic use of permissions is to simply request the permissions necessary to run the code. For a typical application this means requesting permissions for all protected resources or operations possible during use. Many applications will simply request the default permissions for the Internet or for the local intranet – and program within that model of security limitations – rather than even need to know about individual permissions.

Of course security policy will decide if the code actually gets the requested permissions granted or not. Applications request (a) the minimum set of permissions they need, (b) additional permissions to be optionally granted per policy, (c) permissions that never should be granted. If the code is allowed to run it will be guaranteed to have the minimum permissions, possibly some of the optional permissions, and never any of the refused permissions.


Resource protection: demand

The most common use of permissions is for protection of resources. Classes that expose the resource must demand the callers for the appropriate permission before doing the requested protected operation.

Library classes that expose basic resources such as files etc. must demand the permission of callers for the corresponding (e.g. FileIOPermission) permission. For typical application and component developers the permission demand is in effect an “automatic” function of the file classes; as long as the application code has the necessary permission granted no explicit security coding is required.

Similarly, to ensure security it is essential that all code that accesses a resource protected by a custom (application-defined) permission must implement demands for the permission wherever the resource is exposed. The security requirement is exactly the same as the library class case, however the resource is not a standard one but rather is custom.


Hidden use of permission: assert

Sometimes code uses a resource (that it of course has permissions for) in a method that does not actually expose the resource itself to callers, in which case it is often desirable to allow callers without the permission for the resource to still be able to call and have it work. In this case the code is in effect vouching for its callers and promising not to compromise overall security by exposing the resource to potential malicious callers that might use it to indirectly access it.

For example, calling unmanaged APIs in general is a protected operation that requires a high degree of trust. However, all APIs do not necessarily involve security problems: some specific ones may be completely harmless. For example, a system API that returns the current time of day may be deemed to not have dangerous security implications. To enable code to use this functionality without requiring a high degree of trust, the managed code that implements this may assert its high trust and call the API. Otherwise, only code permitted to call any system API would be able to use this safe and useful function.


Shifting permissions: demand & assert

A variation of hidden use involves changing from one permission to another. The standard File class requires the appropriate FileIOPermission before doing file operations, yet it uses system APIs to actually handle files. This is implemented by the library classes as a combination of the above two methods: first a demand of the resource permission ensures that callers have the needed permissions, then the trusted class code asserts its still higher permission to call system APIs to carry out the operation.

The effect of the demand and assert is to take the underlying high-permission operation (system API use) and constrain it to a specific case (file operations) where a demand for a lesser permission will suffice. Again, the highly trusted code making the “shift” between permissions is vouching for the safety of this security level transition. In return, partially trusted code can be enabled to use a rich subset of functionality without compromising security.


Limiting access by code identity

Some code may be secured by requiring that only code with a certain identity (per strong name, from a certain site, etc.) may call it. This approach effectively pushes the effort to secure access onto the specified caller, so is only as good as the precautions taken by the trusted caller. However, when implemented with declarative security the link-time demand (a simple demand with no run-time performance cost, see Link Demand), when sufficient, provides very good performance. Since link-time demand only looks at the immediate referencing code it is subject to a luring attack unless all possible code paths are suitably protected.


Restricting the permissions of code

Code with a certain level of trust may want to execute at times with a reduces set of effective permissions. For example a trusted application may have a “test mode” that can be selected for testing that ensures no live data is going to be modified accidentally. The security system can be used to ensure no file operations are possible by use of a deny of write permission for all files (subject to possible assert).

Similarly, to execute with a specific set of permissions and no more, use a permit-only override. Under the override, only the specified permissions will be allowed (again, subject to possible assert).


Combining repeated security demands for performance

As an optimization, a number of security demands may be combined into one by use of the following technique. If within a single method a number of operations are performed that all result in the same security demand by the underlying classes, by doing a demand and assert of the permission it is possible to reduce the overhead of the resulting security demands. Especially if the caller stack depth above the method is large this technique will result in significant performance gain.

Suppose a certain method is going to perform, say, 100 operations that require the X permission; of course to be allowed the code and all callers must have X permission. Without declarative security, each operation will require a stack walk of all callers to demand that X permission is granted, thus if the stack above X is n levels deep, 100n demands are required. The optimization is to first demand X that will do (depth n) stack walk to ensure that all callers indeed have permission X, and then assert permission X. Asserting X means that subsequent stack demands will stop at this method and succeed, reducing the number of demands by 99n.


Declarative security

Declarative security allows both programmers and administrators to define and set security settings of various kinds as attributes associated with code. Since declarative security is represented as attributes of the code rather than being embedded within the code itself, it offers the following advantages over imperative security (runtime security demands by code).

  • Declarative security may be browsed through metadata, makes security self-describing
  • Declarative security may be added after code is developed by a specialist, or a standard set of declarations defined by an expert can be easily attached to code by other developers
  • Declarative security may be modified without requiring code changes
  • Future optimizations are possible by analyzing (static) declarative security

Thus, declarative security is preferred when it can be used since – unlike imperative security embedded within code – it is explicitly defined at compile-time, and therefore subject to possible changes and optimizations not available to imperative security which in general being part of the program logic is not really subject to manipulation apart from the code itself.

Declarative security is subject to the following restrictions:

  • declarative security may only be applied at the beginning of a method
  • all parameters of the declaration must be fixed and known at compile-time
  • changing declarative security on assemblies breaks any digital signature (requires re-signing)


Declarative security model

Declarative security consists of declared actions on specified permissions attached to specific targets. The actions indicate if the declaration is a request for permissions, or a demand for permissions, or an assertion of permission, and so forth – a complete description of the security actions follows. The declared target specifies what the action applies to: Assembly, Class or Method.


Security targets

Security declarative targets may be on the Assembly, Class or Method. If the same action declared for both class and a particular method within, the more specific method declaration overrides the class declaration.

Assembly (mdAssembly): Security action applies to the entire assembly. This target is valid only for permission request actions.

Class (mdTypeDef): Security action applies to all methods and properties in the class. A class does not automatically inherit the security attributes of its parent class. Note: Security declarations may not be applied to an interface. Compilers should treat attributes placed on interfaces as an error and notify the user.

Method (mdMethodDef): Security action applies to the method.
A property may have (possibly different) security declarations on its get/set methods.
Also events support security actions, securing the method that corresponds to the event declaration.

There is no way to set declarative attributes on a field. Use accessor methods with to control field access.

If security declarations appearing at different levels (e.g. class and module) could apply to the same element then the most specific declaration alone applies. In other words, every method is contained in a class – where there is overlap in scope the more specific (i.e. method over class) is effective. Put another way, declarations apply to the element they are applied to or if none exist, any declaration from a superior containing element (if one exists) applies. An example should clarify:

Declaration Element Demand Precedence

   Class C1 – demand P2

C1

P2

C1

      Method X1 – demand P3

X1

P3

X1

      Method X2 (no declaration)

X2

P2

C1

   Class C2 (no declaration)

C2

(none)

      Method X3 – demand P4

X3

P4

X3

      Method X4 (no declaration)

X4

(none)

   Class C3 – demand P5

C3

P5

C3

      Method X5 – demand P6

X5

P6

X5

      Method X6 (no declaration)

X6

P5

C3

   Class C4 (no declaration)

C4

(none)

      Method X7 – demand P7

X7

P7

X7

      Method X8 (no declaration)

X8

(none)


Declarative Actions

There are three kinds of declarative security, depending on when it has effect, all of which take the form of an action together with permission(s) the action applies to.

  • Load-time, JIT-time: demand the granted permissions of code during loading or at JIT time
  • Run-time: emits code to demand permission of all callers on the stack at runtime
  • Grant-time: declarative requests for permissions are use at grant time by policy
    See policy specification for more details.

The action types are defined as follows:

Declaration of security action | Time of action | Targets supported | Description
public enum SecurityAction { |  |  |
LinkDemand, | JIT-time | Class, Method | Demand permission of a caller
InheritanceDemand, | Load-time | Class, Method | Demand permission of a subclass
Demand, | Run-time | Class, Method | Demand permissions of all callers
Assert, | Run-time | Class, Method | Assert permission so callers don’t need
Deny, | Run-time | Class, Method | Deny permission so checks will fail
PermitOnly, | Run-time | Class, Method | Reduce permissions so checks will fail
RequestMinimum, | Grant-time | Assembly, | Request minimum permissions to run
RequestOptional, | Grant-time | Assembly, | Request optional additional permissions
RequestRefuse, | Grant-time | Assembly, | Refuse to be granted these permissions
} |  |  |

Grant-time code requests are only valid for use in the assembly manifest, and are ignored elsewhere.

Link Demand

Demands at JIT-time that any code binding to the target being protected has the specified permission. The link-demand has a limitation in that it checks for the permission of the immediate caller only. It does not check all callers on the call chain. Specifically, linking means any bind to the type reference – this includes not just method calls but function pointer references as well. In a subclass, for methods inherited from the superclass the link-demand declaration, if any, of the superclass method applies; if the subclass overrides a method then the link-demand declaration, if any, of the subclass applies.

For example, suppose class A declares link-time demand of permission X. If another class B contains a call to a method of A then the runtime will demand that B has permission X granted before linking B to call A. If it does not have the requisite permission the link is not allowed. Code still loads but an attempt to bind to the method references will result in a security exception at runtime. Rationale for runtime exception is that only when the code actually tries the reference will the exception happen, plus if loaded the linking code itself may attempt to catch the exception and response.

Note that since link-time demands are limited to immediate caller, and not the full chain of callers, are subject to luring attacks. For instance class C that does not have permission X will be allowed to call B (assuming no additional declarative security in our example) so it is B’s responsibility to not weaken the security demand imposed by A by becoming a “gateway” for indirect calls to A, thwarting the intended security restriction. Only if B (and all other callers, and callers’ callers) also has the same declarative security or equivalent protection in place can these attacks be prevented.

Important: Developer documentation needs to be clear that this is not a stack walk and code must guard itself against luring attacks as described above. Use runtime demand when runtime security is needed.

Inheritance Demand

This security declaration has different meaning for classes (allows subclassing) than for methods (allows overriding). If both class and some methods of the class have inheritance demands, then the class declaration applies to the class and methods without declarations, and for methods with declarations the class declaration is ignored.

Classes: Demands at load-time that any code inheriting (subclassing) from the target class being protected has the specified permission. Note that this demand applies to all superclasses. For example, suppose class A declares inheritance demand of permission X. If class B subclasses A then the runtime will demand that B has permission X granted, and B will only load if it has the requisite permission. Then if class C that does not have permission X subclass B it will fail to load: which B has no declarative inheritance demand, A does and that is what fails.

Methods: For a given method with the declaration in a superclass, demands that any subclass overriding the method must have the specified permission. By using this declaration on specific methods it is possible to selectively control the ability of subclasses to override the methods.

Demand

Demands at run-time that all callers in the call chain have the specified permission. If any caller does not have the permission a security exception is thrown. The permissions of the code itself are not involved; rather the check begins from the immediate caller of the code and proceeds up the stack.

Implementation note: tail calls work differently because the stack walk starts one level higher than a call.

Assert

Asserts that it has the specified permission to access a resource even if its callers do not have the specified permission. Normally a permission demand involves walking the stack and checking each caller for the necessary permission. However, an assertion will stop the stack walk and succeed for any permission matching the permission asserted. The effect of an assertion is to force permission checks to succeed even if some callers above it on the stack do not have the requisite permission – the asserting code is taking responsibility at its interface not to compromise security by thus overriding the security system.

This is an extremely dangerous statement and must be used with care. The security model is built on the premise that all code in the call chain has the requisite permission to access a resource. Using an assert removes that requirement and, if improperly used, is a source of security holes.

Since assertion is potentially a source of security weakness, code must be granted the SecurityPermission in order to be able to assert. Further, only granted permissions may be asserted.

Declarative assertion of permission to call unmanaged code (SecurityPermission.UnmanagedCode, or a permission set containing it) is an important special case. Since a permission demand on UnmanagedCode is guaranteed to succeed within the method code, these normal permission demands for calls to unmanaged code will be omitted, providing a technique for code to avoid the performance cost of security checks when the code is “vouching” for security. Since only fully trusted code may so assert, integrity of security is ensured, however such code needs to be scrupulously written and checked against possible abuse by malicious callers. Even with imperative security overrides, no UnmanagedCode demand ever happens.

Deny

Denies the specified permission(s), effectively removing them from the granted permission set for the duration of the method. As a result, subsequent demands (until the method with the deny returns) of any denied permission will fail even if the code and its callers all have the permission granted. Like an assertion, denying permission will preemptively stop the stack walk, however deny forces the demand to fail.

Deny of any permission that is not actually granted has no effect, as stack walk would fail in any case.

Permit Only

Reduces the effective granted permissions for the duration of the method to be only the specified permission(s), effectively removing from the granted permission set any permissions not in the permit-only. This declaration is just another form of deny, with the set of remaining permissions specified, rather than the set by which granted permissions should be reduced. As a result, subsequent demands (until the method with the permit-only returns) of any granted permission not in the permit-only will fail even if the code and its callers all have the permission granted. Like deny, permit-only will preemptively stop the stack walk failing a demand.

Permit-only of any permissions not actually granted has no effect, as stack walk would fail in any case.

Request Minimum

Request a specified set of permissions as the minimum required for the code to be run. Grant-time declarative security may only be used on an assembly basis. Declarations of request on other security targets are not valid and are ignored. If omitted, code will be granted all permissions allowed by policy.

Applications are guaranteed to always have the requested permissions whenever the code runs, so it is not normally necessary to write security exception handling code so long as these permissions are all that are needed. However, it is possible for other code to reduce the effective permissions (such as by means of deny declaration) in which case security exceptions for lack of even these permissions may arise.

Request Optional

Request an additional set of permissions that should be granted if policy allows, but are not required for the code to run successfully. This allows code to request permissions beyond the minimum required, so ever attempted use of these permissions needs to be done with attendant exception handling to gracefully handle execution when permissions may not be granted. Grant-time declarative security may only be used on an assembly basis. Declarations of request on other security targets are not valid and are ignored. If omitted, the default meaning is whatever additional permissions policy chooses to grant will be given to the code.

Request Refuse

Request a specified set of permissions should never be granted. Grant-time declarative security may only be used on an assembly basis. Declarations of request on other security targets are not valid and are ignored. If omitted, any permission may be granted to the code by policy. This request allows a large set of permissions to be requested optionally, yet ensure that certain specific permissions are not in the grant.

Applications can use request refuse to ensure that they never get certain permission. For example, an application that browses data but never modifies it may refuse any file write permissions – doing so ensures that even in the event of a bug or malicious use the code will not be able to damage the data it operates on.

Another way to refuse permissions is to make the optional request set small or empty. However, in some cases unspecified permissions may be desired (requires request optional = “Everything”) yet some specific permissions are to be refused – this is the situation when this declaration becomes most useful.

Request refuse is only meaningful to reduce the optional permission set if it cannot be completely specified. The following are the common permission request situations that arise (last one is when refuse is needed):

  • code only needs a specific set of mandatory permissions the optional request should be empty (“Nothing”); there is no need to explicitly refuse permissions
  • code only needs a specific set of mandatory and optional permissions the requests should be for just those permissions only; there is no need to explicitly refuse permissions never requested
  • code wishes to get any unspecified permissions that policy allows, yet to never be granted certain specific permissions to be able to be granted unspecified permissions the optional request must be “Everything”, in which case request refuse is necessary to refuse specific permissions

Static suppression of security check

A special optimization is available to code that has permission to call unmanaged code (SecurityPermission.UnmanacedCode), allowing it to do so without the performance cost of asserting its permission before every transition to unmanaged code (either via P/Invoke or interoperability to COM).

Normally a call into unmanaged code triggers a demand of unmanaged code permission, which walks the stack checking that all callers have that permission. Applying the custom attribute SuppressUnmanagedCodeSecurityAttribute to the call into unmanaged code suppresses the demand, in essence creating a "open door" into unmanaged code. Only code that has the unmanaged code permission may use this attribute: it has no effect if used elsewhere.

WARNING: use this attribute with extreme care; incorrect use can create security weaknesses

Use of this attribute should never allow less trusted code (that does not have unmanaged code permission) to call into unmanaged code. This attribute is best applied only to privately declared entry points to unmanaged code so that code in other assemblies cannot access and take advantage of the security suppression. Typically the highly trusted managed code that uses it first demands some permission of callers before invoking the unmanaged code on the caller’s behalf.

In the rare case that the unmanaged code is completely safe for all possible circumstances, it can be exposed to other managed code directly (made public instead of private). Not only must the functionality of the unmanaged code be safe, but it must be impervious to attack by malicious callers: for example, the code must operate safely even if with unintended arguments fabricated to specifically cause the code to malfunction.

EXAMPLE (C#)

[SuppressUnmanagedCodeSecurityAttribute()]
[sysimport(dll = "some.dll")]
private static extern EntryPoint(args);


Syntax for declarative security

Declarative security syntax is based on attribute classes. All permission classes that are to be used declaratively will require definition of separate companion attribute classes. Work in progress with compilers on specific implementation issues: here’s roughly what we expect to end up with. (The limited expressive capability of attribute classes – must be representable as constructor arguments with simple types – makes declaration syntax awkward and requires some post-processing, as detailed below.)

using System.Security.Permissions;

// assert file read for c:\temp
[FileIOPermissionAttribute(SecurityAction.Assert, Read=”c:\temp”)]
// demand callers have HKCU registry write access
[RegistryPermissionAttribute (SecurityAction.Demand, Write=”HKCU”)]
// demand callers have named permission set “Everything”
[PermissionSetAttribute (SecurityAction.Demand, Name=“Everything”)]
// inherit only if subclass has permissions specified in file “custom.xml”
[PermissionSetAttribute (SecurityAction.InheritanceDemand, File=“custom.xml”)]

Declarations that are fully self-contained (first two examples) are fully representable with this approach, however references to permission set files (fourth example) need to be resolved to the developer machine at compile time. Specific problem is that the permissions in a file for a declaration are representable in attribute classes are just a reference – thus, if unresolved when the code is run on another machine the file may not be present. Thus, the use of attribute classes for declarative security will require a phase in metadata generation to actually get the file and embed those permissions in the declarative metadata so it will be represented without variation on all target machines. References to named permission sets (third example) do not require this extra step as they are limited to the named permission sets that are shipped as part of default policy and therefore present all machines where managed code may run (specifically, they are limited to the immutable permission sets the contents of which can not be altered—see the Code Access Security Administration specification for more detail).


Permission attribute classes for declarations

A permission object must have a companion attribute class in order for it to be usable declaratively. Attribute classes are defined for the standard and identity permissions. Each custom permission that is to be used declaratively should also have an attribute class defined for it. Attributes are coded as object constructors. The syntax of each permission attribute class constructors is dependent on the specifics of the permission itself. The attribute need only support constructors that can represent the useful declarative forms – for link-time, run-time, and grant-time – of the permission, which is not necessarily the complete set of states of the permission object.

Declarative security attribute classes derive from SecurityAttribute class, which is a subclass of the Attribute class for declarative security attributes. Attribute classes should be in the same namespace as the corresponding permission object. Name the attribute class SomethingPermissionAttribute for a permission named SomethingPermission (PermissionSetAttribute is an exception).

The CodeAccessSecurityAttribute base class provides common functionality for all security declaration attributes. A get method for the security action as defined above. The CreatePermission () method on an attribute class creates an instance of the corresponding permission object for the declaration.

public class CodeAccessSecurityAttribute: Attribute {
   public CodeAccessSecurityAttribute(SecurityAction action);
   public SecurityAction Action; {get;set;}
   public SecurityAction Unrestricted; {get;set;}
   public IPermission CreatePermission; // create permission obj.
}
// a few common attribute classes are shown here
[attribute(VOSElementType.All)]
public class FileIOPermissionAttribute : SecurityAttribute {
   public FileIOPermissionAttribute (SecurityAction action);
   public String Read; {get;set;}   // property forms
   public String Write; {get;set;}   // property forms
   public String Append; {get;set;}   // property forms
   public IPermission CreatePermissionInstance; // create permission obj.
}
[attribute(VOSElementType.All)]
public class RefelctionPermissionAttribute : SecurityAttribute {
   public RefelctionPermissionAttribute (SecurityAction action);
   public IPermission CreatePermissionInstance; // create permission obj.
}
[attribute(VOSElementType.All)]
public class SecurityPermissionAttribute : SecurityAttribute {
   public SecurityPermissionAttribute (SecurityAction action);
   public boolean Assertion; {get;set;}
   public boolean UnmanagedCode; {get;set;}
   public boolean Execution; {get;set;}
   public boolean SkipVerification; {get;set;}
   public boolean ThreadControl; {get;set;}
   public IPermission CreatePermissionInstance; // create permission obj.
}
[attribute(VOSElementType.All)]
public class PermissionSetAttribute : SecurityAttribute {
   public PermissionSetAttribute (SecurityAction action, string nameOrFile);
   public PermissionSet PermissionInstance; {get;}
}


Assembly manifest code request

Security policy mechanism grants permissions to managed code based on the evidence from the code, plus the request the code makes for permissions. The assembly is the basic packaging unit for application code; each assembly manifest contains a declarative security request for the permissions it needs to run. For more information on code requests and how the work with policy to product the grant, see Security Policy Specification.

LM serializes each of the provided permission sets (using ConvertPermissionSet exported from mscoree.dll) and writes the result as an uninterpreted blob into a custom value associated with the assembly metadata token. If a given (optional) permission set is not provided, the associated custom value is not created. The names of the custom values are as follows:

  • SecPermReq_Reqd
  • SecPermReq_Opt
  • SecPermReq_Deny

Macros for all of these are defined in cor.h (COR_PERM_REQUEST_REQD_CUSTOM_VALUE etc).

Adding code request to an assembly

Use the LM tool to set the code request on an assembly. Only one request (which may have three parts) exists for each assembly, so this operation overwrites any pre-existing code request. Tools that product assemblies may support specifying code requests directly without use of LM.

The XML format input file consists of up to three permission sets. At least one permission set must be present in the input file in order to be valid. This first permission set declares the permissions that are required in order for the program to run. The first permission set declaration can be optionally followed by one or two further permission set declarations. (There is no separating syntax, it's just a straight list of permission sets.) These represent optional permissions and permissions that should never be granted, respectively. If the user wishes to provide a refused set but not an optional set, they can place an empty declaration in the optional sets place (<PermissionSet></PermissionSet>). XML parsing is case sensitive.

Example:

   lm -n Sample -f sample.exe util.dll -s permissions.xml

Where permissions.xml is a file of the form:

<PermissionSet>
   … required permissions (see format below)
</PermissionSet>
<PermissionSet>
   … required permissions (see format below)
</PermissionSet>
<PermissionSet>
   … required permissions (see format below)
</PermissionSet>

Signing assemblies

Assemblies may be signed by software publishers using the standard AuthentiCode signing tools. For more info, see Windows Platform SDK – http://msdn.microsoft.com/library/sdkdoc/crypto/cryptotools_0bl1.htm

Since any change to an assembly breaks a digital signature, signing should be the last step in producing code. Specifically, code requests should be specified before signing.

A simple example of code signing follows.

makecert /sk XYZ /n "CN=XYZ Company" XYZ.cer

Generates a test X.509 certificate called XYZ.cer.

cert2spc XYZ.cer XYZ.spc

Generates a test Software Publisher Certificate (SPC) from the X.509 certificate called XYZ.cer.

signcode /spc XYZ.spc /k XYZ  XYZ.exe

Signs XYZ.exe with the XYZ.spc Software Publisher Certificate.


Viewing code requests

To view the code requests in an assembly, use the PermView command line tool.

PermView [/GUI] [/OUTPUT <filename>] <manifest file>

By default, any embedded permission requests will be dumped to the console. If /OUTPUT <filename> is given, output will be dumped to a file in a format compatible with the LM -s option. /GUI will display the permission requests in a graphical window as a tree view.


Representations

The basic representation of permissions is done using XML format. For the specific format of these representations refer to the Schema section of the Runtime Code Access Security Administration specification.


Encoding

Many managed objects used by security need to be represented in a persistent form encoded as a sequence of bytes. Roughly, objects with this requirement fall into two groups: permissions – such as for use in security declarations or code requests – and policy which is a superset of these including policy levels, code groups, and so forth.

There are two formats supported which satisfy differing requirements corresponding to various usages within the system:

  • Binary – an efficient and compact-sized representation of the objects
  • XML – a human-readable and –editable representation of the objects

Binary serialization of security objects uses the standard binary serialization facility. Refer to serialization specification for details of the format and how it is processed.

Security objects use a unique format for representation in XML that is different from the standard “XML serialization format” for objects. The primary reasons security is different derive from some unique aspects of the design of the code access security infrastructure. Primarily, security policy must typically be evaluated early on for every application that runs managed code: for this reason it is important that the security overhead be held to a minimum for optimum performance. Further, since policy uses managed code to run, it is important to minimize the amount of managed code involved in spinning up the security system – if security policy computations involve references to other assemblies that must be loaded to execute, then policy in turn needs to be evaluated in order for that code to run – it is easy to get into an avalanching situation handling policy for the very code that is part of the policy system.


Uses of security encodings

The runtime security infrastructure uses persistent representations of various objects to implement security, as detailed below.

  • Declarative security represents permission (and permission set) objects encoded in binary format
  • Code requests are represented as permission sets: tools use the XML format as input, and represent the request as permission sets in binary format
  • Policy and the code groups (etc.) they contain are represented as XML for configuration

To support all of these uses, the System.Security objects support encoding and decoding from representations as sequences of bytes – either binary or XML format. When application code extends these classes it is necessary to compatibly support these encodings. The following sections describe how the encoding works, and how it can be extended.

security objects

The following classes support encoding – these are referred to here as security objects since the process is identical for all types, only with the respective semantics of the corresponding classes:

  • permissions (classes implementing IPermission)
  • permission sets (PermissionSet)
  • policy levels (PolicyLevel) including contained code groups and so forth
  • subclasses of the above


Binary encoding of security objects

Binary encoding and decoding of Security objects – permissions, permission sets, policy levels and their code groups – uses standard binary serialization. Custom code need do nothing special to support binary serialization, which is handled via reflection as it is for all kinds of objects. Refer to System.Serialization classes for details.

The following pseudo-code shows how the encoding and decoding process works for security objects.

// Encode security object obj to Stream stream
   BinaryFormatter formatter = new BinaryFormatter();
   formatter.Serialize( stream, obj );

// Decode security object obj from Stream stream
   BinaryFormatter formatter = new BinaryFormatter();
   obj = formatter.Deserialize( stream );


ISecurityEncodable interface

Any security object that needs to be XML encoded by the security system must implement this interface. These System.Security classes implement this interface: permissions, permission sets, policy level, code groups, and membership conditions.

Application code developers need to implement this interface for the following kinds of objects that may extend the standard security system types:

  1. new permission types
  2. new policy types: new membership condition or code group types
interface ISecurityEncodable
{
        void ToXML (SecurityElement RootElement) ;
        void FromXML (SecurityElement RootElement) ;
}

The XML format for encoding consists of an outer element corresponding to a class instance – for example <Permission …> element for each permission, and so forth – that contains text and arbitrary elements representing the state of the object. Code in the corresponding class can only add sub-elements to the container element for the object, and only sub-elements are looked at later in reconstituting the object.

The following subsections describe what objects do to support the methods of this interface.

For examples, we will discuss the FooPermission that has three fields to represent its state. The fields and the XML template representation are shown here to illustrate:

Boolean flag; String str;

<Permission class=”FooPermission” assembly=”name”>

text contents of String str …

<flag/>

</Permission>

ToXML

ToXML(element) puts information in the element to create a representation of the state of the security object (this). For instance the FooPermission might do this:

SecurityElement ToXML ()
{
   SecurityElement e = new SecurityElement ("Permission");
   Type type = this.GetType();
   String assemblyName = type.Module.Assembly.FullName;
   assemblyName = assemblyName.Replace( '\"', '\'' );
   element.AddAttribute( "class", type.FullName + ", " + assemblyName );
   e.AddAttribute("Version","1");
   if(flag) e.AddChild(new SecurityElement(“flag”));   // add <flag/> iff true
   e.Text = str;   // text of the element
   return e;
}

FromXML

The encoded XML representation can subsequently be used to reconstruct an identical object. The FromXML() method handles the type-specific interpretation of the XML created by the corresponding ToXML(), and should result in an identical (for purposes of the particular security object semantics) valued object. For instance the FooPermission might do this:

void FromXML (SecurityElement e)
// pseudocode sans error checking
{
   flag = (NULL != SearchForChildByTag(“flag”));  // any <flag/>?
   str = e.Text;   // text of the element
}

How ToXML/FromXML are called

This section describes how the security system calls ToXML and FromXML. While custom code that extends security objects will need to implement the ToXML and FromXML methods to make the objects security-encodable, application code never actually needs to call these methods directly. Nonetheless, a description of how the ISecurityEncodable methods are called is useful in understanding how to implement them.

The following pseudo-code shows how the encoding and decoding process works for a permission. These are not public APIs but just examples of how ToXML and FromXML are used by the security system.

EncodeSecurityObject(Object obj, Stream stream)
// Example shows how security object obj is encoded in XML to stream.
// Actual implementation is different but equivalent calling ToXML().
{
   SecurityElement e = obj.ToXML();   // add object contents to the object model
   e.ToStream(stream); // writes the XML object model to the stream
}

Object DecodeSecurityObject(Stream stream)
// Example shows how security object obj is encoded in XML to stream.
// Actual implementation is different but equivalent calling FromXML().
   SecurityElement e = SecurityElement.FromStream (stream);   // Get XML
   if (e.Tag != “Permission”)
      raise SerializationException(“wrong kind of serialized data”);
      // error case of say policy level where perm set data expected
   String classname = e.Attribute(“class);
   if (classname==NULL)
      raise SerializationException(“require class attribute”);
      // error case of unexpected class
   obj = CreateInstance(GetType(classname));
   // construct instance from the object model
   obj.FromXML(e);
   return obj;
}


SecurityElement class

The SecurityElement class is defined to represent the XML object model for encoding security objects. The design is intended to be a lightweight implementation of a simple XML object model for use within the security system, and not intended for use as a general XML object model.

XML object model

The simple XML object model for an element consists of the following parts:

  • tag is the element name as shown below
  • text is a string contained between delimeters <tag> … </tag>
  • children are zero or more elements nested within <tag> … </tag>
  • attributes are zero or more name/value attribute pairs as shown below

The following XML forms summary the corresponding XML forms that are supported:

  • <tag/>
  • <tag> … </tag>
  • <tag name … />
  • <tag name … > … </tag>
  • <tag name=value … />
  • <tag name=value … > … </tag>

Elements with a NULL text string are represented in the <tag/> form, otherwise text is delimited by <tag> … </tag> tokens. Both forms may or may not be combined with attributes, which are shown if present or omitted otherwise.

Attributes that appear as <tag name> instead of <tag name=value> are designated by the distinguished object SecurityElement.NoValue. The attribute name must be non-NULL and of one character or longer.

The tags, attributes, and text of elements are always case-sensitive.

The XML form will contain quotations and escapes where necessary. String values that include characters that are invalid for use in tag/name/value/text will result in ArgumentException being raised. The above rules apply to all properties and methods.

NOTE: for performance reasons character validity is only checked when the element is encoded into XML text form, and not on every possible set of a property or method. Static methods allow explicit checking where needed.

Constructors

Constructor Void SecurityElement (String tag)

Creates an element corresponding to:

<tag/>
Constructor Void SecurityElement (String tag, String text)

Creates an element corresponding to (or text==NULL is equivalent to above):

<tag>text</tag>

Properties

Properties of the SecurityElement class are described in terms of a general XML element.

<tag name=value …>
   text…
   <child … />
   …
</tag>
Property String Tag; {get;set;}

Get/set the string tag of the element. A tag is required to be a valid element.

Property Int Attributes; {get;}

Get the number of attributes on the element.

Property Int Children; {get;}

Get the number of child elements of the element.

Property String Text; {get;set;}

Get/set the string text of the element. A NULL value specifies <tag … /> form, where “” string value is <tag …></tag> form, which are considered distinct.

Basic Methods

Void AddAttribute (String name, String value)

Adds attribute of the form “name=value” to the element. If an attribute of the same name (case sensitive) exists it is changed to the new value. You cannot have more than one attribute of a given name.

Void AddChild (SecurityElement)

Adds the element as the last child of the element. Duplicate child elements are allowed. Use the Children property for more complex manipulations.

SecurityElement GetChild (Int index)

Returns the index-th child of the element. Valid for values 0 = index < Children.

Boolean Equal (SecurityElement other)

Compares the element to the other and returns true if the tag, attributes, text, and all child elements are equal. Child elements must be equal in number and are compared one to one in order they exist. Attributes must be in the same order to compare as equal: that is, the following will not be considered equal.

<tag attr1=x attr2=y>

<tag attr2=y attr1=x>

static Boolean IsValidTag (String)
static Boolean IsValidName (String)
static Boolean IsValidValue (String)
static Boolean IsValidText (String)

These methods test if the string is valid for use as tag/name/value/text in an element.

Text encoding Methods

void ToStream (Stream stream)

Writes a textual representation of the SecurityElement to the stream, including its attributes, text, and children. Indentation and line breaks are unspecified.

void ToStream (Stream stream, Int indent, Int LineLength)

Writes a “pretty” textual representation of the SecurityElement to the stream, including its attributes, text, and children. The top element is indented by the specified amount of whitespace.

The LineLength parameter gives a desired maximum line length. The formatted output will attempt to keep line lengths shorter than this many characters, however if indenting or long tag/attribute/text words may cause this limit to be exceeded where necessary to produce a true representation of the element.

static SecurityElement FromStream (Stream stream)

Creates an equivalent SecurityElement to the one that wrote its textual representation with ToStream method.

String ToString ()

Return string representation of the SecurityElement and its attributes, text, and children.

static SecurityElement FromString (String s)

Creates an equivalent SecurityElement to the one that made the string representation with ToString method.

Searching Methods

String Attribute (String name)

Returns the string value of attribute by name, or NULL if none.

SecurityElement SearchForChildByTag (String name)

Returns element of any immediate child element that has a matching tag. This is useful for representations of the form <tag><child1/><child2/>…</tag>.

String SearchForTextOfTag (String tag)

Searches for the text of an element with the given tag, beginning with the element (this) itself, then SearchForTextOfTag() on each child in order, which searches the child’s child and so forth, depth-first.

Other details

Errors parsing XML cause XMLSyntaxException to be raised.

XML is case sensitive.


Versioning Persistent Representations of Objects

This section describes how persistent representations of objects are versioned: specifically, how we handle the case of version N+1 code decoding a representation created by encoding with version N of the same object class. That is, we support backward compatibility but not forward compatibility.

Binary formats support versioning through the binary serialization formatters. Version numbers correspond to versions of the runtime.

Security objects need to provide backward compatible conversion paths, and mark representations with necessary version information to satisfy the requirement stated at the top of this section. The standard way to associate version with the XML representation is with a version tag in the element with the class attribute. The class that implements ISecurityEncodable determines the form of the version attribute.

<Permission class=”Foo” version rev=”1.0.0.0”>
   … content is class-dependent …
</Permission>

Versioning will detect if older code is ever asked to decode data from a newer version of the code that it cannot interpret. In this case, raise SerializationException. Note that some security objects may implement more sophisticated versioning as an extension on top of the basic versioning described here.


Permission object

This section describes the behavior of permission objects: each type of permission has its own particular semantics for defining and interpreting the permission, however all permissions support a common set of operations described here. Specific permission details are documented in their respective API documents.

Only developers of libraries and classes that define or expose protected resources need to use permission objects. Permission objects should have class names of the form, SomethingPermission.

In working with permission objects, an important special case to handle properly is the no-permission case. In a permission grant, the absence of the permission object in the grant means that permission was not granted. Also, the result of set operations on permissions may result in NULL to represent the no-permission case. Code that may generate such a NULL permission must check for it explicitly to avoid a possible fault attempting to call a method on a null object. However, all permissions must support the no-permission state in order to implement the standard constructor as defined in the following section.


Classes and interfaces

The basic functionality of any permission object is defined in the IPermission interface. All methods below are defined on this interface except as explicitly noted.

A code access permission is a kind of permission that may be tested on all callers via the stack walk mechanism. All standard permissions available through default policy are code access permissions, however other kinds of permissions may be defined (an example of a non-code access permission is PrincipalPermission). The CodeAccessPermission class adds stack walk functionality to IPermission.


Creating a permission

Permissions are created in either a totally restrictive (no permission) or totally unrestrictive (all permissions corresponding to the particular resource) state. For example, the file permission constructor would create an object representing either no access to any files, or all access to all files. Each type of permission clearly defines to extreme states representing either all or none of the permissions expressable within the type. Thus, it is possible to create a generic permission in a completely restricted or unrestricted state without knowledge of the particular permission, however intermediate states can only be set according to the specific permission semantics. These states can are expressed using the PermissionState object as in the examples below.

  • SamplePermission(PermissionState.Unrestricted) creates the permission in the unrestricted state – passes all demands
  • SamplePermission(PermissionState.None) creates the empty permission – fails all demands

Other constructor forms may be defined according to the particular permission to specify other states of the permission as well. Null constructors (of form: SomePermission() ) are equivalent to using the PermissionState.None constructor.


Demanding a permission

A permission check ensures that all callers have been granted necessary permission. A demand requires that caller(s) have the permission, or if not a security exception is raised. Typically a demand of a permission is made in response to a request to perform a protected operation, checking either granted permission or identity.

The most common form of demand applies to all callers: a stack walk checks the permission working up the stack frame by frame, starting with the immediate caller. A variant kind of demand walks the stack to the first caller above ourside the code’s own assembly and app domain – that is, the first level that could possibly have different grant or identity. Note: load-time declarative security demands apply to the immediate caller only – that is, the first stack frame above the code with the declaration; there is no imperative API exposing this, however it is mentioned to contrast the semantics.

Demand method

To perform a permission demand granted permissions of all callers, use the Demand().

   void CodeAccessPermission.Demand ();

Note that checking begins with the immediate caller of the method that calls Demand() – it does not check against the granted permissions of the method itself. (Example: Main calls X which calls Y; Y does a Demand(). The first permission check is against the granted permissions of X, then of Main, succeeding if both pass; the check can pass even if Y is not granted the permission being checked for.)

CheckDemand method

This method must be implemented by each permission class as the fundamental underlying test of a code access permission against a given demand. This method is called by Demand(),and DemandImmediate(), to perform the basic permission check at each level of a stack walk, however it is never called directly by application code to perform a permission check.

When a demand happens, the CheckDemand() method is called with the grant permission set of each caller on the stack (according to the particular type of demand) as argument: if the grant satisfies the demand then the method succeeds and returns, otherwise the check fails and a SecurityException is raised.

void CheckDemand (CodeAccessPermission demand);

For many permissions CheckDemand() will succeed if and only if IsSubsetOf() is true, however there may be an important difference. CheckDemand() actually is the method that decides if a check will succeed, which in turn will typically mean a protected operation will be allowed -- IsSubsetOf() simply compares two permissions to test subset relationship. If the decision to allow the operation is only a function of the grant, then the two are essentially the same. However, if the decision depends on additional conditions other than the grant then the result of CheckDemand() may sometimes differ from IsSubsetOf().

For example, a hypothetical code access permission might give access to certain data but only during certain hours. In which case the subset relationship would be time-invariant yet the result of a permission check would be different depending on the current time.

It is noteworthy that identity permissions often check the subset relationship opposite from the way other permissions do. For example, a demand of site “*.Microsoft.com” is satisfied by a granted identity of “www.microsoft.com” because the latter is a subset of the former identity set, however a file demand on C:\TEMP\FILE is satisfied by a grant of C:\TEMP because the former is a subset of the latter.

IsGranted method

The IsGranted() method returns Boolean true if the calling method itself has been granted the permission. Code that requests optional permissions may use this method to test what permissions it was granted by policy so as to adjust its functionality appropriately. Granted permissions may only be tested against specific permissions and cannot be inspected directly. Inspection of grant set may reveal private information or knowledge of details of its grant set could be used to may exploit easier by malicious code. For identity permissions, this method allows code to test its own identity.

   bool IsGranted ();


Overriding granted permission

Assert, deny, and permit-only (collectively overrides) may be used to alter the effective granted permissions of code at the current stack frame. Assert a permission to stop stack walk check and succeed if the demand reaches the frame. Deny a permission to stop stack walk and fail. Permit-only is a form of deny where the set of permissions that should only succeed is specified – it is equivalent to deny of all permissions not specified.

   void CodeAccessPermission.Assert ();
   void CodeAccessPermission.Deny ();
   void CodeAccessPermission.PermitOnly ();

Overrides are effective until the invoking method returns and the stack frame is freed. Only one assert, one deny, one permit-only can be effective at a time for a given stack frame, however an override with a permission set may assert (deny, etc.) many permission types at the same time. Subsequent assert, deny, permit-only will replace any previous. To remove the effect or an override, do an override on an empty permission set (or in case of permit-only, a permission set of all permissions granted the code).

While rare in practice, up to one assert, one deny, one permit-only may co-exist on a stack frame. The following rules describe the aggregate effect. Consider the case of a demand for permission P on a stack frame with assert A, deny D, and permit-only O: here is how the permission demand happens at each frame.

  1. If P intersects D – check permission is in the deny set – fail the demand.
  2. If O is set, then if P is not a safe subset of O – demand permission is not allowed by permit-only – fail the demand.
  3. If P is a safe subset of A – demand permission is allowed by the assert – the demand succeeds.
  4. If P is not a safe subset of the grant set – demand permission is not allowed by the grant – fail the demand.
  5. If at top stack frame then succeed, else continue to the next caller frame up the stack from step 1.


Set operations

Each permission describes a set of protected operations; to implement policy and enforcement the set operations subset, intersection, and union are defined on each type of permission.

  • subset: true if and only if all operations in one permission object are included in the other
    bool IsSubsetOf (CodeAccessPermission permission);
     // name change from IsSafeSubsetOf() – “safe” confusing
  • intersection: the permission to do exactly those operations included in both permissions
    CodeAccessPermission Intersection (CodeAccessPermission permission);
  • union: the permission to do exactly those operations included in either of the permissions
    CodeAccessPermission Union (CodeAccessPermission permission);

Since the semantics of each type of permission differ, each permission class must implement the appropriate set operations accordingly.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Other operations

The Copy() method creates a copy of the permission object.

   Copy ();


Permission set

All of the operations defined for permission objects are useful for working with sets of different types of permissions. Permission objects refer to a generic object of a specific type – file, registry, etc.: all these methods on permission objects can equally be applied to a set of permissions of various types. The PermissionSet class object represents such sets of permission objects, and supports all features of the base permission classes (implements IPermission) so it can be used as if a single permission.

Permission sets are explicitly constructed from zero or more permission objects.

   PermissionSet ();   // empty permission set
   PermissionSet (IPermission permission);   // set of one perm.
   PermissionSet (IPermission p1, IPermission p2);   // set of two
   PermissionSet (IPermission [] permission);   // set of many

Permission sets may be created from named permission sets as defined for policy (see policy spec).

   PermissionSet (String name);

The following pseudo-code demonstrates how common operations are defined for permission sets. Given two permission sets containing two permission types A and B, for P1={A1,B1} and P2={A2,B2}:

  • P1.Demand(): succeeds if both A1.Demand() and B1.Demand() do
  • P1.Assert(): asserts both A1 and B1 (note that permission set is only way to do both at once)
  • P1.IsSubsetOf(P2): A1.IsSubsetOf(A2) && B1.IsSubsetOf(B2)
  • P1.Union(P2): permission set consisting of A1.Union(A2) and B1.Union(B2)


Custom permissions

Permissions are the core upon which the runtime security infrastructure is built, in that they define what is protectable and with what granularity. And since the permissions that ship with the runtime only protect those resources that are exposed by the standard class libraries, extensibility is a key feature to enabling the system to apply to a broad range of customer scenarios.

This document describes what custom permissions are and when to use them, how to define them, and a set of programming details that must be considered in the implementation.


When to use a custom permission

Custom permissions – like any permission – fundamentally define protection for a resource. Once defined, a custom permission can be used in exactly the same manner standard permissions are used.

When writing code that exposes a completely new kind of resource, there being no standard permission that covers this resource, a custom permission needs to be defined to allow access rights to be administered.

Custom permissions should not be used in an attempt to redefine an existing permission – wherever possible permissions should be non-overlapping. Having multiple permissions for the same resource presents an extremely confusing problem for administrators – to protect a single resource they need to deal with both permissions at the same time. (Otherwise, if they “close the door” with one permission, then code using the other way to access the resource that is protected by the other permission will still be able to access it.)

For example, managed code that exposed a new input device would need a new permission to define access to those devices.

Another example might be application-defined resources such as personnel records may best protect the resource with a specifically defined permission when the access modes and granularity do not correspond to the permission for the underlying system resource. Suppose in this example that one employee record is stored in a file, however read and write access needs to be controlled independently on different pieces of the data: since the file permission only specifies access on the entire file as a whole, a custom permission is needed to specify more fine-grained access. Alternatively, a custom permission that more directly corresponds to the programming model for a resource can make the code more understandable in addition to issues of the expressability of the permission.

To implement a custom permission, the developer must:

  • define the new permission object
  • declare appropriate constructors and methods to define the permission semantics
  • if code access permission, CheckDemand()
  • implement IPermission: copy, set operations, XML en/decoding
  • implement ISerializable (only if necessary for performance or to support special data types) – see Serialization specification for details
  • add support for declarative security
  • enforce the permission wherever the associated resource is exposed

The following subsections detail these steps.


Define the new permission

Permission objects define access to protected resources and what operations are allowed. Refer to the standard permissions (FileIOPermission, SecurityPermission, and so forth) for examples of permission objects.

Granularity of permissions

After defining exactly what resources a permission object is designed to protect, the key design decision is to what degree of granularity the resource needs protection. Taking the FileIOPermission as an example, let’s consider several alternate designs:

  • a permission with a single bit that meant “use all files” or “use no files”
  • a permission with two bits meaning “read all files” and “write all files”, or not, respectively
  • a permission with 26 bits meaning “use all files on drive …” (A, B, C, …)
  • a permission with an array of strings listing all files permission to use is given

and so forth. Clearly there are various tradeoffs among the various choices available: the single bit permission is very simple and fast and easy to understand, however it presents a rather stark all-or-nothing choice for administrators. Deciding on granularity is ultimately a subjective decision and a difficult one, but very important in that it determines the degree of control administrators will have in allowing access to the protected resource.

Additionally, overlapping permissions are to be avoided, meaning that once a permission object is defined it should be used thereafter as the exclusive way to protect the resource. Therefore, if the performance cost is not great, some additional granularity and expressability is often a good choice in the design of new permissions.

Avoid overlapping permissions

When the protected resource already has a permission object associated with it use that, don’t invent a new permission. As described above this will simplify the security model that administrators work with.

The careful reader will note that the example above a new permission to protect personnel records in a file actually violates the rule of avoiding overlapping permissions: the personnel record custom permission and the FileIOPermission for the file containing them both potentially may access the data. Other examples exist within the standard permissions: UnmanagedCode permission allows file and just about every other kind of resource access.

While this kind of overlap there are cases where it is workable. In the personnel record permission case, it is clearly infeasible to apply per-record granularity to the entire file system. Furthermore, it is easy to imagine other differently structured files that would call for a completely different kind of granularity of access protection, making an increasingly complex model applied to all files. Alternatively, in the case of UnmanagedCode permission, strict non-overlapping would imply that this is just about the only permission – administrators would be back to making binary “good/bad” decisions about code. Clearly the finer levels of gradation of trust that code access security enables are worth the overlap and the issues it causes.

Given overlap situations, how can this be made a workable situation – how do we save the administrator from contradictions resulting from competing permissions on the same resource? The key to managing this kind of overlap is when one of the permissions is more fine-grained and typically can be granted with lower trust.

For example, in the personnel record access case, there will be many situations where personnel record access permission is granted but no file access is. That is, where limited personnel record access is granted but no file access per se is – that is, code may go through the personnel system to access certain records, but it may not directly use files. Also note that the opposite makes little sense (no personnel record access, but free use of the file that holds the records), and is hence not an important case since no administrator would likely want to do such a thing. Put another way, in high trust situations where direct file access is given out, in all likelihood the personnel record access will naturally be given as well.


Define the permission object semantics

Once the resource protection semantics and the appropriate granularity of permission control are determined, the next step for the developer is to define the permission object itself.

There are a few typical kinds of permissions that are used depending on the resource being protected and granularity of protection. Most permissions will be one or these standard design patterns or a combination thereof, although other designs are possible when circumstances so indicate.

  • Boolean permissions – this simplest kind of permission object holds one or more bits that correspond to “permission to do X”. This approach provides no degrees of permission between can and cannot. For example, the SecurityPermission contains several Boolean permissions such as UnmanagedCode (permission to call it).
  • Levels of permissions – a slightly granular form beyond Boolean permissions, typically this represents the permission as numbers from zero (no permission at all for this) to some higher number meaning unrestricted permission, with a few levels in-between. For example, the UIPermission has a few levels of permission to use windows with a few safer gradations between none and full unrestricted permission. (Note that Boolean is just a 0/1 level permission.)
  • Object list permissions – provides a very granular permission that specifies what is or is not allowed. The FilePermission is a classic example where it uses lists of files that certain kinds of access are allowed on. Permissions with lists are most useful for protecting resources that contain a large number of named objects.


Implement the permission

All permission objects should implement IPermission, and most will implement IUnrestrictedPermission if they define an unrestricted state (permission allowing any operation on the protected resource). Nearly all permissions that protect resources are code access permission objects (see Types of Permissions for more about code access security and other kinds of permissions) and derive from CodeAccessPermission.

CheckDemand – Enforcement of the permission happens with this method (although almost any bug in the permission can compromise security with respect to the resource it protects). CheckDemand method is called on the granted permission for some code with the same-type permission object that is being demanded as an argument. If the granted permission does not satisfy the demand a security exception must be raised, otherwise the method returns indicating success. For a code access permission, CheckDemand is repetitively called during a stack walk.

For a complete description of CheckDemand, see the section on the CheckDemand method.

Set operations – permissions must implement Intersect and Union methods that are required by the IPermission interface. These set operations must be defined to operate consistently with CheckDemand, specifically:

  • IsSubsetOf is true if the other permission includes everything allowed by the permission. That is, X.IsSubsetOf(Y) if for all D satisfying X.CheckDemand(D) Y.CheckDemand(D) also succeeds.
  • Intersect results in a permission that allows all operations allowed by the two input permissions. That is, if A=B.Intersect(C) then for any demand D:
    A.CheckDemand(D) if any only if both B.CheckDemand(D) and C.CheckDemand(D)
  • Union results in a permission that allows all operations allowed by either of the two input permissions. That is, if A=B.Union(C) then for any demand D:
    A. CheckDemand (D) if any only if either B. CheckDemand (D) or C. CheckDemand (D)

For a complete description of IPermission and its methods, see Permission object section.


Handle XML encoding & decoding

Permissions must support XML encoding with the ISecurityEncodable interface, allowing the permission to be represented as XML and the permission object state restored from such an XML representation. For a complete description of encoding permissions, see the section on Encoding.

The content of the XML element that represents the object state is determined by the object itself – the FromXML method can use any representation so long as the ToXML can interpret it and restore the same state. However, the containing “Permission” element must be of standard form, e.g. for XyzPermission:

<Permission class=”SamplePermission, Xyz.dll” version=”1”>

The Permission element contains two attributes, “class” containing type name disambiguated by the name of the assembly that contains it, and the “version” attribute specifying the version. The following code fragment creates such a permission object XML element.

SecurityElement element = new SecurityElement( "Permission" );
Type type = this.GetType();
String assemblyName = type.Module.Assembly.FullName;
assemblyName = assemblyName.Replace( '\"', '\'' );
element.AddAttribute( "class", type.FullName + ", " + assemblyName );
element.AddAttribute( "version", "1" );

Note the Replace method applied to the assembly name. Attributes in SecurityElement can't handle having quotes in them, but some assembly name information is quoted. To handle this situation, double quotes (") in the assembly name are converted to single quote (').


Declarative security support

While not strictly required, a new permission should support declarative security in order to enable developers to use the permission as a declaration. The following code implements the attribute for a Boolean permission named SamplePermission.

The AttributeUsage defines how the attribute may be applied. Security attributes for declarations are special in that they must derive from the SecurityAttribute class – in this case (being a code access permission) the attribute class derives from CodeAccessSecurityAttribute, which derives from SecurityAttribute.

The attribute class consists of constructors and properties that allow specification of an instance of the corresponding security permission object. In this instance the object has a single Boolean flag as state: the attribute would be used as illustrated by the following demand declaration.

[SamplePermissionAttribute(SecurityAction.Demand, Flag=true)]

Security attribute classes must implement the CreatePermission method which servers to create an instance of the permission object from the associated attribute class. In this case CreatePermission checks the internal m_flag variable which hold the value set by Flag=true in the example above, and creates a corresponding instance of the SamplePermission object accordingly. (While only the flag=true case is useful, the attribute class should support all possible states of the permission object.)

[AttributeUsage(AttributeTargets.Method | AttributeTargets.Constructor |
              AttributeTargets.Class | AttributeTargets.Struct |
              AttributeTargets.Assembly, AllowMultiple = true,
              Inherited = false )]
class SamplePermissionAttribute : CodeAccessSecurityAttribute
{
bool m_flag = false;

public SamplePermissionAttribute ( SecurityAction action )
            : base( action )
{}

public SecurityPermissionFlag Flag {
get { return m_flag; }
set { m_flag = value; }
}

public override IPermission CreatePermission()
{
         if (m_flag)
return new SamplePermission( PermissionState.Unrestricted );
         else
            return new SamplePermission (PermissionState.None);
}
}


Enforcing the permission

In order to actually enforce the new permission checks must be placed so as to protect access to the associated resource by code without the permission.

For example, the FileIOPermission protects access to files and directories, therefore all managed code that exposes these resources is responsible for demanding the corresponding FileIOPermission of its callers.

Note that protection is only necessary when the resource access is made based on that code’s permissions. For example, managed code system libraries that are highly trusted and access file resources via native code interfaces must strictly adhere to this rule. On the other hand, code that simply calls those same system libraries need not: the library will demand that the intermediary code and all its callers above that have the necessary permission. However, if the managed code calling the system library code asserts its file permissions then it needs to either demand appropriately of its callers or otherwise ensure that security is not compromised.


Updating policy

Finally, to grant code the new permission, policy must be updated. First create a named permission set for the permission you want to grant. To do this write the XML representation of the permission, and give it a name.

caspol –addpset  file.xml  name

Next, change or add a code group consisting of the set of code you want to grant the permission to.

One restriction applies to the code assembly of the new permission object: you cannot grant the permission to the code that implements it. For example, the code assembly that implements SamplePermission may only belong to code groups in security policy that assign permission sets that do not include the SamplePermission permission (in any form). This restriction results from implementation issues of the runtime security system, however there is no practical value since a permission can’t really “enforce security” on itself – if the developer of the code is malicious it is certainly a simple matter to bypass any kind of security in code they already control.

For details, refer to the Security Administration documentation.


Platform Invocation Services User Guide

This document is aimed at developers of components and applications, writing NGWS runtime managed code, which also needs to run unmanaged code in existing DLLs. These DLLs can have been generated by Microsoft (for example, those which implement the Win32 API), or by ISVs or customers. This document includes sample code, written in VC7


Overview of Platform Invocation Services

Platform Invocation Services, abbreviated throughout this spec to “PInvoke”, allows managed code to call unmanaged functions that are implemented in a DLL. PInvoke takes care of finding and invoking the correct function, as well as marshaling its managed arguments to and from their unmanaged counterparts (integers, strings, arrays, structures, etc).

PInvoke was intended primarily to allow managed code to call existing, unmanaged code, typically written in C. A good example is the several thousand functions that comprise the Win32 API. [Whilst this was its original intent, PInvoke can be applied to other scenarios]

When PInvoke calls an unmanaged function, it locates the DLL where that function lives, loads that DLL into process memory, finds the function address in memory, pushes its arguments onto the stack (marshaling if required), enables pre-emptive garbage collection, and transfers control to the address for the unmanaged code.

Note: although it is possible to export methods of a class from a DLL, PInvoke does not support calling such methods. PInvoke only supports calling global functions exported from the DLL.PInvoke knows what to do from metadata, stored into the image file. This metadata, in turn, was generated by the compiler in response to extra information that the programmer writes into his source code, that describes the function and its arguments.

A similar process is used when NGWS runtime managed code inter-operates with existing COM code. Many of the steps are similar – locate the target methods, push arguments, marshaling if required, and transferring control. In fact, PInvoke and COM Interop share much of that marshaling code inside the NGWS runtime, so understanding one helps understand the other.

In the current release of the NGWS runtime, PInvoke is limited as follows: you can start executing managed code, and call unmanaged code. And from that unmanaged code, you can even return to managed code via a callback (see later). However, for the moment, you cannot start in unmanaged code and call managed code.

By a callback, we mean a pointer to managed code, that is passed as an argument to the unmanaged function. An example would be calling an unmanaged QuickSort function; one of its arguments is a pointer to a comparison function; if that comparison function is implemented as managed code, we call it a callback, for obvious reasons.

[There is one particular scenario, called “It Just Works” where you can start in unmanaged code, and call managed code. However, this is a special case that does not involve ‘classic’ PInvoke data marshaling. Such IJW scenarios are not discussed further in this document]


Authoring Styles for PInvoke

There are two general approaches you can adopt when declaring methods that match unmanaged functions:

  1. The first approach is used when the developer wants to call a method in an unmanaged DLL from his code. So, the developer declares that method (but doesn't supply an implementation for it) in such a way that the compiler will be able to put the information into metadata that the runtime needs to locate and correctly marshal the call to the DLL. This declaration includes, among other things, substituting types that the runtime understands for the method's types. The developer can call these methods from his code as if they were NGWS runtime methods.
  2. The second approach entails writing an NGWS class that is just a collection of declarations (like the one described above) for an API exported by an unmanaged DLL. A developer could provide this class for her own use, or she could use a class provided by another party. The developer that uses this type of class has the advantage of not having to declare and annotate the methods for the unmanaged DLL that she wants to call. All the developer has to do is call them as if they were regular NGWS runtime methods.

In both cases, the code that invokes such an annotated method need not be aware of the underlying implementation. The annotated method looks and behaves like a managed NGWS runtime method, but when the Execution Engine encounters the method, Platform Invocation Services makes the transition from managed to unmanaged code and invokes the unmanaged method.


Authoring Semantics for PInvoke

The specific syntax used to annotate the method declarations will vary by language. This document gives examples using the VC7 syntax. However, it might be instructive to present a more general idea of just what any NGWS runtime enabled language will be (or need to be) supporting.

The key thing is the ability to annotate a method with information used by Platform Invocation Services to correctly marshal the data and call the appropriate underlying DLL entry point. A component or application developer will declare the method with a signature that consists solely of valid NGWS runtime managed data types. The developer will annotate that method to flag it as not having a local implementation but, rather, as having an unmanaged implementation that can be accessed via a call to an unmanaged API. Although a particular tool may provide some defaults and/or syntactic “sugar”, the annotations will consist of:

  • Name/identity of the exporting module.
  • Name or entry point ordinal of the function, if different from the NGWS runtime method name.
  • Optional flags to clarify the underlying calling convention: ansi (foo becomes fooA), unicode (foo becomes fooW), auto (Platform Invocation Services chooses the appropriate ANSI or Unicode entry point at runtime based on the target platform), or ole (signature is mangled to return HRESULT and move return value to retval parameter position). The ansi / unicode flags are also used to auto-marshal string data types correctly.

If any of the parameters is an unmanaged struct, additional information is required to correctly marshal the struct, preserving layout and alignment. In the NGWS runtime, the unmanaged struct is represented as an NGWS class (the language may elect not to expose this fact) that embodies the layout rules for the struct.


PInvoke Samples - VC7 Compiler

VC7 allows you to annotate your C++ source code in order to call unmanaged functions automatically, using PInvoke. The annotation is specified with the [sysimport] directive. This section show a few simple examples of how to call unmanaged code from your managed C++ programs.


Puts

Below is a simple ‘Hello World’ example. This shows how you use a [sysimport] directive to ‘annotate’ your code. You supply three ‘arguments’ to the [sysimport] directive:

  • dll to specify the DLL that holds your target function (msvcrt.dll)
  • name to specify the entry point in the DLL you want (“puts”). You could miss this out: PInvoke would then just use name of the function prototype that follows (“puts”).
  • charset to specify how managed Strings should be marshaled: in this case, by saying “ansi”, PInvoke will marshal any string arguments from their managed format (Unicode) to “ansi” on the unmanaged side.

This information tells the C++ compiler all it needs to emit the correct PInvoke metadata into the object file it is building for you.

#import <mscorlib.dll>     // to get Microsoft::Runtime::String

[sysimport(dll="msvcrt", name="puts", charset="ansi")]
int puts(String*);

void main(void) {
     String *pStr = L"Hello World!";
     puts(pStr);
}

Notice how the managed String is Unicode, and therefore prefixed with “L” and that charset=”ansi”.


MessageBox

Below is another “hello world” example. Rather than “puts” which writes to a console window, this example calls the windows MessageBox function, provided as part of the Win32 API in USER32.DLL:

#import <mscorlib.dll>        // to get Microsoft::Runtime::String
typedef void* HWND;
[sysimport(dll="user32", charset="ansi")]
extern "C" int MessageBox(HWND hWnd,
                          String*  pText,
                          String*  pCaption,
                          unsigned int uType);
void main(void) {
     String* pText = L"Hello World!”;
     String* pCaption = L"PInvoke Test";
     MessageBox(0, pText, pCaption, 0);
}

Notice that the function is declared as MessageBox, even though USER32.DLL exports only MessageBoxA and MessageBoxW (but no MessageBox). PInvoke notes that you specified a charset of ”ansi”, and therefore looks for “MessageBoxA” if it cannot find “MessageBox”. Similarly, if you had specified charset=”unicode”, PInvoke would have found MessageBoxW for you automatically. [charset therefore specifies two types of behaviour – how PInvoke marshals text strings, and how it looks up function names in the DLL. It is possible to have the first, without the second, by setting a “NoMangle” flag. This is used implicitly by VB7, but not exposed by the VC7 compiler]


GetSystemDirectory

This example illustrates how the charset ‘argument’ to [sysimport] can be used to provide explicitly ask for ANSI or Unicode versions of a particular unmanaged function – in this case, the GetSystemDirectory in the Win32 API. The different versions of GetSystemDirectory pass back in the parameter a string in different forms, ANSI or Unicode; however, Platform Invocation Services transforms this into an NGWS runtime string, based on the usage of “ansi”, “unicode” or “auto” on charset.

#import <mscorlib.dll>
typedef unsigned __int32 UINT;

[sysimport(dll="kernel32", charset="ansi")]
extern "C" UINT GetSystemDirectoryA(StringBuilder* lpBuf, UINT uSize);

[sysimport(dll="kernel32", charset="unicode")]
extern "C" UINT GetSystemDirectoryW(StringBuilder *lpBuf, UINT uSize);

[sysimport(dll="kernel32", name="GetSystemDirectory", charset="auto")]
UINT MyMethod(StringBuilder *lpBuf, UINT uSize);

#if _UNICODE
     [sysimport(dll="kernel32", charset="unicode")]
     extern "C" UINT GetSystemDirectory(StringBuilder *lpBuf, UINT uSize);
#else
     [sysimport(dll="kernel32", charset="ansi")]
     extern "C" UINT GetSystemDirectory(StringBuilder *lpBuf, UINT uSize);
#endif

#define MAX_STRING_PATH 256

int main(void) {
     StringBuilder *pSysDir = new StringBuilder(MAX_STRING_PATH);

     GetSystemDirectoryW(pSysDir, MAX_STRING_PATH);
     Console::WriteLine(pSysDir);

     GetSystemDirectoryA(pSysDir, MAX_STRING_PATH);
     Console::WriteLine(pSysDir);

     MyMethod(pSysDir, MAX_STRING_PATH);
     Console::WriteLine(pSysDir);

     GetSystemDirectory(pSysDir, MAX_STRING_PATH);
     Console::WriteLine(pSysDir);

     return 0;
}

This program defines four managed methods, called:

  • GetSystemDirectoryA – this will call the Win32, unmanaged function called GetSystemDirectoryA, which generates an Ansi string as its answer. PInvoke will marshal these Ansi characters back to a Unicode StringBuilder in our managed calling program
  • GetSystemDirectoryW – this will call the Win32, unmanaged function called GetSystemDirectoryW, which generates a Unicode string as its answer. PInvoke will copy these Unicode characters back to a Unicode StringBuilder in our managed calling program
  • MyMethod – because it specifies a charset of “auto”, PInvoke at runtime will look up GetSystemDirectoryA (if executing on an ANSI platform), or GetSystemDirectoryW (if executing on a Unicode platform). The string marshaling will then work as described above. This example illustrates how we can call the managed function something completely different from the entry point in the DLL
  • GetSystemDirectory – we decide, at compile time, to ask from Unicode or ANSI marshaling. No need to append an “A” or a “W” – PInvoke takes care of that for you

Note that we specify each function with extern “C” – this tells VC7 to emit the plain, unmangled name of the method that follows, into the object file. If you miss this out, then VC7 will mangle the name, writing out a string that begins “?GetSystemDirectoryA@@YA_”. So the program will compile and link ok, but at runtime PInvoke will not be able to find that function.


GetSystemInfo

This example shows how to define a managed struct, and pass it as an argument to the GetSystemInfo routine in the Win32 API. GetSystemInfo fills in the struct and returns that information to the managed caller.

[Note that this example is particularly simple: each field in the managed struct is a simple 2-byte or 4-byte ‘number’ that matches what the unmanaged code in GetSystemInfo expects to see. More complex examples define structs where the field sizes do not match: for example, a managed BOOLEAN (1-byte) corresponding to an unmanaged C int (4-byte)]

The example includes typedefs for the WORD, DWORD, UINT and LPVOID types found so often in the Win32 API. This is not necessary (for example, we could have simply used “unsigned short” everywhere instead of WORD) but it makes the code look more familiar to Win32 users.

#import <mscorlib.dll>
typedef unsigned short WORD;
typedef unsigned long DWORD;
typedef unsigned int UINT;
typedef void *LPVOID;

[managed] struct SYSTEM_INFO {
     DWORD  dwOemId;
     DWORD  dwPageSize;
     LPVOID lpMinimumApplicationAddress;
     LPVOID lpMaximumApplicationAddress;
     DWORD  dwActiveProcessorMask;
     DWORD  dwNumberOfProcessors;
     DWORD  dwProcessorType;
     DWORD  dwAllocationGranularity;
     WORD   wProcessorLevel;
     WORD   wProcessorRevision;
};

[sysimport(dll="kernel32")]
extern "C" void GetSystemInfo(SYSTEM_INFO* pSI);

int main(void) {
     SYSTEM_INFO* pSI = new SYSTEM_INFO;
     GetSystemInfo(pSI);
     Console::Write(L"The number of CPUs: ");
     Console::WriteLine((long)pSI->dwNumberOfProcessors);
     return 0;
}


EnumWindows

This example shows how unmanaged code, called via PInvoke, can call back again into managed code. It uses the familiar EnumWindows function from the Win32 API. Its signature is:

BOOL EnumWindows (WNDENUMPROC lpEnumFunc, LPARAM lParam)

When you call EnumWindows, the operating system steps through the list of windows on your computer. For each one, it calls the function you provide as the lpEnumFunc argument. You can do whatever you like in this function; typically, you might print out the name of the window. When you return from that function, the operating system continues its search.

The function you supply as the lpEnumFunc argument is known as a callback, for obvious reasons – the operating system calls you back for each window it finds. If your callback returns TRUE, the operating system continues its enumeration; if not, it stops.

Below is a tiny example of how you might use EnumWindows; this is written totally written in unmanaged, VC6:

#include <windows.h>
#include <stdio.h>

BOOL CALLBACK MyCallBack(HWND hwnd, LPARAM lParam) {
   printf("Window handle is %x \n", hwnd);
   return TRUE;
}

void main(void) { EnumWindows(MyCallBack, 0); }

Each window that EnumWindows finds, it calls MyCallBack. This simply prints out the value of the window handle, and returns TRUE to continue the enumeration.

We now rewrite this code to work with VC7, as follows:

#import <mscorlib.dll>

[delegate] bool CallBack(int hwnd, int lParam);  // a delegate type

bool Report(int hwnd, int lParam) {            // report the window handle
   Console::Write(L"Window handle is ");
   Console::WriteLine(hwnd);
   return true;
}

[sysimport(dll="user32")]
extern "C" int EnumWindows(CallBack* x, int y);  //

void main(void) {
   CallBack* myCallBack = new CallBack(Report);
   EnumWindows(myCallBack, 0);
}

We have added a [sysimport] directive to say that the EnumWindows function is to be found, as unmanaged code, in the user32.dll. Also, we have declared a delegate type, called “CallBack” which takes two int arguments and returns a bool. You can think of a delegate as like a function pointer in VC6.


Custom Marshaling

PInvoke allows you to over-ride the default marshaling provided for function return values, for function arguments, and for fields within structs used as function arguments. For example, if you want a managed String marshaled, not to an LPTSTR (the default), but to a an LPBSTR, this is possible by saying so in the metadata.

However, as of this release, neither VC7 nor VB7 expose this feature to their users. This spec will be updated to include the relevant syntax once it becomes available.

For a full description of how PInvoke (and COM Interop) data is marshaled, see the “Data Marshaling” Spec.


Name Matching Algorithm

PInvoke looks up the name of the unmanaged function in the DLL you specify, in order to load and call that code. However, in order to make things easier for the programmer, PInvoke actually carries out a fuzzy match of the name provided. For example, you may say you want to call the function “MessageBox” in the user32.dll, but PInvoke will actually call MessageBoxA (the version that accepts ANSI strings) or MessageBoxW (the version that accepts wide, or Unicode, strings), depending upon whether you are executing on a platform where strings are ANSI or Unicode. In fact, there is no such function as “MessageBox” in user32.dll at all.

The function name provided in the [sysimport] directive can be in one of two forms: either a regular name, such as “MessageBox”, or the ordinal of the target function in the DLL, specified as a decimal number preceded by #. For example, “#123”.

If the function ‘name’ is its ordinal, then PInvoke simply calls a GetProcAddress for that ordinal number.

If the function name is a regular name, then PInvoke does a match. For example, suppose the code asks for function “Foo”. Then:

  1. Ansi – if “Foo” exists return “Foo” elseif “FooA” exists return “FooA”
  2. Unicode – if “FooW” exists return “FooW” elseif “Foo” exists return “Foo”
  3. Auto – follow 1. or 2. above, depending on whether the platform is Ansi or Unicode
  4. Still not found? – if “_Foo@7” exists return “_Foo@7” where 7 is the number of bytes required for the parameter list.

If the noMangle bit is specified however, the matching is exact – given “Foo” we look only for “Foo”.

Note that the matching rule for Ansi or Unicode is quite different:

  • with Ansi, asking for “Foo” returns “Foo” if it exists; only where “Foo” is absent from the DLL, but “FooA” is present do we return “FooA”. So, given a choice in the DLL of “Foo” or “FooA”, we choose “Foo”
  • with Unicode, asking for “Foo” returns “FooW” if it exists; only where “FooW” is absent from the DLL, but “Foo” is present do we return “Foo”. So, given a choice in the DLL or “Foo” or “FooW”, we choose “FooW”.


Default PInvoke Marshaling

For arguments that are simple data types, such as bytes, or integers, PInvoke marshals the managed argument to the corresponding unmanaged data type. The next table summarizes the rules.

If you need to marshal the data in a different way, simply wrap the call with a routine that does the required marshaling.


Simple Data Type Marshaling

This table shows how the data types used in the Win32 API, should be expressed in Managed C++ (VC7) and Managed VB (VB7) in order for PInvoke to correctly marshal your data.

wtypes.h | Unmanaged C | VC7 | VB7 | Additional Info
HANDLE | void* | void* | Long |
BYTE | unsigned char | unsigned char | Byte |
SHORT | short | short | Integer | 2 bytes
WORD | unsigned short | unsigned short | Integer | 2 bytes
INT | int | int | Long | 4 bytes
UINT | unsigned int | unsigned int | Long | 4 bytes
LONG | long | int | Long | 4 bytes
BOOL | long | int | Long | 4 bytes
DWORD | unsigned long | unsigned int | Long | 4 bytes
ULONG | unsigned long | unsigned int | Long | 4 bytes
CHAR | char | char | Char |
LPSTR | char* | String for in, StringBuilder* for inout | String | decorate with ansi
LPCSTR | const char* | String | String | decorate with ansi
LPWSTR | wchar_t* | String for in, StringBuilder* for inout | String | decorate with unicode
LPCWSTR | const wchar_t* | String | String | decorate with unicode
FLOAT | float | float | Single | 4 bytes
DOUBLE | double | double | Double | 8 bytes

Marshaling Numbers

For simple number types, PInvoke simply copies the bits, without any conversion. This includes the NGWS runtime data types for 1, 2, 4, and 8 byte integers; 1, 2, 4, and 8 byte unsigned integers; and 4 and 8 byte real numbers.

The NGWS runtime also supports data types for agnostic-sized integers and reals.

Marshaling Strings

PInvoke copies string arguments, converting from the NGWS runtime format (Unicode) to the platform unmanaged format. Strings are treated as immutable, so they are not copied back from unmanaged to managed space when the function returns.

On the other hand, PInvoke does copy back any argument of type StringBuilder, since this is a mutable string. Notice that this simulates "by-reference" semantics by using copy-in/copy-out.

Marshaling Agnostic Pointers

These are not permitted to be pointers to garbage-collected (managed) memory. They are copied directly, providing a mechanism for passing references to (unmanaged) data.

Marshaling Objects

PInvoke handles arguments that are specified as Object (the root of the VOS class hierarchy) as a special case. Specifying an argument as Object indicates that the data type isn't known at compile or link time, but can be determined at runtime. The value being passed must be a managed object (this will be enforced by static type checking) and must be one of the following types (which are then handled as described separately):

  • An array of elements, passed by copy-in/copy-out
  • A string, passed by copy-in.
  • A Microsoft.Runtime.StringBuilder, passed by copy-in/copy-out.

Marshaling Interfaces

Platform Invocation Services handles arguments that are marked as an NGWS runtime interface type by extracting the "classic COM" interface pointer and passing it to the unmanaged code. Interfaces are immutable and are therefore not copied back out.


Marshaling Arrays

Arrays can be passed, by copy-in/copy-out, if they have elements of any of the "Primitive Types": boolean, char, integer (1, 2, 4, or 8 bytes, signed or unsigned), real number (4 or 8 bytes), or string.

Beware: if you call an unmanaged function, passing a byref array argument, PInvoke copies all its elements to an unmanaged buffer. However, when copied back, PInvoke has no longer knows the size of that unmanaged buffer, so it copies back just 1 element. On return, the array has been resized to 1.


Marshaling Delegates and Callbacks

A callback is simply a call, back into managed code, from an unmanaged function you called earlier, via PInvoke. If all the code were unmanaged, you would specify the callback as an argument, of type pointer-to-function. However, when the callback crosses the boundary between unmanaged and managed code, you must supply the callback as a Delegate.

PInvoke converts a delegate object into a callback thunk. Currently, the arguments to the delegate / callback are only permitted to be integers.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Platform Invocation Services VC7 Syntax

© 2000 Microsoft Corporation. All rights reserved.


[sysimport] attribute

This attribute is only allowed on method declarations. It specifies that the attributed function is actually an entrypoint into some unmanaged DLL. It takes a number of arguments:

dll | Required. Names the DLL: eg dll=”foo.dll”. If no extension is specified then “.dll” is assumed
name | Optional. Name of the actual entrypoint in the DLL: eg name=”MyMethod”. If entrypoint is the decorated function name (e.g. _MyMethod@12), then the argument is not required. To specify an ordinal entrypoint instead of a name, specify name="#123". The ordinal is prefixed with a “#”
charset | Optional. Example charset=”ansi”, “unicode”, “auto”, or “none”. Specifies how String and StringBuilder arguments are to be marshaled. If not specified, defaults to ansi


Exceptions in Unmanaged Code

When exceptions are thrown in unmanaged code, they are reflected back to managed caller.


PInvoke Security

Code compiled for the NGWS runtime uses Code Access Security. So, the runtime environment checks the code to see it has the requisite permissions before it tries to access a resource or perform other sensitive tasks. However, this level of checking is not available with unmanaged code at runtime. Hence, any PInvoke calls to unmanaged code must be checked for security implications and care must be taken to ensure that no security holes are introduced. The Execution Engine introduces the following constraints:

  • The calling NGWS class must be fully trusted. This means that the class must be signed with “Full Trust” access and that the administration policies have allowed to code to run on the system with full trust. This level of security is essential since the unmanaged code runs with no runtime checks
  • Any code calling a class that makes PInvoke calls must be fully trusted. Since the Execution Engine walks up the stack to ensure that all code in the call chain has the right level of trust, all NGWS runtime code that is on a chain that ultimately makes an PInvoke call must also be fully trusted


PInvoke Annotations and Implications on VOS Inheritance

All methods whose implementation is mapped to an underlying DLL entry point must be declared as static methods. All VOS inheritance rules for static methods apply.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Developer Services for Platform Invocation Services

© 2000 Microsoft Corporation. All rights reserved.


CORDBG - Debugging

The command-line debugger, CORDBG, that ships with the NGWS SDK, allows you to debug managed code. However, it does not step from managed code into unmanaged code; it jumps over the entire unmanaged code.


METAINFO - Inspecting Metadata

The utility called METAINFO, produces a listing of the file’s metadata. Below is an example of its output for the MessageBox example shown earlier. This output is produced with the command:

Metainfo MessageBox.exe

To view the several options supported by Metainfo, type:

Metainfo /?
Microsoft (R) NGWS SDK Runtime Meta Data Dump Utility   Version 1999.13.1020.9
Copyright (C) Microsoft Corp. 1998-1999
////////////////////////////////////////////////////////////////

File messagebox.exe:
ScopeName :
GUID      : {00000000-0000-0000-0000-000000000000}
MVID      : {60C6BAD1-013A-4E19-9C0E-36921AA0EFCD}
   CustomAttribute #1 (0c000001)
   -------------------------------------------------------
      CustomValueName: System.Runtime.Diagnostic.SymbolStore.CORDB_SymbolStoreMatchingReader
      Value Blob length : 16
      Value     : c2 96 82 10 1e 28 d3 11 bd 22 00 00 f8 08 49 bd

===========================================================
Global functions
-------------------------------------------------------
   Method #1
   -------------------------------------------------------
      MethodName: main (06000001)
      Flags     : [Public] [Static] [ReuseSlot]  (00000011)
      RVA       : 0x00001000
      ImplFlags : [IL] [Managed] [Implemented]  (00000011)
      CallCnvntn: [DEFAULT]
      ReturnType: Void
      No arguments.

   Method #2
   -------------------------------------------------------
      MethodName: _mainMSIL (06000002)
      Flags     : [Public] [Static] [ReuseSlot]  (00000011)
      RVA       : 0x00001040
      ImplFlags : [IL] [Managed] [Implemented]  (00000011)
      CallCnvntn: [DEFAULT]
      ReturnType: I4
      3 Arguments
         Argument #1:  I4
         Argument #2:  Ptr Ptr I1
         Argument #3:  Ptr Ptr I1

   Method #3
   -------------------------------------------------------
      MethodName: MessageBox (06000003)
      Flags     : [Public] [Static] [ReuseSlot] [PinvokeImpl]  (00002011)
      RVA       : 0x00000000
      ImplFlags : [IL] [Managed]  (00000001)
      CallCnvntn: [DEFAULT]
      ReturnType: I4
      4 Arguments
         Argument #1:  Ptr Void
         Argument #2:  Class String
         Argument #3:  Class String
         Argument #4:  UI4
      4 Parameters
         (1) MethodToken : (06000003) Name : hWnd flags: [none] (00000000) default:
         (2) MethodToken : (06000003) Name : pText flags: [none] (00000000) default:
         (3) MethodToken : (06000003) Name : pCaption flags: [none] (00000000) default:
         (4) MethodToken : (06000003) Name : uType flags: [none] (00000000) default:
      MemberRef #1
      -------------------------------------------------------
         Member: (0a000002) MessageBox:
         CallCnvntn: [DEFAULT]
         ReturnType: I4
         4 Arguments
            Argument #1:  Ptr Void
            Argument #2:  Class String
            Argument #3:  Class String
            Argument #4:  UI4
         CustomAttribute #1 (0c000005)
         -------------------------------------------------------
            CustomValueName: __CustomMetaData
            Value Blob length : 4
            Value     : 04 00 00 00

         CustomAttribute #2 (0c000006)
         -------------------------------------------------------
            CustomValueName: __DecoratedName
            Value Blob length : 11
            Value     : 4d 65 73 73 61 67 65 42 6f 78 00

      Pinvoke Map Data:
      Entry point:      MessageBox
      Module ref:       1a000001
      Mapping flags:    [CharSetAnsi]  (00000002)
      Ordinal:          00000000

Global fields
-------------------------------------------------------

Global MemberRefs
-------------------------------------------------------

TypeDef #1
-------------------------------------------------------
   TypDefName: __IsConst  (02000002)
   Flags     : [Public] [AutoLayout] [Class] [Sealed] [AnsiClass]  (00000101)
   Name Space: __Microsoft_Visual_CPP__
   Class ID  : {00000000-0000-0000-0000-000000000000}
   Version   : 0:0:0:0
   Extends   : 01000001 [TypeRef] Object

TypeDef #2
-------------------------------------------------------
   TypDefName: __IsVolatile  (02000003)
   Flags     : [Public] [AutoLayout] [Class] [Sealed] [AnsiClass]  (00000101)
   Name Space: __Microsoft_Visual_CPP__
   Class ID  : {00000000-0000-0000-0000-000000000000}
   Version   : 0:0:0:0
   Extends   : 01000001 [TypeRef] Object

TypeDef #3
-------------------------------------------------------
   TypDefName: __IsReallyLong  (02000004)
   Flags     : [Public] [AutoLayout] [Class] [Sealed] [AnsiClass]  (00000101)
   Name Space: __Microsoft_Visual_CPP__
   Class ID  : {00000000-0000-0000-0000-000000000000}
   Version   : 0:0:0:0
   Extends   : 01000001 [TypeRef] Object

TypeDef #4
-------------------------------------------------------
   TypDefName: __IsReallySigned  (02000005)
   Flags     : [Public] [AutoLayout] [Class] [Sealed] [AnsiClass]  (00000101)
   Name Space: __Microsoft_Visual_CPP__
   Class ID  : {00000000-0000-0000-0000-000000000000}
   Version   : 0:0:0:0
   Extends   : 01000001 [TypeRef] Object

TypeDef #5
-------------------------------------------------------
   TypDefName: __CustomMetaData  (02000006)
   Flags     : [Public] [AutoLayout] [Class] [Sealed] [AnsiClass]  (00000101)
   Name Space: __Microsoft_Visual_CPP__
   Class ID  : {00000000-0000-0000-0000-000000000000}
   Version   : 0:0:0:0
   Extends   : 01000001 [TypeRef] Object

TypeDef #6
-------------------------------------------------------
   TypDefName: __DecoratedName  (02000007)
   Flags     : [Public] [AutoLayout] [Class] [Sealed] [AnsiClass]  (00000101)
   Name Space: __Microsoft_Visual_CPP__
   Class ID  : {00000000-0000-0000-0000-000000000000}
   Version   : 0:0:0:0
   Extends   : 01000001 [TypeRef] Object

TypeDef #7
-------------------------------------------------------
   TypDefName: $MultiByte$24  (02000008)
   Flags     : [Public] [ExplicitLayout] [ValueType] [AnsiClass]  (00000051)
   Name Space:
   Class ID  : {00000000-0000-0000-0000-000000000000}
   Version   : 0:0:0:0
   Extends   : 01000001 [TypeRef] Object
   Layout    : Packing:1, Size:24

TypeDef #8
-------------------------------------------------------
   TypDefName: $MultiByte$26  (02000009)
   Flags     : [Public] [ExplicitLayout] [ValueType] [AnsiClass]  (00000051)
   Name Space:
   Class ID  : {00000000-0000-0000-0000-000000000000}
   Version   : 0:0:0:0
   Extends   : 01000001 [TypeRef] Object
   Layout    : Packing:1, Size:26

TypeRef #1
-------------------------------------------------------
ResolutionScope: 0x1a000000
NameSpace:   System
TypeRefName: Object

TypeRef #2
-------------------------------------------------------
ResolutionScope: 0x1a000000
NameSpace:   System
TypeRefName: String
   MemberRef #1
   -------------------------------------------------------
      Member: (0a000001) .ctor:
      CallCnvntn: [DEFAULT]
      hasThis
      ReturnType: Void
      1 Arguments
         Argument #1:  Ptr Char

TypeRef #3
-------------------------------------------------------
ResolutionScope: 0x1a000000
NameSpace:   System.Runtime.Diagnostic.SymbolStore
TypeRefName: CORDB_SymbolStoreMatchingReader

TypeRef #4
-------------------------------------------------------
ResolutionScope: 0x1a000000
NameSpace:   System.Runtime.Diagnostic.SymbolStore
TypeRefName: CORDB_SymbolStoreDocumentLanguage

TypeRef #5
-------------------------------------------------------
ResolutionScope: 0x1a000000
NameSpace:   System.Runtime.Diagnostic.SymbolStore
TypeRefName: CORDB_SymbolStoreDocumentLanguageVendor

TypeRef #6
-------------------------------------------------------
ResolutionScope: 0x1a000000
NameSpace:   System.Runtime.Diagnostic.SymbolStore
TypeRefName: CORDB_SymbolStoreDocumentType

ModuleRef #1
-------------------------------------------------------
      ModuleRef: (1a000001) user32.dll:
      GUID : {00000000-0000-0000-0000-000000000000}
      MVID : {00000000-0000-0000-0000-000000000000}

Signature #1
-------------------------------------------------------
      CallCnvntn: [LOCALSIG]
      2 Arguments
         Argument #1:  Class String
         Argument #2:  Class String

Observe that the output of METAINFO is language-neutral. That is, the data types and PInvoke annotations that the VC7 developer typed were emitted by the VC7 compiler into metadata using the NGWS runtime data types in the signatures and metadata for the PInvoke information.


ILDASM - Inspecting Code

Just as you can use the METAINFO utility to inspect the NGWS runtime metadata inside a file, you can use the ILDASM to inspect (disassemble) its code. Below is an example of its output for the MessageBox example shown earlier. This output is produced with the command:

Ildasm MessageBox.exe

To view the several options supported by ILDASM, type:

Ildasm /?
//  Microsoft (R) NGWS SDK IL Disassembler.  Version 1999.13.1020.9
//  Copyright (C) Microsoft Corp. 1998-1999

// VTableFixup Directory:
// No data.
.namespace __Microsoft_Visual_CPP__
{
  .class public auto ansi sealed __IsConst
  {
  } // end of class '__IsConst'

  .class public auto ansi sealed __IsVolatile
  {
  } // end of class '__IsVolatile'

  .class public auto ansi sealed __IsReallyLong
  {
  } // end of class '__IsReallyLong'

  .class public auto ansi sealed __IsReallySigned
  {
  } // end of class '__IsReallySigned'

  .class public auto ansi sealed __CustomMetaData
  {
  } // end of class '__CustomMetaData'

  .class public auto ansi sealed __DecoratedName
  {
  } // end of class '__DecoratedName'

} // end of namespace '__Microsoft_Visual_CPP__'

.class value public explicit ansi $MultiByte$24
{
  .pack 1
  .size 24
} // end of class '$MultiByte$24'

.class value public explicit ansi $MultiByte$26
{
  .pack 1
  .size 26
} // end of class '$MultiByte$26'

//Global methods
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.method public static void main() il managed implemented
{
  .entrypoint
  // Method begins at RVA 0x1000
  // Code size       45 (0x2d)
  .maxstack  4
  .locals (class System.String pCaption,
           class System.String pText)

  IL_0000:  ldnull
  IL_0001:  stloc.0
  IL_0002:  ldnull
  IL_0003:  stloc.1
  IL_0004:  ldptr      D_00003000
  IL_0009:  newobj     instance void System.String::.ctor(wchar*)
  IL_000e:  stloc.0
  IL_000f:  ldptr      D_00003018
  IL_0014:  newobj     instance void System.String::.ctor(wchar*)
  IL_0019:  stloc.1
  IL_001a:  ldc.i4     0x0
  IL_001f:  ldloc.0
  IL_0020:  ldloc.1
  IL_0021:  ldc.i4     0x0
  IL_0026:  call       int32 MessageBox(void*,class System.String,class System.String,unsigned int32)
  IL_002b:  pop
  IL_002c:  ret
} // end of global method 'main'

.method public static int32 _mainMSIL(int32 A_0,int8** A_1,int8** A_2) il managed implemented
{
  // Method begins at RVA 0x1040
  // Code size       11 (0xb)
  .maxstack  1

  IL_0000:  call       void main()
  IL_0005:  ldc.i4     0x0
  IL_000a:  ret
} // end of global method '_mainMSIL'

.method public static pinvokeimpl("user32.dll" ansi) int32 MessageBox(void* hWnd,class System.String pText,class System.String pCaption,unsigned int32 uType) il managed{ }
.data D_00003000 = bytearray (
                 48 00 65 00 6C 00 6C 00 6F 00 20 00 57 00 6F 00  // H.e.l.l.o. .W.o.
                 72 00 6C 00 64 00 00 00)                         // r.l.d...
.data D_00003018 = bytearray (
                 50 00 49 00 6E 00 76 00 6F 00 6B 00 65 00 20 00  // P.I.n.v.o.k.e. .
                 54 00 65 00 73 00 74 00 00 00)                   // T.e.s.t...
//*********** DISASSEMBLY COMPLETE ***********************


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

NGWS Runtime Security Policy Specification

© 2000 Microsoft Corporation. All rights reserved.


Introduction to Security Policy

Policy uses security configuration based on evidence from code to decide if it is safe to run it, and if so to grant permissions to it. This document describes the policy model, how the policy mechanism works, and the programming model for administration of policy.

Security policy allows the administrator to restrict the permissions granted to code running on the machine. The policy system looks at evidence about all managed code – where it comes from, name or publisher identity, and so forth – as the basis of a trust decision about the code, that results in a set of permissions the system is willing to grant the code. The code itself may require certain permissions, and only when policy grants all permissions required is the code actually allowed to run. Additionally, code may optionally request more permissions or may explicitly refuse to be granted certain permission. Thus, the interplay between security policy and code requests provides a negotiation of permissions suitable to both the system as configured and the code itself.


Security policy model

The function of security policy is to grant permissions to code based on evidence. The policy model consists of declarations of code groups with sets of permissions associated to be granted to any code that is a member of the group. Code group membership is determined by evidence about the code such as digital signature, the origin of the code, and so forth. Administrators add code groups to the default policy to further control the permissions granted to code belonging to the respective groups.

The security policy model is structured as a policy hierarchy – with machine and user levels – each level of which consists of a code group hierarchy as described in detail in the following sections.


Security Policy hierarchy

Security policy is structured as a hierarchy whose highest level is the policy that applies to all code running on a given machine. Beneath that is the user level that is compromised of individual policies for the different users on that machine. At the lowest level—the app domain policy level—a host can specify a policy that applies only to the code running within that app domain (see the NGWS Runtime Application Domains specification for more detail on app domains). Machine, user and app domain policy (if set by the host) are considered together so that only those permissions allowed by all are granted to code. In this way, each level of the security policy hierarchy can further restrict the final permission grant but no one level can cause code to be granted permissions not allowed by the others.

Each policy in the security policy is a code group hierarchy as described in the following section.


Code group hierarchy

Code groups are defined in a hierarchy whose root is the group containing all code. If code is a member of the parent code group then its child code groups are tested for membership. Otherwise, a given piece of code cannot be a member of any of the descendent code groups below a parent code group it does not belong to. A simple example follows to illustrate the hierarchy of code groups concept.

Evaluation of code groups for a particular piece of code begins at the top – all code belongs to the root code group, so next membership in the four immediate child groups is tested (in any order). Suppose in this example code is published by Microsoft and is in the Intranet zone – the leftmost and second from right code groups in the diagram above. Next the children of these nodes only are tested – suppose this code has strong name “MS.Office” and is from site “localweb”. If these groups had children they would be further tested, but since they don’t the process completes. Result is that the code belongs to these code groups: All code, Publisher: Microsoft, Zone: Intranet, Strong name: “MS.Office”, and Site “localweb”. The permission sets associated with these code groups would then be combined together to determine the set of permissions allowed this code.

The arrangement of code groups into a hierarchy allows the policy configuration to be partitioned and refined – child code groups of a parent code group are only going to impact code that is a member of the parent. The hierarchy model allows administrators to manage policy configuration in terms of groups of code with further levels of refinement (in a flat model, the potential interaction of all code groups makes it much more difficult to manage as the number of groups grows). A performance benefit is gained as well since as code group membership is evaluated, large sub-trees can be excluded from further evaluation making the policy operation correspondingly faster.

Thus the permission sets (and therefore the permissions) policy may grant code are determined from the code groups it belongs to (see Policy decision and permission grant process for more detail). This allows the administrator to configure policy to trust some code more than others by making that code a member of a group with additional permissions associated with it.

The security policy hierarchical model does not allow multiple inheritance, however there are strategies administrators can use to achieve similar aims. First, instead of putting a code group under multiple parent code groups, consider if the code group can be elevated to be a peer of those code groups. Alternatively, often the new code group can become the single parent with existing code groups subsidiary to it: this achieves the same effect that code must belong to both but avoids duplicate code groups. Where the specificity of certain parent code groups is necessary, a common named permission set can be associated with new duplicate code groups under the respective parents.


Code groups

Code groups are defined by membership conditions, with any code passing the membership test being included in the group. Group membership is determined by code identity, based on evidence about the code such as digital signature or location.

Standard membership conditions include:

  • all code
  • software publisher, based on public key of a valid Authenticode™ signature
  • zone of origin of the code
  • strong name, based on cryptographically signed namespace
  • web site of origin of the code, e.g. www.microsoft.com or *.microsoft.com
  • URL of origin of the code, including final wildcard e.g. http://site/app/*


Special attributes of code groups

Exclusive: flag that means only the specified permission set should be allowed for members of the group, ignoring any other permission sets from other matching groups; it is an error for code to belong to more than one group with this attribute.

LevelFinal: flag that indicates no policy levels below the one containing this code group should be considered when checking code group membership and granting permissions. For example, if this flag is set for a code group at the machine level and some code matches the membership condition of this code group then the user level policy would not be applied to the code.


Named Permission sets

Named permission sets represent collections of permissions that may be assigned to code groups to grant those permissions to code in that group. More than one code group may be assigned the same named permission set. For example, to grant the same set of permissions to several extranet business partner sites, a single named permission set can be defined and assigned to several code groups for the respective sites.

A description can be associated with named permission sets to describe the permissions they define. There are three kinds of named permission sets:

  • standard permission sets that are pre-defined and cannot be changed,
  • pre-defined permission sets that may be modified by the administrator,
  • and user-defined custom permission sets (names of custom permission sets must not conflict with the standard or pre-defined permission sets).

In the hierarchical administration model, named permission sets are defined and used within a level and cannot be referenced from other levels. The standard and pre-defined permission sets are available at all.

Named permission sets may not contain identity permissions.

The standard permission sets are as follows:

  • Nothing: does not have permission to run
  • Execution: only permission to run, but no other permissions to allow use of protected resources
  • Everything: all standard permissions, except SkipVerification
  • SkipVerification: permission to run even if unverifiable, but no other permissions to allow use of protected resources.
  • FullTrust: unrestricted access to protected resources, including all standard permissions and any custom ones.

The pre-defined permission sets are:

  • Internet: the default policy permission set suitable for content from unknown origin
  • LocalIntranet: the default policy permission set for within an enterprise


Default policy for standard permission grants

The following table describes the default machine policy installed with the runtime. Shown are permissions granted by zone and the corresponding named permission set.

Permission | Zone
 | Local | Intranet | Trusted sites | Internet | Restricted sites
Environment | Unrestricted permissions | USERNAME TEMP TMP | — |  | —
FileDialog |  | unrestricted (read-only access) | unrestricted (read-only access) |  | —
FileIO |  | — | — |  | —
IsloatedStorage |  | AssemblyIsolationByUser unrestricted quota | DomainIsolationByUser 10240 bytes 365 days |  |
Reflection |  | unrestricted | — |  | —
Registry |  | — | — |  | —
Security |  | Execution, Assertion | Execution |  | —
UI |  | unrestricted | SafeSubWindows, OwnClipboard |  | —
 | Full-Trust | LocalIntranet | Internet |  | Nothing
 | Standard Permission Set


To turn security off

Policy setting for all-code that grants the Everything permission constitutes a special setting that effectively turns off security. Turning security off means that all permission checks are either skipped completely or succeed. (Use of this special case policy avoids need to a separate on/off state for security policy. Implementation may special-case this policy setting using an internal flag to skip security processing wherever possible.)


Evidence for policy

Evidence is the set of information that constitutes input to security policy decisions, such as what permissions can be granted to code. Common forms of evidence include signatures and location of origin of code, but can potentially be anything.

This section describes evidence and how it is represented and processed.


What is Evidence?

Objects of any type that are recognized by security policy represent evidence. Security policy is composed of code groups; a particular assembly (the basic unit of code for granting security permissions) is a member of a code group if it satisfies its membership condition. Evidence is the set of inputs to policy that membership conditions use to determine what code groups an assembly belongs to.

The Evidence class is a collection (ICollection) that holds a set of objects that represent evidence. (Herein big-e Evidence refers to the class; little-e evidence to objects that represent generic evidence.) The Evidence class holds two sets: host evidence and assembly evidence, corresponding to the source of the evidence.

Sources of evidence

Policy can get evidence from two different sources in evaluating permissions for code.

Host evidence is provided by the host, and can only be provided by hosts granted the SecurityPermission.ControlEvidence permission. Typically, this is evidence of location of origin of the code and digital signatures on the assembly.

Evidence about location origin typically includes URL, Site, and Zone evidence. Signatures refer to software publisher (AuthentiCode™) and strong name identities. Both kinds of digital signature based identity are built into the assembly, but must be validated and passed to policy by the host; when loaded the security system verifies the signature and only if it is valid will the corresponding evidence be collected and passed to policy.

Assembly evidence is part of the assembly itself. Developers or administrators can attach custom evidence to the assembly to extend the set of evidence for policy. See the section Extending evidence for additional details. (Note that signed assemblies may not be modified, and hence cannot have additional evidence added without breaking the existing signature.)

Evidence and identity permissions

Some – but not all – evidence types have associated identity permissions. Both the evidence object itself and the corresponding identity permission for it contain essentially the same information, but there is a subtle semantic distinction.

Parenthetically, also note that the related membership condition is described as well for completeness.

  • the evidence says, “this code has attribute XYZ”
  • the permission says, “this code is XYZ”
  • (the membership condition tests, “is this code a member of XYZ group?”)


Standard types of evidence

The following table describes the standard evidence types implemented by the runtime: the class of the basic evidence type, its source (explained below), a description, and the associated identity permission and membership condition.

The source column in the following table refers to the source of the evidence, either host or assembly, indicating that the evidence is either provided by host or derived from the bits of the assembly itself by the system.

Class | Source | Description | Identity permission | Membership condition
Publisher | Host | Software publisher signature | PublisherIdentityPermission | PublisherMembershipCondition
Site | Host | Site of origin | SiteIdentityPermission | SiteMembershipCondition
StrongName | Host | Strong name of assembly | StrongNameIdentityPermission | StrongNameMembershipCondition
URL | Host | URL of origin | URLIdentityPermission | URLMembershipCondition
Zone | Host | Zone of origin | ZoneIdentityPermission | ZoneMembershipCondition

All other types of evidence are “custom evidence”. Such custom evidence can be provided by either hosts or assemblies. Of course, such custom evidence can is only effective when there is policy that recognizes the additional evidence object types.


Extending evidence

Assemblies may provide evidence of their own – in addition to software publisher signatures and strong names – by including them as a resource. The “Security.Evidence” resource contains a single Evidence object serialized in binary format. The evidence objects inside the Evidence collection may be of any type but they may not be any of the standard evidence types listed above. (Inclusion of a Publisher identity, for instance, could be used to forge a publisher signature since this evidence is directly passed to policy without any kind of verification check.)

The LM tools supports addition of evidence to an assembly by use of the command:

LM –e evidencefile

The developer must write code to construct the appropriate evidence objects, collect them in an Evidence object, and serialize them to a file. This file is then included as a resource by the LM command line above.


Policy decision and permission grant process

Policy configuration consists of code group declarations (herein CG) with membership conditions so that code can be determined to be in a specific set of code groups. Each code group consists the defining membership conditions (MC), an allowed permission set (PA), and special attribute exclusive (EX). Thus,

CG = { MC, PA, EX }


Determining code group membership

Policy grants permissions to code on a per-assembly basis, and for each assembly the policy mechanism determines the subset of code groups in the configuration that it belongs to: CG1, CG2, CG3, … as detailed in the section Security Policy Hierarchy.


Aggregation of permission sets

If none of the matching code groups CG1, CG2, CG3 … contain the EX exclusive flag the PA allow permission sets are combined.

PA = PA1 + PA2 + PA3 + …

If exactly one code group has the exclusive flag –say CG1— then only that code group applies and PA=PA1.

It is considered an error for more than one code group that matches a given assembly to have the exclusive flag. Should this occur and exception is thrown and the code is not loaded.


Code permission requests

An assembly may contain declarative security code request to tell the security policy manager what permissions it needs or wants. Code request consists of three permission sets as follows:

  • RequestRequired (RM): to specify the minimum set of permissions it must have in order to run
  • RequestOptional (RO): to request additional permissions it would like in addition to the minimum set
  • RequestRefuse (RR): to specify permissions it never wants to be granted to it
  • If either RequestRequired or RequestOptional is absent, code will be granted all permissions allowed by policy minus any refused by RequestRefuse. If all three code requests are absent then code will get whatever policy allows. Typical code that uses protected resources should request mandatory permissions for the minimum set of permissions it makes sense to use the code in any limited situation. Attempted operations on protected resources needing permissions not in mandatory set may raise a PolicyException where the permission is not granted, which the code should be prepared to handle and provide a reasonable user experience.
  • Policy decides to run the code or not by the following tests – all must pass in order to run:

    PA contains the Execution permission

    RM is equal to or a subset of PA

  • If any of the above tests fail the security policy prevents the code from running.


Granting permissions

If policy determines to run code if will be granted a set of permissions based on trust as defined in the configuration. The granted permission set, G, is computed as follows:

G = RM + ( intersection of RO and PA ) – RR

In terms of the model, the granted permission set is the minimum request and some subset of the optional request as allowed by policy, less any permissions either policy or the code request refused.

Implementation note: instead of computing set subtraction for grant set G above, implementation may hold grant as two sets (RM + (RO PA)) and RR with permission check testing for inclusion in the former and checking that not included in the latter. This allows a grant to express a subtractive set (e.g. “C:\” except “C:\WINDOWS”) without having to express these complex sets in individual permissions.


Policy for application domains

Application domains are the unit of execution for single applications (this is a simplification for this discussion only). The fundamental goal of security on application domains is to ensure that code in the domain runs with restrictions appropriate to the domain, and that it cannot interact with applications in other domains (which in the case of managed code includes the case of being in the same process) except through well-defined interfaces to services and resource. For example use of the File object allows one domain to affect another on the same system; we strive for isolation of domains other than through this sort of service.

The prime motivation for this feature is to support the web browser security model. On the web, the DNS site name of a page (so-called “DOCBASE”) corresponds to the “application”. On a given page there may be a component such as an ActiveX control that may be downloaded from a different web site (so-called “CODEBASE”). By running the component (CODEBASE) in a domain with associated permissions derived from the application (DOCBASE), the system ensures that if these have different permissions only the lesser – the intersection – will apply.


Domain permissions

Domain permissions only apply to managed code hosted within the web browser.

All code in an application domain is restricted to permissions associated with the domain for purposes of code access security. In code access security, all callers up the stack must have the necessary permission to do an operation. Permissions are granted to application domains just as they are to code: in addition to all callers, the application domain must also have permission for a demand to succeed. In terms of the stack walk, the domain permissions are checked after the top of stack as reached (think of them as being at the “virtual top of stack”); overrides such as assert/deny may cause a demand to end prematurely, in which case domain permissions are not checked just as callers further up the stack may not be.

The most common case that illustrates the reason domains have permissions – in addition to code having them – is that of the web browser. A typical web page might have some text and graphics, script, and perhaps a control (like an ActiveX control). The web site of the page defines the application domain and through policy the permissions granted to it: usually web sites are not highly trusted so the permissions granted are modest and safe. The control, however, may already be installed on the machine and hence be given a higher level of trust than the web – most users trust code on their machine more than code on the web.

In our example when the script runs it gets permissions commensurate with being from the web, but the control gets higher permissions. When script calls the control the normal code access stack walk ensures that only operations allowed for the script’s lower permissions are possible: if script tries to make control do something it doesn’t have permissions for, the stack walk will see the script permissions and stop it. However, if the control runs at the top of the stack (not called by script) it has its full permissions intact: this is where the additional check of domain permissions knocks it down to the web site’s level of trust. This prevents a powerful control – that, say, displays and modifies private information – from running in the inappropriate context of an Internet site web page. While it may be argued that the control is running uninfluenced by web script and hence should get full rights, there are spoof attacks wherein the mere context of appearing in a web page is enough to confuse or trouble users so as to represent a problem. Note: it is possible for the control to assert permissions and potentially open up as security vulnerability that web pages could exploit (code access security depends on code only getting high levels of trust that do not have this sort of flaw).


Domain hosts

A domain host is the code that creates a domain and controls it. Typically, the same host that creates the domain is the only code to control it subsequently – loading code into it, for instance – although there is no strictly enforce restriction that only the same host may later do other operations. Any code with the necessary permissions (detailed below) and access to the object may perform any operations.

Domain hosts may or may not be trusted to provide evidence about code loaded in the assembly (requires SecurityPermission.ControlEvidence permission).

Security for domains created by hosts without SecurityPermission.ControlEvidence permission is determined by the security enforced on the host; any evidence passed to CreateDomain() is ignored. Thus, if a host does not have ControlEvidence permission then any domain it creates will have the same evidence as its host. (The ability to provide evidence potentially enables an escalation of permission; hence, this is a protected operation.)


Policy for the domain

The domain is the boundary for policy: hosts can influence policy on a domain basis within the same process. However, even the most trusted host cannot completely control policy – it will always be affected by the machine and user policy, over which the host has no influence. Hosts can control policy to the extent of possibly reducing permissions from the maximum allowed by machine policy, but never to increase it.

The basic structure of security policy applied to managed code is as shown in the diagram below – policy levels are shown from top (first evaluated) to bottom (subject to restrictions from above). Machine policy and per-user policy – if specified – will always apply. A host may provide additional policy that can only make further restrictions (never escalations) to the permissions that will ultimately be granted to code.

 |  | specified by | applies to
Machine policy | <— | administrator | all code on the machine
Per-user policy | <— | administrator | code in user’s processes
Host policy | <— | host code | code in host’s domain

Host provided policy

Typically, immediately after creating a new domain, a host specifies the policy it wants to provide to be applied to code within the domain. This host-specified policy is called application domain policy and may only be set once for an domain, so if a set has been previously done a new set operation will fail and have no effect. To provide application domain policy the host must have SecurityPermission.ControlDomainPolicy permission.

   AppDomain.SetAppDomainPolicy (PolicyLevel DomainPolicy);

Setting application domain policy will cause all subsequently loaded assemblies to be granted permissions under the new policy. All previously loaded policies will get grants under the pre-existing policy (i.e. Machine/User only) with no application domain policy. That is, assemblies that have already been loaded into the domain will not have their grants reevaluated under the newly set application domain policy.

Care should be taken by the host to set application domain policy before any assemblies are loaded that it is intended for the application domain policy to apply to. If an assembly is explicitly loaded, or if any code in an assembly is run prior to setting application domain policy then that policy will not apply to its grant.


Policy configuration management

Security policy configuration is managed as a collection of code groups that are defined by membership conditions and contain permission sets and special attributes. All policy viewing and changes are subject to administration access permission, as defined in the Security Permissions specification.

Policy configuration should be managed using the Code Access Security Policy tool (CASPol.EXE). For detailed information about maintaining policy and using CASPol, refer to the Runtime Code Access Security Administration specification. Note that CASPol allows administration of machine and user policies only—app domain policy can only be set by NGWS runtime hosts at runtime.


Representation of security policy

The complete security policy for a local machine consists of a collection of code groups. The XML representation of each code group and of the set is the serialized form of the corresponding object(s).

Policy will be stored in a file in the system directory where the core runtime files reside. For more specific information about the location or structure of this file, refer to the Runtime Code Access Security Administration specification.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Publishing Processes and Application Domains

© 2000 Microsoft Corporation. All rights reserved.


Publishing Overview

This document describes a set of interfaces that can be used by an application to enumerate the processes and application domains on a machine.


Publishing Scenarios

A typical scenario is a developer who wishes to debug some managed code running on a server within an application domain hosted by some hosting environment. The hosting environment may be running more than one application domain within a process. The developer would like to use a graphical user interface or some other means to list all the processes running on the server and pick the specific process that he or she is interested in. The listing should list all the application domains within processes that are running managed code. The developer can then identify the specific application domain that he or she is interested in and attach a debugger to that application domain.


Publishing Interfaces

The Publishing Service provides the following 6 interfaces to enumerate processes and application domains on a machine: ICorPublish, ICorPublishProcess, ICorPublishAppDomain, ICorPublishEnum, ICorPublishProcessEnum, and ICorPublishAppDomainEnum. ICorPublish is used to get an enumerator, ICorPublishProcessEnum, that can be used to get the ICorPublishProcess objects that describe the processes on a machine. The methods of the ICorPublishProcess interface can be used to check if the process is running managed code, get the process ID, get the process name, and get an enumerator, ICorPublishAppDomainEnum. ICorPublishAppDomainEnum can be used to get the ICorPublishAppDomain objects that describe the application domains in the process. The methods of ICorPublishAppDomain interface can be used to get the application domain ID and to get the friendly name of the application domain. ICorPublishEnum is an abstract enumerator.

The following sections describe the interfaces in detail. The interfaces are described in alphabetical order, and are found in CorPub.idl.


ICorPublish

This is the top-level interface for publishing processes on a machine.

IDL Declaration.

interface ICorPublish : IUnknown
{
  HRESULT EnumProcesses([in] COR_PUB_ENUMPROCESS type, [out] ICorPublishProcessEnum **ppIEnum);
   HRESULT GetProcess([in] unsigned pid, [out] ICorPublishProcess **ppProcess);
};

EnumProcesses

HRESULT EnumProcesses([in] COR_PUB_ENUMPROCESS type, [out] ICorPublishProcessEnum **ppIEnum);

Retrieves a list of processes on a given machine. The list of processes may or may not be managed depending on the filter flags specified. The list is based on a snapshot of the processes running when the enumeration method is called. If new processes start or existing ones stop while enumerating, the list will remain the same.

Parameter | Description
type | The filter to be used in returning the processes.
ppIEnum | Pointer to a pointer to an enumerator for the requested processes.

GetProcess

HRESULT GetProcess([in] unsigned pid, [out] ICorPublishProcess **ppProcess);

Gets an ICorPublishProcess object for the process with the given process ID.

Parameter | Description
pid | The ID of a process.
ppProcess | Pointer to a pointer to an ICorPublishProcess object for the requested process.


ICorPublishAppDomain

This interface describes an application domain on a machine.

IDL Declaration.

interface ICorPublishAppDomain : IUnknown
{
    HRESULT GetID( [out]   ULONG32 *puId);
    HRESULT GetName([in] SIZE_T cchName, [out] SIZE_T *pcchName, [out, size_is(cchName), length_is(*pcchName)] WCHAR szName[]);
};

GetID

HRESULT GetID( [out]   ULONG32 *puId);

Gets the identification number of this application domain. Note that this number is unique to this AppDomain, but only within the given process.

Parameter | Description
guid | Pointer to the ULONG32 that represents the identification number of the application domain.

GetName

HRESULT GetName([in] SIZE_T cchName, [out] SIZE_T *pcchName, [out, size_is(cchName), length_is(*pcchName)] WCHAR szName[]);

Gets the name of the application domain.

Parameter | Description
cchName | The size of the buffer passed to the method.
pcchName | The actual length of the name. This may be more than the size of the buffer in which case the method should be called again with the correct buffer size.
szName | Pointer to the buffer to receive the characters in the name.


ICorPublishEnum

An abstract enumerator.

IDL Declaration.

interface ICorPublishEnum : IUnknown
{
   HRESULT Clone([out] ICorPublishEnum **ppEnum);
   HRESULT GetCount([out] ULONG *pcelt);
   HRESULT Reset();
   HRESULT Skip([in] ULONG celt);
};

Clone

HRESULT Clone([out] ICorPublishEnum **ppEnum);

Copies a pointer to the current position in the list to another enumerator object.

Parameter | Description
ppEnum | Pointer to pointer to the target enumerator object.

GetCount

HRESULT GetCount([out] ULONG *pcelt);

Gets the number of elements pointed to by this enumerator object.

Parameter | Description
pcelt | Pointer to the number of elements pointed to by the enumerator object.

Reset

HRESULT Reset();

Sets or resets the position of the enumerator to the beginning of the list.

Skip

HRESULT Skip([in] ULONG celt);

Moves the position of the enumeration forward. The number of objects to be skipped is based on a parameter passed to the method.

Parameter | Description
celt | The number of elements to be skipped.


ICorPublishProcess

ICorPublishProcess describes a process on a machine.

IDL Declaration.

interface ICorPublishEnum : IUnknown
{
   HRESULT EnumAppDomains([out] ICorPublishAppDomainEnum **ppEnum);
   HRESULT GetDisplayName([in] SIZE_T cchName, [out] SIZE_T *pcchName, [out, size_is(cchName), length_is(*pcchName)] WCHAR szName[]);
   HRESULT GetProcessID([out] unsigned *pid);
   HRESULT IsManaged([out] BOOL *pbManaged);
};

EnumAppDomains

HRESULT EnumAppDomains([out] ICorPublishAppDomainEnum **ppEnum);

Enumerate the list of know application domains in this process.

Parameter | Description
ppEnum | Pointer to pointer to the enumerator for application domains in this process.

GetDisplayName

HRESULT GetDisplayName([in] SIZE_T cchName, [out] SIZE_T *pcchName, [out, size_is(cchName), length_is(*pcchName)] WCHAR szName[]);

Gets the display name for a process.

Parameter | Description
cchName | The size of the buffer passed to the method.
pcchName | The actual length of the display name. This may be more than the size of the buffer in which case the method should be called again with the correct buffer size.
szName | Pointer to the buffer to receive the characters in the display name.

GetProcessID

HRESULT GetProcessID([out] unsigned *pid);

Gets the operating system identification number of this process.

Parameter | Description
pid | Pointer to the OS process ID of this process.

IsManaged

HRESULT IsManaged([out] BOOL *pbManaged);

Checks whether or not this process is running managed code. Note that this method can return false at one point in time, but true later later on, if managed code is loaded into the process after the enumerator was built.

Parameter | Description
pbManaged | Pointer to a boolean that is true if this process is running managed code, false otherwise.


ICorPublishProcessEnum

Interface for enumerating a list of processes based on the filter criteria specified when the enumerator object was created.

IDL Declaration.

interface ICorPublishProcessEnum : IUnknown
{
   HRESULT Next([in] ULONG celt, [out, size_is(celt), length_is(*pceltFetched)], ICorPublishProcess *objects[], [out] ULONG *pceltFetched);
};

Next

HRESULT Next([in] ULONG celt, [out, size_is(celt), length_is(*pceltFetched)], ICorPublishProcess *objects[], [out] ULONG *pceltFetched);

Gets the requested number of processes that are next in the process list represented by this enumerator.

Parameter | Description
celt | The number of processes requested.
objects | Array of pointers to process objects to receive the returned process objects.
pceltFetched | The actual number of processes fetched.


ICorPublishAppDomainEnum

Interface for enumerating a list of application domains in a process.

IDL Declaration.

interface ICorPublishAppDomainEnum : IUnknown
{
   HRESULT Next([in] ULONG celt, [out, size_is(celt), length_is(*pceltFetched)], ICorPublishAppDomain *objects[], [out] ULONG *pceltFetched);
};

Next

HRESULT Next([in] ULONG celt, [out, size_is(celt), length_is(*pceltFetched)], ICorPublishProcess *objects[], [out] ULONG *pceltFetched);

Gets the requested number of application domains that are next in the application domain list represented by this enumerator.

Parameter | Description
celt | The number of application domains requested.
objects | Array of pointers to application objects to receive the returned application domain objects.
pceltFetched | The actual number of application domains fetched.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Class Definitions

© 2000 Microsoft Corporation. All rights reserved.


CorpubPublish

This class implements the publishing interfaces.

IDL Declaration.

class CorpubPublish
{
   [default] interface ICorPublish;
   interface ICorPublishProcess;
   interface ICorPublishAppDomain;
   interface ICorPublishProcessEnum;
   interface ICorPublishAppDomainEnum;
};


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Type Definitions

This section defines the type definitions used by the publishing interfaces.


COR_PUB_ENUMPROCESS

This enumeration is used to filter the published processes.

IDL Declaration.

typedef enum
{
   COR_PUB_MANAGEDONLY = 0x00000001
} COR_PUB_ENUMPROCESS;

COR_PUB_MANAGEDONLY

COR_PUB_MANAGEDONLY = 0x00000001;

Specifies that only managed published processes are to be returned in a process enumeration.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Remoting Specification

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Remoting QuickConcepts

© 2000 Microsoft Corporation. All rights reserved.


Applications, Messages and Objects

Applications can be thought of as communicating at two levels. At one level messages are sent between applications.

Figure: Applications and Messages

At another level objects in one application are communicating with objects in other application. The Applications at either end could be NGWS Applications, NGWS applications in Windows 2000 Component Services, a PERL script running on UNIX, a Mainframe Application running on MVS, etc. The important thing to note is that either application can replaced and the other application would continue to function correctly, as long as the messages communicated are consistent with the transport, encoding and pattern the application is able to process.

The Applications could be communicating through a set of one way messages or other patterns such as request-response. The Applications could be communicating in an asynchronous or synchronous pattern.

SOAP

Applications that communicate using SOAP message can achieve this “loosely coupled” interoperability. SOAP is an industry standard protocol for encoding and transporting messages. The messages are encoded using XML and can be transported over a number of transports including but not limited to HTTP, SMTP and Message Queues such as MSMQ.

Figure: Applications, Objects and Messages

NGWS runtime remoting natively supports SOAP in the HTTP Channel. Since NGWS runtime remoting channels are pluggable, additional channels with unique transport and encoding requirements can be written and plugged in without changing application code. This is an extensibility point that corporations and 3rd parties can maximize to integrate heterogeneous applications.


Applications, Assemblies, Local Applications and Remote Applications

An NGWS application consists of one or more assemblies and runs within an Application Domain (also known as an AppDomain). Running an application in an AppDomain provides the ability to isolate one application from another application running in a different AppDomain. There can be multiple AppDomains in an OS process. An application running in an AppDomain is isolated from all other applications in all other AppDomains within the OS process and applications running in AppDomains in processes on the same machine or AppDomains on different machines.

AppDomains are hard boundaries. Applications running in different AppDomains share no information, no global variables, no static fields on classes. NGWS runtime remoting is used to communicate between AppDomains.

Local Application

All communication between objects in a NGWS application is highly optimized for local access to fields, methods and properties.

Figure: Local Application

Remote Application in the same Process

All communication between AppDomains is through messages and proxies.

Figure: Local Application with Remote Application in different AppDomain in the same process

Remote Application in an another process

All communication between AppDomains is through messages and proxies. Messages are encoded by the channel and sent. In the receiving application the message is decoded and dispatched to the application.

Figure: Local Application with Remote Application in different AppDomain in another process on the same machine.

Remote Application on another machine

All communication between AppDomains is through messages and proxies. Messages are encoded by the channel and sent. In the receiving application the message is decoded and dispatched to the application.

Figure: Local Application with Remote Application in different AppDomain in another process on a different machine.

Remote Applications

NGWS runtime remoting provides the capability that assemblies that make up the Application can be configured to be part of the Local Application or part of one of many Remote Applications. If the assembly is configured to be part of the Remote Application (using NGWS runtime remoting Configuration), then when classes in that assembly are instantiated, a “proxy” is returned to the Local Application.

In the NGWS runtime, objects can be instanciated from classes in a number of ways: calling new on some class e.g. “new SomeClass()”, calling Activator.CreateInstance or calling Activator.GetObject (GetObject returns a proxy to a well known WebService object in a Remote Application).

All calls on the proxy in the form of instance method calls, instance field and property access are routed to the remote application in the remote AppDomain.

Using NGWS runtime remoting configuration enables the user or administrators to specify which assemblies run within the local application and which assemblies run in remote applications.


Messages, Channels and SOAP

Applications communicate using messages with NGWS runtime remoting providing the transport and encoding of these messages between AppDomains. NGWS runtime remoting Channels transport messages between Applications with NGWS runtime serialization formatters encoding and decoding the messages. Messages can have a number of patterns, with one way and request response as some of most common patterns.

HTTP Channel

The HTTP Channel by default uses the SOAP protocol to transport XML messages between Applications. SOAP is an industry protocol designed to transport XML messages over a number of protocols. The SOAP specification uses the HTTP protocol as an example to demonstrate protocol bindings. The SOAP specification also specifies how SOAP messages can be bound to other protocols.

NGWS runtime remoting Channels can use pluggable NGWS runtime serialization formatters to encode and decode messages. The HTTP Channel encodes SOAP messages using the NGWS runtime Serialization SOAP Formatter. The HTTP Channel can use any NGWS runtime Serialization Formatter e.g. NGWS runtime Serialization Binary Formatter. By specifying the mime type on the channel, 3rd party formatters that have been written can be plugged in.

Figure: Applications communicating using SOAP through the HTTP Channel

TCP Channel

The TCP Channel by default uses plain sockets to transport Binary messages. The TCP Channel encodes Binary messages using the NGWS runtime Serialization Binary Formatter. Just like the HTTP Channel, the TCP can use any NGWS runtime Serialization Formatter e.g. NGWS runtime Serialization SOAP Formatter. 3rd party formatters can also be used by specifying the mime type on the channel. The TCP Channel can be used for fast communication between servers of a farm, but NOT for Internet.

Channels can be either be specified in the NGWS runtime remoting Configuration or they be loaded with APIs.


WebServices, Well Known Objects, Single and Singleton, Client Activated Objects

A WebService can be thought of an entry point into an Application. The entry point is specified by a URL with an object servicing the messages sent to the URL. Logically it can be thought that objects live at the end of URLs. The objects that make up an Application and service message can configured in variety of ways. These are detailed in the topics below.

SingleCall

One popular way, is for each incoming message to be serviced by a new object. When the message arrives a new object is created, the message is processed by the object, an optional reply returned and the object is discarded. In NGWS runtime remoting this is called a “SingleCall WellKnown Object”. Single call objects can use the ASP+ state service to store application and session state if needed. Single call is advantageous in applications where the multiple servers in a serve farm are providing the same application. The incoming messages can be load balanced across the servers in the farm with high scalability being achieved by adding commodity servers to the farm. The load balancing can be performed by simple techniques such as DNS robin or more sophisticated techniques that involve network load balancing at the IP level (NLBS) or application level load balancing.

Figure: WebService Application with single call objects, each message is dispatched to a new object instance

Singleton

Another popular way, is that all message sent to URL in an application are serviced by the same object instance. In NGWS runtime remoting this is called a “Singleton Well Known Object”. Singletons are useful when resources can be “warmed up” and shared across all messages. Singletons also provide the one way to have a conversation of messages with an application.

Figure: WebService with singleton object, all messages are sent to the same object instance

For both SingleCall and Singleton, when the user instances a new object in their application (locally they receive a proxy), no communication occurs until the user calls on the proxy. When the proxy is called, it generates messages that are sent to the channel, which the channel transports to the remote application.

Client Activated Object

Another popular way is for the object to be activated on request from the user. For example when the user instantiates a new object via “new SomeClass()” or Activator.CreateInstance and before the thread returns to the user code, an Activation Message (IConstructionCall message) is sent to the remote application. When the construction message arrives at the remote application it is processed by a Remoting Activator and a new object is created. A message with an Object Reference (ObjRef ) is returned to the Local Application.

One of the useful functions of Client Activated Objects is that constructor arguments can be passed by the Local Application to the constructor of object in the Remote Application. Another useful function is that Client Activated Objects can hold state between messages.

Figure: WebService Application with Client Activated Object, each “new” in the Local Application results in a new object in the Remote Application.

Passing objects between Applications

Objects fall into roughly 3 categories namely Marshal By Value, Marshal By Reference and Not Remotable.

Objects can be passed between applications in a number of ways. The objects may be passed as parameters to or returned from a method call. Likewise an object may be passed between applications are the result of a field or property access. Marshal By Value objects that are embedded as fields within Marshal By Value objects are also that transported By Value. Marshal By Reference objects embedded as fields within Marshal By Value Objects are transported By Reference. Marshal by Value Objects that are embedded in Marshal By Reference Objects as fields or properties are only transported when the user accesses the field or property.

Well Known Objects and Client Activated objects can be specified in the NGWS runtime remoting Configuration or with APIs.

Marshal By Value Objects

For objects that are Marshal By Value (MBV), a complete copy of the object is made when the object is passed from one application to another.

Figure: Passing Marshal By Value Object

Marshal By Reference Objects

For objects that are Marshal By Reference (MBR), a reference to the object is made when passed from one application to another. When the object reference (ObjRef) arrives in the remote application, it is turned into a “proxy” back to the original object.

Figure: Passing Marshal By Reference Object


Leased based Lifetime

For objects that have object references that are transported outside the application, a lease is created. The lease has a lease time. When the lease reaches zero the lease expires and the object is disconnected from the NGWS runtime Remoting infrastructure. Once all the references to the object within the AppDomain have been freed, the object will be collected when the next GC occurs. The lease controls the lifetime of the object.

When a message is dispatched into the application for the object, the lease will be extended if the default renew on call time for the application is not zero. User code that has the proxy can extend the lease by obtaining the lease and renewing the lease. A lease can also have a list of sponsors. When the lease expires NGWS runtime remoting infrastructure asks the sponsors for a renew time. If the Application that provided the sponsor has been shutdown or communication between the two Application is broken for a time greater or equal to the SponsorShipTimeout, then the next sponsor is tried until all the sponsors in the list are exhausted. If this occurs then the lease expires.

Lease based Lifetime with Sponsors


Hosting Applications in ASP+, EXE, NT Services

Applications can be hosted in a variety of hosts. This includes but not limited to ASP+, Internet Explorer, EXEs, NT Services and Windows 2000 Component Services (aka COM+ Service).

Hosting in ASP+

Applications hosted in ASP+ can receive messages on the HTTPChannel. An ASP+ application is configured in IIS Internet Services Manager and ASP+ configuration files. The NGWS runtime remoting configuration file is loaded automatically when the first Remoting message (e.g. SOAP) arrives in the application via the HTTPChannel. IIS by default listens on port 80 for HTTP.

Application Hosted in ASP+

Hosting in Windows 2000 Component Services

COM+ 1.x Applications and NGWS runtime applications can be hosted in Windows 2000 Component Services. These applications can use services such as Automatic Distributed Transactions, Object Pooling etc.

Applications hosted in Windows 2000 Component Services can receive messages through ASP+. An ASP+ application is created that routes the message to the Windows 2000 Component Services Application via COM Interop. The NGWS runtime remoting configuration file is loaded when the first Remoting message arrives in the ASP+ Application.

Figure: Hosting in Windows 2000 Component Services (aka COM+ Services)

Hosting in a Managed EXE

Applications hosted in an EXE must load the NGWS runtime remoting configuration file using an API. The HTTP TCP Listener can listen on any port for HTTP. Effectively any process that hosts the NGWS runtime, running on the machine can receive and send HTTP messages.

Figure: Hosting in an EXE

Hosting in an NT Service

Applications hosted in an NT Service must load the NGWS runtime remoting configuration file using an API. The HTTP TCP Listener can listen on any port for HTTP. Effectively any NT Service host the NGWS runtime, running on the machine can receive and send HTTP messages.

Hosting in an NT Service

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Remoting Programming Tasks

© 2000 Microsoft Corporation. All rights reserved.


Remote Application Programming Tasks

Building a Remote Application by writing a Well Known WebService object

Writing a well known WebService object is a simple as declaring a class and implementing the methods for the class.

Sample Code for HelloService

using System;
namespace HelloService
{
    // Well Known WebService object
    public class Hello : MarshalByRefObject
    {
        // Return the name prefixed with a string
        public String HelloMethod(String name)
        {
            Console.WriteLine("Hello.HelloMethod : {0}", name);
            return "Hi there " + name;
        }
    }
}

The class must derive from System.MarshalByRefObject or any of its children, this allows NGWS runtime remoting to optimize for Remoting objects. See the Remoting Object Model section for details.

Publishing the Remote Application online in ASP+ and MyHost

The Remote Application is published by hosting it in ASP+ or a host such as the sample MyHost. Once the Remote Application is published it can start processing messages.

Hosting in ASP+

Hosting the Remote Application in ASP+ involves creating a IIS root that points to the application directory and adding a “Remoting.cfg” configuration file to that directory. The Remoting configuration file is automatically loaded when the first message arrives in the application.

Remoting.cfg for the HelloService

Name#HelloService
WellKnownObject#HelloService.Hello#HelloService#HelloService/Hello.soap#SingleCall

Name

The 1st line indicates the application name, The format is:

Name#Name of the Application

The application name is HelloService.

WellKnownObject

The 2nd line indicated the well known object that is published for this application.

The format is:

WellKnownObject#[FullTypeName]#[AssemblyName]#[ObjectURI]#[ObjectMode]
Entity | Description | Example
FulllTypeName | The full type name. | MyNamespace.MyType
AssemblyName | The Assembly Name | MyNamespace
ObjectURI | The Object URI on which to publish the object. | foo.soap
ObjectMode | SingleCall Singleton | SingleCall

See the Remoting Configuration topic for further details

Hosting in MyHost

MyHost is a sample managed console EXE that loads a Remoting Configuration file and is useful for debugging purposes since it can be started from the command line and run under a debugger. Hosting the Remote Application in MyHost involves creating a Remoting configuration file and specifying it as a command line argument to MyHost.

HelloService.cfg for the HelloService

Name#HelloService
WellKnownObject#HelloService.Hello#HelloService#HelloService/Hello.soap#SingleCall

An example of starting MyHost and specifying the HelloService.cfg

MyHost -cfg HelloService.cfg

Sending messages to the Remote Application using new or Activator.GetObject

A proxy is used to communicate with the WK WebService in the Remote Application. In simple terms a Proxy is a local object, which is an image of a remote object. Calls on the proxy result in messages that are sent to the Remote Application. The proxy can be obtained in a number of ways that are shown below.

Using Configuration and new

A proxy can be obtained by calling “new” on some class. To use “new”, a Remoting Configuration file must be loaded. The file can be loaded by calling RemotingServices.ConfigureRemoting. Once the Remoting configuration is loaded, future objects created for the class Hello will be created at the well known URL specified in the configuration file.

Sample Code using new and Remoting Configuration

using System;
using System.Runtime.Remoting;
using HelloService;

public class SimpleHello
{
    public static void Main(String[] args)
    {
        String name = "Bill";
        RemotingServices.ConfigureRemoting("MyHello.cfg");
        Hello hello = new Hello();
        String result = hello.HelloMethod(name);
        Console.WriteLine("Hello.HelloMethod returned: " + result);
    }
}

Contents of MyHello.cfg

Name#MyHello
Assembly#HelloService#HelloService#Hello=HTTP://localhost:80/HelloService/Hello.soap
RemoteApplication#HelloService#HTTP://localhost:80/HelloService
Channel#System.Runtime.Remoting#System.Runtime.Remoting.Channels.HTTP.HTTPChannel

An application hosted in ASP+ can send messages to a Remote Application. The Remoting Configuration is loaded in the global.asax. That said one RemoteApplication hosted in ASP+ can send messages to another Remote Application hosted in ASP+.

Sample ASP+ global.asax

<%@ Import Namespace="System.Runtime.Remoting" %>

<script runat="server" language="vb">
  Sub Application_OnStart()
    RemotingServices.ConfigureRemoting
                      HttpContext.Current.Server.MapPath("MyHello.cfg")
  End Sub
</script>

Sample ASP+ Page using new and Remoting Configuration

<%@ Page Language="C#" %>
<%@ Assembly Name="HelloService" %>
<%@ Import Namespace="HelloService" %>

<html>
<head>
    <script runat="server">
        String name;
        String reply;

        void name_change(Object Source, EventArgs E)
        {
            // new the class
            Hello hello = new Hello();

            // Call method with Name from INPUT
            reply = hello.HelloMethod(Name.Value);
        }
    </script>
</head>

<body style="font: 10pt verdana">
<h3> MyHello - Hello.HelloMethod </h3>

    <form runat="server">
        <font face="Verdana">
          <b>Please Enter you name: </b>
          <INPUT id="Name" VALUE="" TYPE="TEXT" runat="server" OnServerChange = "name_change"><p>
        </font>
    </form>

<h3> Reply : <%=reply%> </h3>

</body>
</html>

Using Activator.GetObject

A proxy can be obtained by calling Activator.GetObject and passing a URL to the well known WebServices object. A Remoting Configuration file can optional be used to load the HTTP Channel. Two code samples are provided, one that uses configuration and one that loads the HTTPChannel using ChannelServices.

Sample Code using Activator.GetObject and Remoting Configuration

using System;
using System.Runtime.Remoting;
using HelloService;

public class SimpleHello
{
   public static void Main(String[] args)
   {
      String name = "Bill";
      RemotingServices.ConfigureRemoting("MyHello.cfg");
      Hello hello = (Hello)Activator.GetObject(
                      typeof(Hello),
                      "http://localhost:80/HelloService/Hello.soap");
      String result = hello.HelloMethod(name);
      Console.WriteLine("Hello.HelloMethod returned: " + result);
   }
}

Sample Code using Activator.GetObject and ChannelServices

using System;
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels.HTTP;
using HelloService;

public class SimpleHello
{
    public static void Main(String[] args)
    {
        String name = "Bill";
        ChannelServices.RegisterChannel(new HTTPChannel());
        Hello hello = (Hello)Activator.GetObject(
                      typeof(Hello),
                      "http://localhost:80/HelloService/Hello.soap");
        String result = hello.HelloMethod(name);
        Console.WriteLine("Hello.HelloMethod returned: " + result);
    }
}


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Activation Programming Tasks

Activating an object in a Remote Application

To be supplied


Channel Programming Tasks

Registering a Channel

public class SomeClass
{
    HTTPChannel httpChannel;
    public void SomeMethod()
    {
        httpChannel = new HTTPChannel();
        ChannelServices.RegisterChannel(httpChannel);
    }
}

Unregistering a Channel

public class SomeClass
{
    HTTPChannel httpChannel;
    //. . .
    public void SomeOtherMethod()
    {
        ChannelServices.UnregisterChannel(httpChannel);
    }
}

Lookup all the registered Channels

public class SomeClass
{
    //. . .
    public void LookupAllChannels()
    {
      IChannel[] channels = ChannelServices.RegisteredChannels;
      for(int i=0;i<channels.Length;i++)
      {
          Console.WriteLine("ChannelName: " +
               channels[i].ChannelName);
      }
    }
    //. . .
}


Proxy Programming Tasks

Determining if an object is actually a proxy

public class SomeClass
{
    //. . .
    public void ProxyCheck(Object obj)

    {
      if (RemotingServices.IsTransparentProxy(obj) == true)
        Console.WriteLine("Is a proxy");
     else
        Console.WriteLine("Is not a proxy ");
    }
    //. . .
}

Obtaining the RealProxy from a Proxy

public class SomeClass
{
    //. . .
    public void ObtainRealProxy (Object obj)
    {
        RealProxy realProxy = RemotingServices.GetRealProxy(
                         obj);
    //. . .
}


Message Programming Tasks

Extracting Properties from an IMessage

public class SomeClass
{
    //. . .
    public void ExtractMessageProperties(IMessage msg)
    {
      if (msg is IMethodCallMessage)
        Console.WriteLine("IMethodCallMessage");

      if (msg is IMethodReturnMessage)
        Console.WriteLine("IMethodReturnMessage");

      if (msg == null)
        Console.WriteLine("***** IMessage is null");

      IDictionary d = msg.Properties;
      if (d == null)
        Console.WriteLine("***** Properties are null");

      if (d.Count == 0)
      {
          Console.WriteLine("Zero Properties");
          return;
      }

      IDictionaryEnumerator e = (IDictionaryEnumerator)
                         d.GetEnumerator();

      while (e.MoveNext())
      {
        Object key = e.Key;
        String keyName = key.ToString();
        Object value = e.Value;

        Console.WriteLine(keyName + ":" + e.Value);
      }
    }
    //. . .
}

Marshalling an object

To be supplied


Context Programming Tasks

Obtaining the current Context

The property Thread.CurrentContext is used to obtain the current context.

class Thread
{
    //. . .
    public static Context CurrentContext { get {. . .} }
    //. . .
}

Context Local Data Store Allocated Slots

A data store slot is allocated and returned. The data store slot is used to access the data slot. The data store slot is collected by the GC when all references to it are freed.

Foo foo = new Foo();

// Allocate data slot on all contexts
LocalDataStoreSlot dataslot = Context.AllocateDataSlot();

// Store foo in the Context Local Store
Context.SetData(dataslot, oFoo);

// Retrieve foo from the Context Local Store
Foo foo2 = Context.GetData(dataslot);

Context Local Data Store Named Slots

A slot is allocated using a name and the slot number is returned. The name or the number is used to access the data slot. The slot MUST be freed using the name.

Foo foo = new Foo();

// Allocate data slot on all contexts
LocalDataStoreSlot dataslot = Context.AllocateNamedDataSlot(“Foo”);

// Store foo in the Context Local Store
Context.SetData(dataslot, oFoo);

// Retrieve foo from the Context Local Store
Foo foo2 = Context.GetData(dataslot);

// Free data slot on all contexts
Context.FreeNamedDataSlot (“Foo”);

Marking fields as Context Relative Static

class Foo
{
   int a;
   static int b;
   [ContextStatic()]
   static int c;

   int d=5;
   static int e=6;
   [ContextStatic]
   static int f=7;

   Foo()
   {
      a = 1;
      b = 2;
      c = 3;
   }

   void Method1()
   {
      Console.WriteLine(“{0}”, a);
      Console.WriteLine(“{0}”, b);
      Console.WriteLine(“{0}”, c);
      Console.WriteLine(“{0}”, d);
      Console.WriteLine(“{0}”, e );
      Console.WriteLine(“{0}”, f );
   }
}

Instance member field

a and d are instance members.

a. Instance member field. Uninitialized.

d. Instance member field. Initialized.

Class static member field

b. class static member field. Uninitialized.

e. class static member field. Initialized.

The static is unique for the AppDomain.

Context relative static member field

c. context relative static member field. Uninitialized.

f. context relative static member field. Initialized.

The static is unique for the context. The member field is not shared across contexts.

Zero Initialization

Allow only null initialization of context local statics.

Writing your own context attribute

Example MyContextAttribute

Shown how new context attributes such MyContextAttribute would look like. Note that MyContextAttribute derives from ContextAttribute. The context attribute implements the IContributeServerContextSink interface.

public class MyContextAttribute : ContextAttribute,
   IContributeServerContextSink
{
   . . .
   public IMessageSink GetServerContextSink(IMessageSink nextSink)
   {
      return new MyContextServerSink(nextSink);
   }
   . . .
};

Example Foo attributed with MyContextAttribute

Shown how a Context-bound class Foo can be attributed with the MyContextAttribute. Note that Foo derives from ContextBoundObject.

[MyContextAttribute()]
public class Foo : ContextBoundObject
{
   . . .
};

Example MySpecializedContextAttribute

Shown how context attributes such MySpecializedContextAttribute would look like. It overrides IContextAttribute.IsContextOK and IContextAttribute.GetPropertyForNewContext methods.

public class MySpecializedContextAttribute : ContextAttribute
{
   . . .
   public boolean IsContextOK(Context ctx, out Object cookie)
   {
      . . .
   }
   public IContextProperty GetPropertyForNewContext(Object cookie);
   {
      . . .
      // return an instance of MySpecializedContextProperty
      . . .
   }
   . . .
};

Example MySpecializedContextProperty

Shown how context properties such MySpecializedContextProperty would look like. The context property implements the IContributeServerContextSink interface.

public class MySpecializedContextProperty :
 IContextProperty, IContributeServerContextSink
{
   . . .
   public String Name  {get {return "MySpecializedContextProperty";} }
   public boolean IsNewContextOK(Context* newCtx)
   {
      . . .
   }

   public IMessageSink GetServerContextSink(IMessageSink* nextSink)
   {
      . . .
   }
   . . .
};

IContribute*Sink interfaces | Description
IContributeServerContextSink | Calls into Context (e.g. Synch or Tx)
IContributeClientContextSink | Calls out of Context (e.g. Sync with reentrancy)
IContributeObjectSink | Specific object (e.g. JIT Activation)
IContributeEnvoySink | Project the server context rules as an envoy into the client context

Example Baz attribute with MySpecializedContextAttribute

Shown how a Context-bound class Baz can be decorated with the MySpecializedContextAttribute.

[MySpecializedContextAttribute()]
public class Baz : ContextBoundObject
{
   . . .
};


Remoting MetaData Programming Tasks

Obtaining an XML Schema for a WebService URL

To be supplied

Converting the XML Schema into MetaData

To be supplied


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

DynamicProperty Programming Tasks

Adding a new DynamicProperty

To be supplied


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Tracking Service Programming Tasks

Adding a new TrackingService handler

To be supplied


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Object Reference Programming Tasks

Extracting information from an ObjRef

To be supplied


Call Context Programming Tasks

Sample code

Shown how code can get and set the call context.

void SomeMethod()
{
  MyObject myObject  = new MyObject();
   CallContext.SetData(“MyItem”) = myObject;
}

void SomeOtherMethod()
{
   MyObject myObject = CallContext.GetData(“MyItem”);
}


One Way Programming Tasks

Marking a method as one way

public class Foo
{
  [OneWayAttribute()]
  public void Method1(int i, float f, String s, Object o);
}

Call a one way method

SomeObject someObject = new SomeObject();
Foo foo = new Foo();
foo.Method(1, 2.3, “Hi”, someObject);


Async Programming Tasks

Calling an Async method

using System;
using System.Runtime.Remoting;
using System.Threading;

using HelloService;

public class SimpleHello
{
   public static ManualResetEvent evt;

   public static void Main(String[] args)
   {
      evt = new ManualResetEvent(false);
      SimpleHello simpleHello = new SimpleHello();
      simpleHello.InstanceMain(args);
      evt.WaitOne();
   }

   public void HelloCallBack(IAsyncResult ar)
   {
      HelloDelegate helloDelegate  = (HelloDelegate)ar.AsyncObject;

      String result = helloDelegate.EndInvoke(ar);
      Console.WriteLine("Hello.HelloMethod returned: " + result);
      evt.Set();
   }

   public delegate String HelloDelegate(String name);

   public void InstanceMain(String[] args)
   {
      String name = "Bill";
      RemotingServices.ConfigureRemoting("MyHello.cfg");
      //RemotingServices.ConfigureRemoting("MyHello.DirectHTTP.cfg");

      Hello hello = new Hello();

      AsyncCallback asyncCallback = new AsyncCallback(this.HelloCallBack);
      HelloDelegate helloDelegate = new HelloDelegate(hello.HelloMethod);

      IAsyncResult ar = helloDelegate.BeginInvoke(name, asyncCallback, null);
   }
}


Remoting Security Programming Tasks

Passing credential through the Channel Message Sink

MyFoo foo = Activator.GetObject(“http://. . .”);
dictionary = ChannelServices.GetChannelSinkProperties(foo);
dictionary["UserName"] = “joe”;
dictionary["Password"] = “secret”;
dictionary["Domain"] = “Sales”;

Basic Authentication

To be supplied

Digest Authentication

To be supplied

Cookie Authentication

To be supplied

Integrated Windows Authentication

To be supplied

Secure Communication

Activator.GetObject Example

MyFoo foo = Activator.GetObject(“https://. . .”);

Configuration Example

Assembly#StockNet#Trader#StockQuote=https://localhost/TraderSecure/StockQuote.soap

ASP+ HttpRequest.IsSecureConnection in the HttpContext returns a boolean that indicates whether the Http connection is secure i.e. HTTPS.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Remoting Object Model

© 2000 Microsoft Corporation. All rights reserved.


Remoting Object Model Concepts

This section provides guidelines for placing objects in the various class buckets:

  • System.Object
  • System.Object (marked as Serializable custom attribute, and optionally I ISerializable
  • System.MarshalByRefObject
  • System.ContextBoundObject

Remoting Object Categories

Runtime object types are categorized into 3 categories.

Runtime category | Must derive from
Not Marshaled (Agile) | System.Object
Unbound (Agile, Marshal By Value) | System.Object Marked with [serializable]. May have Iserializable & related Constructor
AppDomain-bound (Agile, Marshal By Reference) | System.MarshalByRefObject
Context-bound (Contextful, Marshal by Reference) | System.ContextBoundObject

Agile objects

Agile objects are accessed directly from any context and are called directly from the caller context with an AppDomain.

To invoke a method on an agile object from anywhere within an AppDomain, a direct reference would suffice. The Base Class Libraries (System.*) consists mostly of agile classes with a few exceptions.

Unbound (Agile, Marshal By Value) and AppDomain-bound (Agile, Marshal By Reference) are collectively referred to as agile.

“Not Marshaled” type Category

class Foo
{
  int Bar;

  int Foo;

  //. . .
}

class Baz : Foo
{
  //. . .
}

Since Foo is “Not Marshaled” then Baz is also “Not Marshaled”.

Key Points:

  • Default for all objects that do not derive from System.MarshalByRefObject or System.ContextBoundObject and that do not have the [serializable] custom attribute.
  • Derive from System.Object or any of its children.
  • “Not Marshaled” objects are appropriate when it makes the object should not leave the AppDomain. The object was designed for local use only.
  • Fields, properties and methods are always accessed directly with an AppDomain i.e. from any context in the AppDomain.
  • They are always accessed directly because all references to them are to an actual instance.
  • Instanced cannot be sent across an AppDomain boundary since they are not serializable
  • Highly optimized by EE and JIT.
  • No extra costs.

Unbound (Object By Value) objects type category

[serializable]
class Foo
{
  int Bar;
  int Foo;

  //. . .
}

class Baz : Foo
{
  //. . .
}

Since Foo is Unbound then Baz is also Unbound.

Key Points:

  • Similar to the “Not Marshaled” category however the object can be passed across AppDomain boundaries by value i.e. a copy is made.
  • Derive from System.Object or any of its children.
  • Unbound objects are appropriate when it makes sense for performance or processing reasons to move the complete state of the object with the execution to the target AppDomain. In many scenarios this reduces boundary crossing (network, process, appdomain) round-trips. For example self contained objects.
  • Fields, properties and methods are always accessed directly with an AppDomain i.e. from any context in the AppDomain.
  • They are always accessed directly because all references to them are to an actual instance.
  • Instances are not confined to any boundary.
  • No distributed identity.
  • No proxy is ever created i.e. a copy is made when necessary. When passed in a remote call across an AppDomain boundary, a copy is made from the source App Domain to the target App Domain (“Marshal By Value”). The following scenarios are automatically taken care of by the NGWS runtime:
    • Between AppDomains in the same physical process.
    • Between AppDomains in different physical processes on the same machine.
    • Between AppDomains in different physical processes on different machines.
  • An Unbound object, can expose Iserializable, if it wants to participate in its own serialization when marshaled across AppDomain boundaries.
  • Highly optimized by EE and JIT.
  • No extra costs.

Marshal By Reference (Object By Reference) object type category

Base class for Remoting objects that need to be marshal by reference. This includes WellKnown SingleCall and WellKnown Singleton WebService objects and Client Activated Objects.

class Zap : MarshalByRefObject
{
  //. . .
}

class Waz : Zap
{
  //. . .
}

Since Zap is AppDomain-Bound then Waz is also AppDomain-Bound.

Key Points:

  • Derive from System.MarshalByRefObject or any of its children (except from context bound objects which derive from System.ContextBoundObject).
  • Have distributed identity.
  • Instances of MarshalByRefObject classes are confined to the App Domain in which they were created, their "Home AppDomain", but are accessible directly from any context within that App Domain. An App Domain is the logical equivalent of a sub-process within the Runtime.
  • MarshalByRefObject's are appropriate when the state of the object should stay in the AppDomain that created it and only references should be marshaled when the object is remoted. For example a File IO object that internally has field which is an OS handle should be made AppDomain-bound, since the OS handle would not be useful or appropriate in another AppDomain in another process or another machine.
  • If a MarshalByRefObject is passed outside of the "Home AppDomain" e.g. as a parameter on a remote method call or as a return value from a remote method call, a reference is passed from the source App Domain to the target App Domain ("Marshal By Reference") i.e. a reference is marshaled into the target App Domain. The following scenarios are automatically taken care of by the NGWS runtime:
    • Between AppDomains in the same physical process.
    • Between AppDomains in different physical processes on the same machine.
    • Between AppDomains in different physical processes on different machines.
  • Subsequent calls on the reference in the target AppDomain are marshaled back in to the "Home AppDomain" in which the object was created.
  • MarshalByRefObject's that are remoted out of the AppDomain and subsequently called remotely are called from the default context for the AppDomain.
  • Since a client may hold a proxy, all operations on the class are appropriately indirected such that the infrastructure can intercept and forward them. This is performed automatically by the EE and JIT working together.
  • Fields of App Domain bound objects must be accessed through accessors. This is performed automatically by the EE and JIT working together.
  • All member functions of an App Domain bound objects must be called through indirection. This is performed automatically by the EE and JIT working together.
  • No inlining of the methods, field and properties of App Domain bound classes.
  • Like Context-bound objects, there are costs associated with App Domain bound objects, but those costs are smaller.

MarshalByRefObject as the base class for remote objects

Depending on which JIT is being used, certain optimizations may be used for classes which are not Marshaled by Reference e.g. direct field access by JIT. Inlining of methods is another optimization. One of rules of using MarshalByRefObject across AppDomains is that all the parents need to have the ancestor of MarshalByRefObject, this to enable the EE and JIT to provide consistent access.

The incorrect way The correct way
class Foo

{
  int i;
  float f;
  void Method1();
}

[MarshalByRefObject()]
class Baz : Foo
{
  String s;
  void Method2();
}

void SomeCode(Foo foo)
{
  int myInt = foo.i;
  foo.f = 2.4;
  foo.Method1();
}

class Foo : MarshalByRefObject

{
  int i;
  float f;
  void Method1();
}

class Baz : Foo
{
  String s;
  void Method2();
}

void SomeCode(Foo foo)
{
  int myInt = foo.i;
  foo.f = 2.4;
  foo.Method1();
}

Performance Implications for MarshalByRefObject

Since a client may hold a proxy, all operations on the class are appropriately indirected such that the infrastructure can intercept and forward them. This is performed automatically by the EE and JIT working together.

Fields of MarshalByRefObject must be accessed through accessors. This is performed automatically by the EE and JIT working together.

All member functions of MarshalByRefObjects must be called through indirection. This is performed automatically by the EE and JIT working together.

Performance implications of marshalling between AppDomains

Since AppDomains are completely isolated there is no sharing of state, the parameters on the stack must be converted into a binary format (pretty fast) and carried to the other appdomain and turned back into a stack frame. If MarshalByRefObjects are passed in as parameters, these will turn into a proxy in the other appdomain, when the call arrives.

Context Bound Object type category

class Tip : ContextBoundObject
{
  //. . .
}

class Pot : Tip
{
  //. . .
}

Since Tip is Context-Bound then Pot is also Context-Bound.

Key Points:

  • More specialized that AppDomain bound objects.
  • Derive from System.ContextBoundObject or any of its children.
  • Have distributed identity.
  • Instances of Context-bound classes are confined to the Context in which they were created, their “Home Context”. They are accessed directly from their “Home Context”, but are accessible by proxy from any context. A Context is a boundary that can enforce policies on classes such as: Synchronization, Thread Affinity etc.
  • Context-bound objects are appropriate when the state of the object should stay in the context in which it was created.
  • If an Context-bound object is passed outside of the “Home Context” e.g. as a parameter on a cross context method call or as a return value from a cross context method call, a reference is passed from the source context to the target context (“Marshal By Reference”) i.e. a reference is marshaled into the target context. The following scenarios are automatically taken care of by the NGWS runtime:
    • Between Contexts in the same AppDomain.
    • Between Contexts in different AppDomains in the same physical process.
    • Between Contexts in different AppDomains in different physical processes on the same machine.
    • Between Contexts in different AppDomains in different physical processes on a different machines.
  • Subsequent calls on the reference in the target Context are marshaled back in to the “Home Context” in which the object was created.
  • Since a client may hold a proxy, all operations on the class are appropriately indirected such that the infrastructure can intercept and forward them.
  • Fields of Context-bound objects must be accessed through accessors.
  • All member functions of an Context-bound objects must be called through indirection.
  • No inlining of the methods, field and properties of Context-bound classes.
  • Context-bound objects have a cost. Even if there is only one context in an entire program, which all instances live in there are still costs due to the possibility that a new context could appear at some point.


Remoting Object Model Architecture

Runtime object types are categorized into 4 categories.

Objects and Type hierarchy

Logical and actual class model

The logical hierarchy is:

Figure 1: Logical class hierarchy.

In summary, Objects may be agile (Unbound or App-Domain bound) or Context-bound. Agile objects can be accessed through a direct reference from anywhere in the App-domain whereas access to a Context-bound object one would via a proxy.

Figure 1 shows the inheritance type hierarchy for all runtime classes. With respect to contexts, only Context-bound classes and agile classes are interesting. Objects belonging to context-bound class are guaranteed to be called only inside the context in which they were born. On the other hand, objects belonging to agile classes are guaranteed to be called in the context of their caller.

The logical class hierarchy is unaffected, with Object as the root. But the actual class hierarchy doesn't have Object as the root. Object is the root of all Unbound classes. MarshalByRefObject is the root of all AppDomain-bound classes. ContextBoundObject is the root of all Context-bound classes.

Context-bound classes are in a separate section of the hierarchy.

Once a class is marked as Unbound, AppDomain-bound or Context-bound, all its subclasses are similarly constrained. This falls out naturally from considering Agility to be a location in the class hierarchy.

The class hierarchy yields the following benefit:

  1. Compiler need not invent new keywords for defining Context-bound and Appdomain-bound classes. A class is context-bound if it derives from ContextBoundObject and Appdomain-bound if it derives from MarshalByRefObject.

Objects are never shared across app-domains. Another way of picture the hierarchy is to think that context-bound objects are proxied outside of their context, AppDomain-bound objects are proxied outside of their AppDomain, and unbound objects are copied by value outside of their AppDomain.

Hierarchy and System.Object

The System.Object class is ambiguous with respect to agility as it is the root of both context-bound and agile classes. A similar argument holds for interfaces as they can be implemented both by context-bound and agile classes. The implementation hierarchy for runtime objects is shown below, in Figure 2

The Virtual Root Object is conceptual. Treating System.Object class as agile, optimizes for a world where most of the classes are agile. For example, the Class Libraries (System.*) contains mostly agile classes. Cross-context transparent proxies are not agile.

Instance Fields and Accessors

There are three cases:

  • If the class is unbound (agile and remotes by value) then the JIT always performs direct memory access for fields. No proxies are created within the AppDomain, since the instance is unbound. No proxies are created between AppDomains because the instance marshals by value. This is the combination that [is / will be] used for as much of the ClassLibs as possible. In particular, arrays must be unbound (Agile and marshal by value). Array element access can be thought of as access to fields that are numbered, rather than named. Therefore it can be direct, as it is today.
  • Otherwise if the JIT can determine that the object is “this” or any of its parents, access is also direct memory. Context-bound classes don’t use direct field access.
  • In any other case the JIT/EE inserts a check at runtime to see whether the object is a proxy. If not a proxy, then field access is via direct memory. Otherwise it is via assessors provided in the proxy.

Statics

Static fields and methods are agile regardless of the class they are on. That means they are never remoted and field access is via direct memory.

Static fields are like instance fields of Agile objects - they don’t imply any context crossings. For example, accesses to a static field of a class that requires (Context-style) synchronization will not be synchronized.

Passing Objects as Parameters

Agile objects may be transmitted and received by value (unbound) or by reference (AppDomain bound). Context-bound (i.e. non agile) objects must be transmitted by reference (by proxy) only.

Non-virtual Instance Methods

Virtual methods don’t require special checks for proxies. The VTable of the proxy automatically handles this case. However non-virtuals need to be deal with differently. Here’s the set of non-virtual instance method cases:

  • Constructors. The outermost call to a constructor is different from all chained calls to sibling or super constructors. Chained calls are all being made on ‘this’ or ‘super’ so they clearly go to the inner entrypoint.
  • Methods introduced as final. Methods that are marked as final in a subclass, but were not final in a superclass, are still virtual methods. The only non-virtual case is when the method is actually introduced as final.
  • Private methods.

Delegates

Delegates are Unbound (Marshaled by Value). Address and Code (thunk or direct pointer) as appropriate.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Remoting Base Infrastructure

© 2000 Microsoft Corporation. All rights reserved.


RemotingServices

RemotingServices Reference

RemotingServices Reference

RemotingServices provides a number of methods to help in using and publishing Remoted objects and proxies.

Application need not distinguish between proxy references and object references. However, service providers dealing with issues such as activation, lifetime management, and transactions need to distinguish between proxy references and object references.

public class RemotingServices
{
  public static MarshalByRefObject Connect(Type classToProxy, String url);
  public static MarshalByRefObject Connect(Type classToProxy,
                           String url,
                           Object data);
  public static void ConfigureRemoting(String fileName);
  public static Object DeactivateObject(MarshalByRefObject target);
  public static bool Disconnect(MarshalByRefObject obj);
  public static Object GetLifetimeService(MarshalByRefObject obj)
  public static RealProxy GetRealProxy(Object proxy);
  public static bool IsObjectOutOfContext(Object tp);
  public static bool IsTransparentProxy(Object proxy);
  public static ObjRef Marshal(MarshalByRefObject obj);
  public static ObjRef Marshal(MarshalByRefObject obj, String uri);
  public static ObjRef Marshal(MarshalByRefObject obj, String URI,
                               Type requestedType);
  public static MethodBase MethodBaseFromMethodCallMessage(
                              IMethodMessage mcm);
  public static void ReactivateObject(
                           IConstructionCallMessage targetPrms,
                           Object cookie);
  public static void RegisterWellKnownType(
            String assemblyName,
            String typeName,
            String URI,
            WellKnownObjectMode mode);

  public static MarshalByRefObject Unmarshal(ObjRef objectRef);
}

Connect

Takes in the Type of the server class, and the URL to the server class and immediately creates a proxy object that points to an endpoint that is served by an Well Known Object. See Remoting Activation for details on how Well Known Objects are created.

Connect

Takes in the Type of the server class, URL, data object to the server class and immediately creates a proxy object that points to an endpoint that is served by an Well Known Object. See Remoting Activation for details on how Well Known Objects are created. The data object is used to communicate information the channel in IChannelSender.CreateMessageSink.

ConfigureRemoting

Read the configuration file and configure Remoting. See Remoting Configuration and Remoting Activation for further details.

DeactivateObject

Release an object without releasing its infrastructure. A cookie for Reactivate is returned.

Disconnect

Used to disconnect an object from the Remoting infrastructire and stop the object from receiving any further Messages through its channel. If passed a proxy will disconnect the proxy from the server object.

GetLifetimeService

This method is used return a lifetime service object that is used to control the lifetime policy to the object. For the default Lifetime service this will be an object of type ILease.

Return Object to control lifetime Service.

GetRealProxy

Takes an object reference to a transparent proxy. Returns the real proxy backing the transparent proxy.

IsObjectOutOfContext

Takes an Object reference. If passed a proxy will return true if the current context is not the same as the context in which the object lives.

IsTransparentProxy

Takes an object reference. Returns a boolean indicating whether the object is a transparent proxy or a real object.

Marshal

Takes an object. Registers the object with the Remoting infrastructure. Return an instance of the ObjRef class, which in turn, can be serialized for transmission between AppDomains, including over the network. The context from the current thread is used, not the context that was active when the object was created. A URI is auto generated by the Remoting Identity infrastructure. See ObjRef for further details.

Marshal

Takes an object and URI. Registered the object with the Remoting infrastructure. Return an instance of the ObjRef class, which in turn, can be serialized for transmission between AppDomains, including over the network. The context from the current thread is used, not the context that was active when the object was created. The specified URI is used by the Remoting Identity infrastructure. See ObjRef for further details.

Marshal

Takes an object, URI and Type. Registers the object with the Remoting infrastructure. Return an instance of the ObjRef class, which in turn, can be serialized for transmission between AppDomains, including over the network. The context from the current thread is used, not the context that was active when the object was created. The specified URI is used by the Remoting Identity infrastructure. The specified Type is used by the Remoting Identity Infrastructure to limit the scope of the exposed type hierachy. For example if Foo derives from Baz which derives from Zip and Marshal is called, then the client can cast the proxy between Zip and Baz but not to Foo. See ObjRef for further details.

MethodBaseFromMethodCallMessage

Returns the method base from the IMethodMessage.

ReactivateObject

Reattach the specified object to the Remoting infrastructure. The cookie must be the one returned from Deactivate.

RegisterWellKnownType

Registers a well known type using the passed in parameters. See Remoting Activation and Remoting Configuration for details.

Unmarshal

Takes an ObjRef and creates a proxy object out of it. The ObjRef may have been created by deserializing the bytes transmitted over the network from another AppDomain.

WellKnownObjectMode Reference

Enum for wellknown object modes.

public enum WellKnownObjectMode
{
  Singleton   = 1,
  SingleCall  = 2
}

Singleton

Every message is dispatched to the same object instance

SingleCall

Every message is dispatched to a new object instance.


Proxy

Proxy Concepts

In simple terms a Proxy is a local object, which is an image of a remote object.

A client that uses an object across any kind of a remoting boundary is actually using a Transparent Proxy for the object. The Transparent Proxy provides the illusion that the actual object resides in the client’s space. It achieves this by forwarding calls made on it to the real object using the Remoting infrastructure.

The Transparent Proxy is itself housed by an instance of a managed runtime class of type RealProxy. This object implements a part of the functionality needed to forward the operations from the transparent proxy. Note that the proxy object inherits the associated semantics of managed objects such as garbage collectible, support for fields and methods, and can be extended to form new classes. Thus the proxy has a dual nature … on one side, it needs to act as an object of the same class as the remote object (Transparent Proxy) and on the other it is a managed object itself.

A proxy object can be used without regard to any remoting sub-divisions within an AppDomain.

An object of type Proxy must derive from the RealProxy class which implements common functionality. It also defines the member variable that holds a reference on the transparent proxy.

A Proxy would be created under the following circumstances:

  • Using new to create an object that is not compatible with the current remoting sub-division
  • As an incoming / return / out-going parameter of a remoted method accessing a location that contains a reference to an object from another sub-division.

Proxy Reference

RealProxy Reference

Defines the base class from which proxies should derive.

public abstract class RealProxy
{
   public virtual Object GetTransparentProxy();
   public abstract void Invoke(IMessage msg);
}

GetTransparentProxy. Used for traversing back to the Transparent Proxy from the Real Proxy. Returns the TransparentProxy from a RealProxy.

Invoke. Method to which the transparent proxy delegates, when it gets called. Is used by the transparent proxy to call the real proxy.


Message

Message Concepts

Messages can be thought of as a container for communicating data between co-operating message sinks. These message sinks could be anywhere along the message sink chain: in the client sink chains or in the server sink chains. A message object is passed from message sink to message sink, through the chain and carries set of named properties such as, but not limited to, action identifiers, envoy information, and parameters.

An object that implements the interface IMessage meets the minimum qualification to be considered a message object. Also, there is no restriction that the exact object received by a message sink must be passed on to the next sink, though this will often be the case.

Although it is not a requirement that the objects in the property bag be serializable, it is certainly something that the message sink implementer would want to consider since an properties that need to flow out of the application domain will need to be serializable. Objects should also be agile if they are to be used in cross-context, same app-domain calls.

Message Reference

IMessage Reference

interface IMessage
{
   public IDictionary Properties();
}

GetProperties

Returns a dictionary, which represents a collection of associated keys and values. The message properties are made available for read and write via the dictionary.

IMethodMessage Reference

Defines the method message interface. A method message is specialized for methods.

public interface IMethodMessage : IMessage
{
  String URI                      { get; }
  String MethodName               { get; }
  String TypeName                 { get; }
  Object MethodSignature          { get; }

  int ArgCount                    { get; }
  Object GetArg(int argNum);
  String GetArgName(int index);
  Object[] Args                  { get; }

  bool HasVarArgs                 { get; }
}

Args

Return an array of all the arguments. An Object array containing the arguments. Although this method is redundant since the same functionality may be achieved via GetArgCount/GetArg, there may be performance optimization available to the implementor if s/he understands when all of the arguments will be retrieved. IMessage Dictionary property key “__Args”.

GetArgCount

Return the number of arguments. IMessage Dictionary property key “__Args” via array object.

GetArgName

Return the name of a specific argument. A string representing the name of the argument specified or null if this method is not implemented. IMessage Dictionary property key “__Args” via array object.

GetArg

Return a specific argument as an Object. IMessage Dictionary property key “__Args” via array object.

HasVarArgs

Returns a boolean indicating if the message has varargs.

MethodName

Return the method name. A string which is the name of the method to be called. IMessage Dictionary property key “__MethodName”.

MethodSignature

Return the method signature. An array of byte’s containing a method signature. IMessage Dictionary property key “__MethodSignature”.

TypeName

Return the type name. IMessage Dictionary property key “__TypeName”.

URI

Return the URI string to which the message is issued. The URI (Uniform Resource Identifier) of the object and method parameters. The URI determines to specific object a call is destined for. IMessage Dictionary property key “__URI”.

IMethodCallMessage Reference

Defines the method call message interface. A method call message is a message, which represent a method call on an object at the end of the message sink. Method Call Messages implement the IMethodCallMessage interface.

public interface IMethodCallMessage : IMethodMessage
{
  int InArgCount                  { get; }
  Object GetInArg(int argNum);
  String GetInArgName(int index);
  Object[] InArgs                { get; }
}

InArgCount

Return the number of IN arguments. IMessage Dictionary property key “__InArgs” via array object”.

GetInArg

Return a specific IN argument as an object. IMessage Dictionary property key “__InArgs” via array object.

GetInArgName

Return the name of a specific IN argument. A string representing the name of the IN argument specified or null if this method is not implemented. IMessage Dictionary property key “__InArgs” via array object.

InArgs

Return an array of all the IN arguments. An Object array containing the IN arguments. Although this method is redundant since the same functionality may be achieved via InArgCount/GetInArg, there may be performance optimization available to the implementor if s/he understands when all of the arguments will be retrieved. IMessage Dictionary property key “__InArgs”.

IMethodReturnMessage Reference

Defines the method call return message interface. A method call return message is a message, which represent the response to a method call on an object at the end of the message sink.

public interface IMethodReturnMessage : IMethodMessage
{
  int OutArgCount                 { get; }
  Object GetOutArg(int argNum);
  String GetOutArgName(int index);
  Object[] OutArgs               { get; }
  Exception Exception             { get; }
  Object ReturnValue             { get; }
}

Exception

Return the exception object for the method call. If an exception did not occur on the call, the return value is null. IMessage Dictionary property key “__Fault”.

GetOutArg

Return a specific out argument as an Object. IMessage Dictionary property key “__OutArgs” via array object.

OutArgCount

Return the number of out arguments. IMessage Dictionary property key “__OutArgs” via array object”.

GetOutArgName

Return the name of a specific out argument. A string representing the name of the out argument specified or null if this method is not implemented. IMessage Dictionary property key “__OutArgs” via array object.

OutArgs

Return an array of all the out arguments. An Object array containing the out arguments. Although this method is redundant since the same functionality may be achieved via GetOutArgCount/GetOutArg, there may be performance optimization available to the implementor if they understand when all of the arguments will be retrieved. IMessage Dictionary property key “__OutArgs”.

ReturnValue

Return the return value as a variant for the call. IMessage Dictionary property key “__Return”.

IMessageSink Reference

When a method call is made on the proxy, the remoting infrastructure provides the necessary support for:

  • Passing the arguments to the actual object across the remoting boundaries.
  • Calling the actual object method with the arguments.
  • Returning the results back to the client of the proxy object.

Remote method calls is defined in terms of Messages that goes from the client end to the server end and possibly back again. As it crosses interesting boundaries on the way, it passes through a chain of MessageSink objects. Each sink in the chain receives the IMessage object, performs a specific operation and delegates to the next sink in the chain. The Proxy object contains a reference to the first IMessageSink it needs to use, to start off the chain.

For asynchronous calls, at the time of delegation, each sink provides a reply sink (another IMessageSink object) that will be called by the next sink when the reply is on its way back.

The nature of the operation performed by an IMessageSink when it receives a IMessage object to process may be radically different for different types of sinks. For example, one sink could cause a lock to be taken, another could enforce call security, another could be performing flow call control / reliability services and yet another could be responsible for transporting the call to a different appdomain / process / machine. Two or more message sinks in the chain may communicate and interact with each other in regard to each specific action.

public interface IMessageSink
{
  public IMessage SyncProcessMessage(IMessage msg);
  public IMessageCtrl AsyncProcessMessage(
                  IMessage msg,
                  IMessageSink replySink);
  public IMessageSink NextSink { get; }
}

AsyncProcessMessage

Invoked on the message sink by the remoting infrastructure or by a previous sink for asynchronous messages. Returns an IMessageCtrl interface that provides a way to control asynchronous messages once they have dispatched.

NextSink

Retrieves the next message sink held by this sink.

SyncProcessMessage

Invoked on the message sink by the remoting infrastructure or by a previous sink for synchronous messages. Returns the reply message in response to the request.

IMessageCtrl Reference

The IMessageCtrl interface provides a way to control asynchronous messages once they have dispatched via IMessageSink.AsyncProcessMessage.

public interface IMessageCtrl
{
  public void Cancel (int msToCancel);
};

Cancel

Used to cancel an asynchronous message. The number of milliseconds in which to cancel the message is passed.


Object References

Object Reference Concepts

An ObjRef is a serializable representation of an object used to transfer an object reference across an AppDomain boundary. The act of creating an ObjRef for an object is called marshaling. The ObjRef can be transferred via a channel into another AppDomain (possibly on another process or machine). Once in the other AppDomain, the ObjRef must be parsed to create a Proxy for the Object, generally connected to the real Object. This operation is called unmarshaling. ObjRefs have enough information that describes the Type / class of the object being marshaled (like the class hierarchy), something that will uniquely identify the specific object instance (the URI) and how to reach the remoting-subdivision where the object lives (communication related information).

An important notion to be aware of about marshaling are the semantics of Marshal-by-Value versus Marshal-by-Reference. When a reference to an Object is converted to an ObjRef such that the Proxy created after unmarshaling talks to the original object for most method calls via a proxy, this is called Marshal-by-Reference. However, there are objects whose state does not change in any interesting way once initialized. For such objects, it makes sense to send over the entire state of the Object in the ObjRef and then create a local object that is a clone of the remote one within the remoting-boundary of the client. This is called Marshal-by-Value.

Object Reference

ObjRef Reference

Defines the marshaled object reference class and related classes

public class ObjRef : public IObjectReference
{
public:
   //Constructors
   public ObjRef ();
   public IChannelInfo ChannelInfo {get; set;};
   public IEnvoyInfo EnvoyInfo {get; set;};
   public IRemotingTypeInfo TypeInfo {get; set;};
   public virtual Object GetRealObject (StreamingContext context);
   public String URI {get;set};
};

ChannelInfo

This read-write property returns the IChannelInfo interface for the ObjRef. Channels are described in further detail in the channels section.

EnvoyInfo

This read-write property returns the IEnvoyInfo interface for the ObjRef. Envoys are described in further detail in the contexts section.

TypeInfo

This read-write property returns the IRemotingTypeInfo for the objRef. TypeInfo is utilized to refine the proxy to suit the client view of the server object.

URI

This read-write property uniquely represents the identity of the specific object instance.

IChannelInfo Reference

Interface for providing channel information. Users can use this interface to provide custom channel information which is carried along with the ObjRef.

public interface IChannelInfo
{
    Object[] ChannelData {get; set;}
    String[] ChannelNames {get; set;}
}

ChannelData

This read-write property gets/sets the channel data for each channel.

ChannelNames

This read-write property gets/sets the names of the channels.

IEnvoyInfo Reference

Provides envoy information. Users can use this interface to provide custom envoy information which is carried along with the ObjRef.

public interface IEnvoyInfo
{
    IMessageSink EnvoySinks {get; set;};
}

EnvoySinks

This read-write property gets/sets the envoy sinks. The list of envoys which were contributed by the ServerContext and ServerObject chains when the Object was marshaled.

IRemotingTypeInfo Reference

Interface for providing type information. Users can use this interface to provide custom type information which is carried along with the ObjRef.

public interface IRemotingTypeInfo
{
   String TypeName { get; set; }
   bool CanCastTo(Type fromType, Object o);
}

TypeName

This read-write property returns the fully qualified type name.

CanCastTo

Check whether the given type can be cast to the requested type.


Dynamic Properties

DynamicProperty Concepts

Dynamic Properties and Dynamic Sinks provide an extensibility mechanism for user to monitor calls to and from objects.

Context.RegisterDynamicProperty allows users to register a property implementing the IDynamicProperty and IContributeDynamicSink interfaces with the remoting service. Based on the obj and ctx parameters, the property is asked to contribute a sink that is placed at some location in the path of remoting calls.

If multiple properties are registered, their sinks may be called in an arbitrary order that may change between calls.

Parameters to Context.RegisterDynamicProperty

Parameters | Description
name | String name for property
property | An object that implements IContributeDynamicSink
obj | The object/proxy for which the property is registered , may be null.
ctx | The context for which the property is registered.

If obj is non null then if it is a proxy, all calls made on the proxy are intercepted otherwise if it is a real object, all calls on the object are intercepted the ctx argument must be null

If ctx is non-null then the obj argument must be null and all calls entering and leaving the context are intercepted

If both ctx and obj are null then all calls entering and leaving all contexts are intercepted.

Context.UnregisterDynamicProperty enables the user to unregister the property, which implies that the sink will be dropped for subsequent remoting calls.

Unregister dynamic property. See RegisterDynamicProperty. Name, obj, ctx arguments should be exactly the same as a previous RegisterDynamicProperty call.

DynamicProperty Reference

IDynamicProperty Reference

The IDynamicProperty interface is implemented by properties that are registered at runtime through the RemotingServices.RegisterDynamicProperty API..

public interface IDynamicProperty
{
        String Name {get;}
}

Name

Returns the name of the dynamic property.

IContributeDynamicSink Reference

The IContributeDynamicSink interface is implemented by properties that are registered at runtime through the RemotingServices.RegisterDynamicProperty API. These properties can contribute sinks that are notified when remoting calls start/finish.

interface IContributeDynamicSink
{
  public  IDynamicMessageSink GetDynamicSink();
}

GetDynamicSink.

Returns the message sink that will be notified of call start/finish events through the IDynamicMessageSink interface.

IDynamicMessageSink Reference

IDynamicMessageSink is implemented by message sinks provided by dynamically registered properties. These sinks are provided notifications of call-start and call-finish with flags indicating whether the call is currently on the client-side or server-side (this is useful for the context level sinks).

interface IDynamicMessageSink
{
   public IMessageProcessMessageStart(
                  IMessage reqMsg,
                  bool bCliSide,
                  bool bAsync);
   public IMessage ProcessMessageFinish(
                  IMessage replyMsg,
                  bool bCliSide,
                  bool bAsync);
}

ProcessMessageStart

Indicates that a call is starting. The booleans tell whether we are on the client side or the server side and whether the call is using AsyncProcessMessage.

Parameter | Description
ReqMsg | Request Message
bCliSide | True if client side
Basync | True if async call

ProcessMessageFinish

Indicates that a call is returning. The booleans tell whether we are on the client side or the server side and whether the call is using AsyncProcessMessage.

Parameter | Description
replyMsg | Reply Message
bCliSide | True if client side
Basync | True if async call


TrackingService

Tracking Service Concepts

The TrackingService provides for a general tracking service with pluggable tracking handlers. Support is provided to be notified when:

  • An ObjRef has been generated (as the result of a marshal)
  • An ObjRef has been received (as the result of an unmarshal)
  • An object has been disconnected
  • An object has been called - Tracking handlers can contribute messages sinks to the various chains by exposing any of IContribute*Sink interfaces. When messages arrive into the appdomain, they can run through sinks (app domain, context, object). The tracking handler can use the message to enforce its particular behavior.

Tracking Services Reference

TrackingServices Reference

TrackingService provides a way to register, unregister and obtain a list tracking handers.

public class TrackingService
{
  public static void RegisterTrackingHandler(ITrackingHandler policy);
  public static void UnregisterTrackingHandler(ITrackingHandler policy);
  public static ITrackingHandler[] RegisteredHandlers;
};

RegisterTrackingHandler

Register a new Tracking Handler.

UnregisterTrackingHandler

Unregister the Tracking Handler.

TrackingHandlers

Returns an array of the currently registered Tracking Handlers.

ITrackingHandler Reference

The ITrackingHandler interface is called by Remoting when an event happens on an object or proxy.

public interface ITrackingHandler
{
    void MarshaledObject(Object obj, ObjRef or);
    void UnmarshaledObject(Object obj, ObjRef or);
    void DisconnectedObject(Object obj);
}

MarshaledObject

Notify a handler that an object has been marshaled

UnmarshaledObject

Notify a handler that an object has been unmarshaled

DisconnectedObject

Notify a handler that an object has been disconnected


Call Context

Call Context Concepts

Call chain. The state bag is passed down the call chain. The call context provides a set of properties that are carried with the execution code path. Entries can be added to the Call Context as it travels down and back up the execution code path.

Context Switch. The state bag is passed across context boundaries. As a call flows from context to context, the call context is propagated.

AppDomain Switch. The state bag is passed across application domain boundaries. Cloned at AppDomain boundary.

The Call Context is passed in the IMessage as it flows between Contexts and AppDomains. The entry in the message is named “__CallContext”. Message sinks can add and lookup entries in the __CallContext as passes through the message sink chain.

Slots. The Call Context provides data slots for the call path. The data slots are unique per call path i.e. the state is not shared across call paths. The slots are named. The name is used to access the data slot. The slot can be explicitly freed using the name.

“Contextify” System.Delegate.BindToContext. Specifically, the content of the state bag is propagated from System.Delegate.BindToContext() call to the Invoke call.

CallContext Reference

This class exposes the API for the users of call context. All methods in CallContext are static and operate upon the call context in the Thread.

NOTE: CallContext is a specialized form of something that behaves like TLS for method calls. However, since the call objects may get serialized and deserialized along the path, it is tough to guarantee identity preservation.

public sealed class CallContext
{
  public static Object GetData(String name);
  public static void SetData(String name, Object data);
  public static void FreeNamedDataSlot(String name);
}

GetData

Retrieve an object by name from the CallContext.

SetData

Store an object using a name in the CallContext.

FreeNamedDataSlot

Explicitly frees a named data slot.


One Way

One Way Concepts

One way methods have the pattern of fire and forget. The OneWayAttribute is used to indicate that the method has a void return and in only parameters. Byref and return values are not supported. The method can execute synch or asynch with respect to the caller. The caller does not make assumptions that the one-way call has executed on the server object when thread control returns.

One Way Reference

OneWayAttribute Reference

Attribute for marking methods as one way

namespace System.Runtime.Remoting
{
    [AttributeUsage(AttributeTargets.Method)]
    public class OneWayAttribute : Attribute
    {
    }
}


Async Remoting

Async Delegates Concepts

Async Delegates are supported for Remoting. See the Async Programming Model section for details.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Remoting Activation

© 2000 Microsoft Corporation. All rights reserved.


Remoting Activation Concepts

Read the following topics prior to reading this specification:

  • Remoting Object Model
  • Remoting Configuration
  • Remoting Channels

Key Points

  • Serve up existing object instances:
    • Activator.GetObject / RemotingServices.Connect
    • Choice 1: Serve up the same instance for each method call (singleton)
    • Choice 2: Serve up a new instance for each method call (singlecall)
  • Serve up a new object instance for each Activation Request:
    • CreateInstance / newobj / “placement new”
  • May serve up the object instance in same AppDomain / Application, same Context
  • May serve up the object instance in another AppDomain / Application
  • AppDomain / Application may be in the same process, different process, different machine
  • Allow the Application URI to be specified in configuration
  • Allow the Application URI to be specified at the activation callsite

Activation and Serving up a new object instance

New objects are activated in an Application. An Application is hosted within an AppDomain. An application can be configured to hosted in a remote AppDomain, relative to the current application. If the object is activated in a remote application, then the actual object instance is created in the remote application appdomain.

A whole assembly or an individual type within an assembly can be configured to be activated in a remote application. A type can be specified using Activator.CreateInstance to activate in a remote application.

Activation happens in two stages:

In the first stage, a decision is made regarding whether the current application is OK.

  • If the answer is Yes, a local instance of the object is activated. This may involve running a constructor if applicable.
  • If the answer is No, the object is created in the remote application’s AppDomain (this is detailed throughout this specification). Activation in a remote application involves context attributes and properties.

In both the stages, context properties of the current context and context attributes defined on the type e.g. Foo (hereafter referred to only as context properties) are involved.

  • In the first stage, the context properties are asked if the current context is OK and the processing stops immediately after the first property that answers No.
  • In the second stage, the context properties are passed the Message object that represents the Construction Call.

The context properties can perform the following operations:

  • They can add or remove context properties in the constructor message object, which are used for either creating a new context or identifying an existing compatible context.
  • They can replace or chain a new activator object in the construction message. By default, the constructor message object contains a cross-context activator. Note that activators form a hierarchy and context properties should honor the hierarchy in the sense that any property should not replace an AppDomain activator with a context activator, but instead chain the Context activator to be called after the AppDomain activator.

At the end of the second stage, activation is delegated to the activator present in the constructor message object. The activator depending on its type performs either a cross-machine, cross-process, cross-appdomain, or cross-context activation. The interfaces used during the two stages of activation is listed below:

The IActivationProperty interface is exposed by the context properties that wish to participate in activations happening in the context on which they are present.

public interface IActivationProperty
{
  bool IsContextOK(Context ctx, IConstructionCallMessage msg);
  void GetPropertiesForNewContext(IConstructionCallMessage msg);
}

The IContextAttribute interface is implemented by attribute classes defined on a type. The attributes participate in activations of the type on which which they are defined.

public interface IContextAttribute
{
  bool IsContextOK(Context ctx, IConstructionCallMessage msg);
  void GetPropertiesForNewContext(IConstructionCallMessage msg);
}

The IActivator interface is implemented by activators. Note that activators form a hierarchy that must be observed.

public interface IActivator
{
  IActivator NextActivator  {get; set;};
  String Hierarchy {get; };
  IConstructionReturnMessage Activate(IConstructionCallMessage msg);
}

To understand the mechanism for deciding when and how to create new applications, let us consider the properties implicitly added on every context by the runtime.

There are three such properties:

  1. Process property.
  2. AppDomain property.
  3. Application property.

The implementation of the above properties supports extensibility. For example, both Process and AppDomain properties support registration functions for calling arbitrary code at process (basically in all AppDomains) and AppDomain (a specific one) boundaries during call ins and call outs.

Appdomain and Applications are distinct concepts. An AppDomain is a unit of shutdown, debuggability, etc. A single AppDomain can host multiple applications. Applications are a unit of deployment, configuration, etc.

NGWS runtime types can be configured differently in different applications. For example, in application “WingTipToysApp”, type CFoo can be configured as compatible in the sense that it can run inside it. CFoo is activated locally.

However, in the application “TailSpinToysApp”, type CFoo can be configured to run specifically in remote application “WingTipToysApp”, on Wing Tip's machine, which is specified through an Application URI in the Remoting Configuration portion of Tail SpinToysApp's application configuration file. CFoo is activated remotely. Note that the only types that can be configured in this manner are MarshalByRef and ContextBound ones. It is pointless to configure Unbound types as they are always marshaled by value (See the Remoting Object Model section for details on Marshal By Value).

Let us consider the above example scenario of “TailSpinToysApp” activating a CFoo instance in “WingTipToysApp”. Every appdomain supports a well-known activator object that can receive messages on the the object URI “RemoteActivationService” that can be used to activate instances of a given type in that appdomain. It is registered at appdomain startup by the remoting infrastructure. During activation of an instance of CFoo, the application property representing application “TailSpinToysApp” notices that CFoo instances need to be created in “WingTipToysApp” at the Application URI for WingTipToysApp (the Application URI could point to Wing Tip's machine, wingtiptoys), and replaces the activator present in the constructor message with the activator obtained by the following method:

IActivator activator = (IActivator)Activator.GetObject(
       typeof(Activator),
       “http://www.WingTipToys.com/WingTipToysApp/RemoteActivationService.rem”);

Activation to ASP+ over the HTTP Channel

The above call to Activator.GetObject returns a proxy representing the “Activator” type that connects to a well-known instance running at the URL “http://www.WingTipToys.com/WingTipToysApp/RemoteActivationService.rem”. At the end of second stage, the Remoting activation infrastructure, executing as part of the constructor call on the proxy returned by the first stage, delegates activation to the activator retrieved from the constructor message by making the following call.

IActivator activator = constructorMessage.Activator;
IConstructorReturnMessage returnMessage =
               activator.CreateInstance(constructorMessage);
ObjRef objref = returnMessage.GetReturnValue();

The following operations happen during above activator.CreateInstance call above:

  • The proxy creates a SOAP request packet and sends it to the URL http://www.WingTipToys.com/WingTipToysApp/RemoteActivationService.rem
  • The SOAP request arrives at the Web Server IIS process (inetinfo) listening on port 80) at “WingTipToys” and it is routed to ASP+ ISAPI handler.
  • The ASP+ handler parses the URL prefix http://www.WingTipToys.com/WingTipToysApp, locates or creates the appdomain for “WingTipToysApp” and routes the SOAP request to it. If the application has been configured for isolation, the request may be routed to another process. The process may be started by IIS/ASP+.
  • In the appdomain running WingTipToysApp, the SOAP request passes through ASP+ pipeline. At the end of its pipeline, ASP+ notices the “.rem” extension, identifies the IHTTPHandler registered with it by the SOAP channel, and finally hands off the SOAP request to the identified instance of IHTTPHandler.
  • The SOAP channel identifies that the request is destined to the object registered at RemoteActivationService.rem, invokes the CreateInstance method on it and creates SOAP response packet from the invocation results. The objects constructor is called if present.
  • The SOAP response unwinds through the above stack and finally reaches the activation infrastructure executing as part of the constructor call on the proxy.
  • The activation infrastructure unmarshals the ObjRef present in the response thereby connecting the proxy to the activated instance.

Activation Programming Models

Activation Information needed to activate

Activation Attributes

One or more attributes that can participate in activation.

Passed as an array of objects.

URLAttribute(“http://hostname/appname/objectURI”)

Identify Constructor

The constructor needs to be identified. Choices

  • Constructor token.
  • Constructor Metadata signature.

Constructor arguments

Zero or more arguments to the constructor.

Activation - Sample Class

package SomePackage;
public class SomeClass : MarshalByRefObject
{
  public SomeClass(int a, float b)
  {
    //. . .
  }
  public void SomeMethod(String s)
  {
    //. . .
  }
}

Activation – new MyFoo

Key Points

  • Users optional passes constructor args
  • Remote Application determined from Application Configuration file

Consider the following code snippet:

// . . .
CFoo foo = new CFoo(1, “SomeString”);
foo.Method1();
// . . .

When the above code snippet is JITed, the JIT is given different JITHelpers based on the type of CFoo. If CFoo is an Unbound type, the JITHelper is a fast one that always returns a new uninitialized instance of CFoo on which JIT subsequently runs the constructor. If CFoo is either MarshalByRef or ContextBound, a different JITHelper is a returned that may initiate activation in remoting sense. The following is the description of operations performed by the Remote Activation JITHelper.

Activation happens in two stages: In the first stage, a decision is made regarding whether the current Application is OK. If the answer is Yes, an uninitialized instance of CFoo is returned, JIT runs the constructor on it, and the activation is done. If the answer is no, a proxy that can handle only calls to constructors is returned and JIT subsequently calls the constructor on the proxy, and the second stage processing begins. See the section Activation Serving up a new object instance for details on how remote activation works.

Activation – Activator.GetObject

The Activator.GetObject method (in an number of variations) provides the ability to obtain a “proxy” to an already running remote object Well Known object. Only when calls are made on the “proxy” will messages be sent remotely.

public void SomeCode()
{
  //. . .
  String url = “http://hostname/vroot/objectname”;

  // Resolve the URL and return a proxy
  SomeObject so = (SomeObject)Activator.GetObject(
                           typeof(SomePackage.SomeClass),
                           url);

  // Call methods on the remote instance
  so.SomeMethod(“Hello World”);
  //. . .

Activation – Activator.CreateInstance

Key Points

  • Users passes constructor args
  • Users optional passes activation attributes
  • Application can be determined from:
    • Optional array passed as the parameter of CreateInstance
    • or, the Application Configuration file

Activation happens in two stages: In the first stage, a decision is made regarding whether the current Application is OK. If the answer is yes, an uninitialized instance is created, the constructor is run on it, and the activation is done. If the answer is no, then the second stage processing begins. See the section Activation Serving up a new object instance for details on how remote activation works.

public void SomeCode()
{
  //. . .
  // Build up array of constructor arguments
  Object[] args = new Object[2];
  args[0] = “Some Argument”;
  args[1] = 2.3;

  // Build up array of activation attributes
  Object[] activationAttributes = new Object[3];
  activationAttributes[0] =
       new URLAttribute(“http://hostname/vroot/objectname”);
  // Other objects can be passed
  activationAttributes [1] = . . .
  activationAttributes [2] = . . .

  // Activate the remote instance and return a proxy
  SomeObject so = (SomeClass)Activator.CreateInstance(
                               typeof(SomePackage.SomeClass),
                               args,
                               activationAttributes);

  // Call methods on the remote instance
  so.SomeMethod(“Hello World”);
  //. . .
}

Activation – Activator.CreateInstance (AppDomain & ObjectHandle)

Key Points

  • Pass the actual target appdomain in the object array to CreateInstance
  • Pass the assembly and target appdomain in the object array to CreateInstance
  • Can call a variation that returns an ObjectHandle then needs to be unwrapped
Type t = Type.GetType(“SomePackage.SomeClass”);
Object[] objects = new Object[2];
objects[0] = AppDomain.CreateDomain( . . .)
objects[1] = // Specify the assembly
SomeClass sc = (SomeClass)Activator.CreateInstance(t, . . ., objects );

or

ObjectHandle oh = Activator.CreateInstance(t, . . ., objects );
SomeClass sc = (SomeClass)oh.Unwrap();

// For in memory Assemblies pass the byte[]

Type t = Type.GetType(“SomePackage.SomeClass”);
Object[] objects = new Object[2];
objects[0] = AppDomain.CreateDomain( . . .)
objects[1] = // pass the byte[] assemblyInMemory;
SomeClass sc = (SomeClass)Activator.CreateInstance(t, . . ., objects );

Activation Steps in Detail for Serving up a new object instance

Step 0) Message from EE Activation to Attributes

Steps

  • Run through context attributes on the class, Activator properties, call site attributes calling IContextAttribute.IsContextOK on the current context.
  • Activator properties are free to change the Activator to point another activator in the IConstructionCallMessage. If the Activator is in a remote application this is point where a Connect to the remote activator is performed.
  • Attribute can pass cookies in the message.

IConstructionCallMessage

  • Callsite Activation Attributes (__CallSiteActivationAttributes)
  • ServerType (__ServerType)
  • ContextProperties as System.Collections.List (__ContextProperties)
  • Activator (_Activator)

Step 1) Message from EE Activation to Activator

Steps

  • Ask attributes for prototype context properties.
  • EE Activation flows construction call message out of current context by sending it to the first sink in the client context sink chain.
  • Client Context Terminator Sink calls detects that this a construction call and sends the construction call message through current Context Properties that have the IContextPropertyActivator interface. The properties can manipulate the construction call message.
  • Client Context Terminator Sink calls IActivator in the IConstructionCallMessage.Activator property

IConstructionCallMessage

  • Callsite Activation Attributes (__CallSiteActivationAttributes)
  • ServerType (__ServerType)
  • Constructor Arguments (__Args)
  • Constructor Identifier (__MethodName and __MethodSignature)
  • ContextProperties System.Collections.List (__ContextProperties)
  • Any other entries added in Step 0

Step 2) Activator

Steps

  • If this is a cross appdomain / process / machine activation, then it is routed to the appropriate remote activator in the remote application.
  • The activator will run through the context attributes and global attributes asking to contribute context properties GetPropertiesForNewContext i.e. they add to the __ContextProperties List.
  • A new context is created using the List of __ContextPropeties from the construction call message.
  • The Construction message will be sent into the new context through Context “DoCallback”.
  • The ServerContext terminator sink will detect that is a construction call message.
  • The Server Context Terminator Sink sends construction call message through the current Context Properties that have the IContextPropertyActivator interface. The properties can manipulate the construction call message.
  • The Server Context Terminator Sink creates a new instance of the object.
  • The new object will be marshaled (the server object sink chain is created when the object is marshaled).
  • The construction return message is created and the OBJREF is placed in the __Return entry. The OBJREF is placed instead of the actual object to prevent OBJREF to Object to OBJREF transitions as it flows through the system.
  • The construction return message flows back through the sinks.
  • The Server Context Terminator Sink sends construction return message through current Context Properties that have the IContextPropertyActivator interface. The properties can manipulate the construction return message.
  • The construction return message is returned to the caller.
  • The construction return message is routed back to the orginal context / appdomain / process / machine.

IConstructionCallMessage

  • Callsite Activation Attributes (__CallSiteActivationAttributes)
  • ServerType (__ServerType)
  • Constructor Arguments (__Args)
  • Constructor Identifier (__MethodName and __MethodSignature)
  • ContextProperties System.Collections.List (__ContextProperties)
  • Any other entries added in Step 0, 1

Step 3) Activator back to Client Context Terminator Sink

Steps

  • The activator returns and call process back up to the client context terminator sink.
  • The Client Context Terminator Sink sends construction return message through current Context Properties that have the IContextPropertyActivator interface. The properties can examine and manipulate the construction return message.

Step 4) Call unwinds to EE Activation

Steps

  • The EE activation unmarshals the embedded OBJREF, creates the envoy sink chain and connects the proxy to the front of the context chain.

Step 5) EE returns to execution code

Steps

  • The proxy is returned.


Remoting Activation Reference

Activator Reference

The Activator class provides a standard way to create and obtain instances of objects in the NGWS runtime Runtime in late bound way. These are CLS compliant they can be called from many languages.

Activator.CreateInstance. The Activator.CreateInstance method creates an instance of the object with full binding support. Variations of the method will allow the activation call site attributes to be passed as an array of objects. The activation can potentially occur remotely. Once the object has been activated remotely the CreateInstance will return.

Activator.CreateInstance. (ObjectHandle) Create an instance using the name of type and the assembly where it exists. This allows types to be created remotely without having to load the type locally.These will return an ObjectHandle that needs to unwrapped in order to access the real object.

Activator.CreateInstanceFrom. [Assembly, ObjectHandle] Create an instance from a particular assembly.

Activator.GetObject. The Activator.GetObject method (in an number of variations) provides the ability to obtain a “proxy” to an already running remote object Well Known object. Only when calls are made on the “proxy” will messages be sent remotely.

public class Activator
{
    static public Object CreateInstance(
                Type type,
                BindingFlags bindingAttr,
                Binder binder,
                Object[] args,
                CultureInfo culture);
    static public Object CreateInstance(
                Type type,
                BindingFlags bindingAttr,
                Binder binder,
                Object[] args,
                CultureInfo culture,
                Object[] activationAttributes);
    static public Object CreateInstance(
                Type type,
                Object[] args);
    static public Object CreateInstance(
                Type type,
                Object[] args,
                Object[] activationAttributes)
    static public Object CreateInstance(
                Type type);

    static public ObjectHandle CreateInstance(String assemblyName,
                                              String typeName);

    static public ObjectHandle CreateInstance(String assemblyName,
                                              String typeName,
                                              bool shared);

    static public ObjectHandle CreateInstance(
                    String assemblyname,
                    String typeName,
                    bool shared,
                    Object[] activationAttributes)

    static public ObjectHandle CreateInstanceFrom(
                    String assemblyFile,
                    String typeName);

    static public ObjectHandle CreateInstanceFrom(
                    String assemblyFile,
                    String typeName,
                    bool shared);

    static public ObjectHandle CreateInstanceFrom(
                    String assemblyFile,
                    String typeName,
                    bool shared,
                    Object[] activationAttributes);

    static public ObjectHandle CreateInstance(
                    String assemblyName,
                    String typeName,
                    bool ignoreCase,
                    BindingFlags bindingAttr,
                    Binder binder,
                    Object[] args,
                    CultureInfo culture,
                    bool shared,
                    Object[] activationAttributes,
                    Evidence securityInfo)

    static public ObjectHandle CreateInstanceFrom(
                    String assemblyFile,
                    String typeName,
                    bool ignoreCase,
                    BindingFlags bindingAttr,
                    Binder binder,
                    Object[] args,
                    CultureInfo culture,
                    bool shared,
                    Object[] activationAttributes,
                    Evidence securityInfo)

    static public MarshalByRefObject GetObject(Type type,String url);
    static public MarshalByRefObject GetObject(
                    Type type,
                    String url,
                    Object state);
}

IConstructionCallMessage Reference

Used to identify that this is a construction call message.

public interface IConstructionCallMessage : IMethodCallMessage
{
  IActivator Activator {get;set}
  Object[] CallSiteActivationAttributes  { get; }
  Type ServerType                        { get; }
  IList ContextProperties                { get; }
}

Activator. Set or Return the Activator.

CallSiteActivationAttributes. Returns an array of the callsite activation attributes.

ServerType. Returns the server type.

ContextProperties. Returns a List of context properties.

IConstructionReturnMessage Reference

Used to identify that this is a construction return message.

interface IConstructionReturnMessage : IMethodReturnMessage
{
  //. . .
  //. . .
}

IActivator Reference

The IActivator interface is implemented by activators. Note that activators form a hierarchy that must be observed.

interface IActivator
{
  //. . .
  IActivator NextActivator  {get; set;};
  String Hierarchy {get; };
  IConstructionReturnMessage Activate(IConstructionCallMessage msg);
  // . . .
}

NextActivator. Get or Set the next activator in the activation sequence

Hierachy. Return a string version of the hierarchy.

Activate. Activate the object specified by the construction call message.

IActivatorProperty Reference

The IActivationProperty interface is exposed by the context properties that wish to participate in activations happening in the context on which they are present.

public interface IActivationProperty
{
  bool IsContextOK(Context ctx, IConstructionCallMessage msg);
  void GetPropertiesForNewContext(IConstructionCallMessage msg);
}

IsContextOK. Returns whether the context parameter meets the context attribute’s requirements. For activators they can determine that the current application is not OK and will return false.

GetPropertiesForNewContext. The Attribute can add Context Properties directly to the __ContextProperties List in the IConstructionCallMessage. The context properties will be added to the new context. The default implementation in ContextAttribute will add this to the context property List. Context attributes are free to override GetPropertiesForNewContext and can implement their own behaviour e.g. add a new class that implements the context property to the List.

IContextPropertyActivator Reference

IContextPropertyActivator is exposed from all context properties that in interested in participating in activation and that may have not provided a message sink. Client Context Properties could information to the construction message that their counterparts at the remote site could look at during activation.

IContextPropertyActivator is used to collect information from the client and server context property.

IContextPropertyActivator is also used to deliver information from the Server Context Properties to the Client Context Properties when returning to the Client Context.

public interface IContextPropertyActivator
{
    public void CollectFromClientContext(
                          IConstructionCallMessage msg);
    public void CollectFromServerContext (
                          IConstructionReturnMessage msg);
    public bool DeliverClientContextToServerContext (
                        IConstructionCallMessage msg);
    public bool DeliverServerContextToClientContext (
                        IConstructionReturnMessage msg);
}

CollectFromClientContext. Called on each client context property that has this interface, before the construction request leaves the client. The context property can examine and manipulate the contents of the IConstructionCallMessage.

CollectFromServerContext. Called on each server context property that has this interface, before the construction response leaves the server for the client. The context property can examine and manipulate the contents of the IConstructionReturnMessage.

DeliverClientContextToServerContext. Called on each server context property that has this interface, when the construction request arrives at the server from the client. The context property can examine the contents of the IConstructionCallMessage.

DeliverServerContextToClientContext. Called on each client context property that has this interface, when the construction request returns to the client from the server. The context property can examine the contents of the IConstructionReturnMessage.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Remoting Channels

© 2000 Microsoft Corporation. All rights reserved.


Remoting Channel Concepts

This section describes Channels.

Remoting Channel Concepts

Channels transport messages between Applications across remoting boundaries e.g. AppDomains, Processes and Machines. These crossings include both inbound and outbound. A channel can listen on an endpoint for inbound messages or send to an endpoint for outbound messages or both. This provides an extensibility point in the NGWS runtime to plug in a wide range protocols, even though the NGWS runtime may not be at the other end of the channel. NGWS runtime objects can be used to expose and represent a wide and rich set of semantics and entities. The Channel provides the extensibility point to convert the messages to and from the specific protocols.

Channels must expose the IChannel interface, which provides informational properties such as the ChannelName, ChannelPriority and MimeType. Channels are registered via ChannelServices.RegisterChannel. Channels can also be loaded from the Remoting Configuration. See Remoting Configuration for details.

If there are NGWS runtimes at both ends of the channel, then a "Session" is created between the two ends, in order to connect the client and sever sink chains on either side of the boundary together. The client part of the channel is located at the end of Client context sink chain. The server part of the channel is located at the start of the Server context sink chain. Messages are delivered to the client channel via the IMessageSink interface, travel through the channel and are received by the server channel. The server channel delivers the message to the first server context sink.

Remoting Channel Properties Concepts

Channels Properties are used to configure the channel at execution time. Channel Messages Sinks can expose the IDictionary interface. See the Remoting Security section for details on how to use properties for security purposes.

Standard Channel Properties

The following are channel properties that all channel expose.

Name | Example | Description
ChannelPriority |  | Priority of the channel
ChannelName | http tcp | Name of the channel
MimeType | text/xml bin/bin | Mime type the channel uses.

Channels and Configuration

Channel Sinks

HTTP Channel

Key Points

  • Provide communication between sender and receiver using the HTTP protocol as a transport.
  • Provides encoding of payloads in SOAP, an XML encoding standard.
  • Receiver can receive HTTP requests and send HTTP responses in ASP+ and on a TCP socket.
  • SOAP 1.1 Compliant

HTTP Channel Sender Sink

Key Points

  • Sender uses System.Net HTTP support to make HTTP Requests and process HTTP Responses
  • Uses System.Net.WebRequest for HTTP Support
  • Provides Synchronous and Async communication
  • Exposes User and Password as Properties for credentials

HTTP Channel Receiver Sink - Generic

Key Points

  • Receives Requests from TCP Receiver
  • Receives Requests from ASP+ Receiver
  • Enforces the SOAP protocol: SOAPAction

HTTP Channel Receiver - TCP Receiver

Key Points

  • Uses System.Net TCP Support, which delivers HTTP requests to the HTTPChannel and delivers HTTP responses to TCP
  • Parses HTTP Headers
  • ChannelData
    • WellKnown: return an array of strings:
    • http:// DNSHostName:port/vpath/../
    • http:// DNSHostName:port/vpath/../

e.g.

http://www.contoso.com:8080/media/../

http://www.contoso.com:8081/media/../

  • Client Activated: return an array of strings
  • return http://host_ip_address:[port]/vpath/../
  • return http://host_ip_address:[port]/vpath/../

e.g.

http://111.222.233.123:8080/media/../

http://111.222.233.123:8081/media/../

HTTP Channel Receiver - ASP+ Receiver

Key Points

  • ASP+ HTTPRemotingHandler – delivers HTTP requests to the HTTPChannel and delivers HTTP responses to ASP+
  • Pass HTTP Authentication information into CallContext
  • Loads the application configuration for each vroot / application root that it receives only once.

HTTP Channel Sink Sender Properties

Name | Example | Description
UserName | joe | Username uses for Web Security.
Password | secret | Password used for Web Security
Domain |  | Domain used for Web Security. Optional
PreAuthenticate | True | If true, send the credentials on the first request.
URL | http://. . . | Complete URI for the proxy.
ProxyName | myproxy | The name of the HTTP Proxy
ProxyPort | 80 | The proxy port number.

HTTP Channel Receiver Properties

Name | Example | Description
Host | localhost server.domain.com | The hostname of the receiver
Ports | 80;8080 | The ports the Channel can listen on

HTTP Channel Configuration

See the Remoting Configuration Section for details.

HTTP Channel Security

Key Points

  • WebSecurity: basic auth, digest, passport, ntlm, kerberos
  • Utilizes System.Net and System.Web security

See the Remoting Security Section for more details.

HTTP Channel Compatiblity

HTTP Channel is compliant with SOAP 1.1

TCP Channel

Key Points

  • Provide communication between sender and receiver using TCP sockets
  • Provides encoding of payloads in BIN and the industry standard SOAP.

TCP Channel Sender Sink

Key Points

  • Sender uses System.Net TCP support to make Requests and process Responses
  • Provides Synchronous and Async communication

TCP Channel Receiver Sink

Key Points

  • Uses System.Net TCP Support
  • TCP Listener
  • ChannelData
    • WellKnown: return an array of strings:
    • tcp:// DNSHostName:port/../
    • tcp://DNSHostName:port/../

e.g.

tcp://www.contoso.com:8080/../

tcp://www.contoso.com:8081/../

  • Client Activated: return an array of strings
  • return tcp://host_ip_address:port/../
  • return tcp://host_ip_address:port/../

e.g.

tcp://111.222.233.123:8080/..

tcp://111.222.233.123:8081/..

TCP Channel Sink Sender Properties

Name | Example | Description
URL | tcp://host/. . . | Complete URI the proxy.

TCP Channel Receiver Properties

Name | Example | Description
Host | localhost server.domain.com | The hostname of the receiver
Ports | 9000;9010 | The ports the Channel can listen on

TCP Channel Configuration

See the Remoting Configuration Section for details.

TCP Channel Security

  • See the Remoting Security Section for more details.


Remoting Channel Reference

ChannelServices Reference

ChannelServices provides for Channel registration, resolution, URL parsing and message sink creation. Activator.GetObject, RemotingServices.Unmarshal and RemotingServices.Connect utilize ChannelServices in resolving an object URL to setup client side infrastructure such as a proxy. The object URL passed to the registered channels to find a channel that can return a message. The message sink is used to deliver messages to the channel.

public class ChannelServices
{
  public static IMessageCtrl AsyncDispatchMessage(
                        IMessage msg,
                        IMessageSink replySink);
  public static IChannel GetChannel (String name);
  public static IDictionary GetChannelSinkProperties(Object obj)
  public static String[] GetURLsforObject(MarshalByRefObject obj);
  public static void RegisterChannel (IChannel channel);
  public static IChannel[] RegisteredChannels {get;}
  public static IMessageSink ResolveChannelMessageSink(Object obj);
  public static void StartListening (String name, Object data);
  public static void StopListening (String name, Object data);
  public static IMessage SyncDispatchMessage(IMessage msg);
  public static void UnregisterChannel (IChannel channel);
};

AsyncDispatchMessage

This method is used by the channel to dispatch the incoming messages to the server-side chain(s) based on the URI embedded in the message. The URI uniquely identifies the receiving object.

GetChannel

Perform a lookup using the channel name on the registered channels. Returns the channel.

GetChannelSinkProperties

Get the message sink dictionary of properties for a given proxy.

GetURLsforObject

Return an array of all the URLs for an object.

RegisterChannel

Registers a channel. Takes in the IChannel interface from a Channel object.

Throws ArgumentNullException if channel parameter is null.

Throws RemotingException if channel already registered.

RegisteredChannels

Returns an array of the currently registered channels.

ResolveChannelMessageSink

Find the channel message sink associated with a given proxy.

StartListening

Instructs a particular channel to start listening for requests. The data object can be used to pass specific initialization state to the channel.

StopListening

Instruct a particular channel to stop listening for requests. The data object can be used to pass specific initialization state to the channel.

SyncDispatchMessage

This method is used by the channel to dispatch the incoming messages to the server-side chain(s) based on the URI embedded in the message. The URI uniquely identifies the receiving object.

UnregisterChannel

Unregister a paricular channel from the registered channels list.

Throws ArgumentNullException if channel parameter is null.

Throws RemotingException if channel is not registered.

IChannel Reference

public interface IChannel
{
  public int ChannelPriority {get; }
  public String ChannelName  {get; }
  public String MimeType     {get; set; }
}

ChannelName

Returns the name of the channel.

ChannelPriority

This read-only property returns the priority of the channel.

MimeType

This read-write property sets or returns the mime type of the channel.

IChannelSender Reference

The sending side of channels must expose the IChannelSender interface.

public interface IChannelSender : IChannel
{
  IMessageSink CreateMessageSink(
                                  String url,
                                  Object data,
                                  out String objectURI);
}

CreateMessageSink

The Channel returns a channel message sink, that delivers messages to the specified URL or channel data object. The completed String URL or the Object state can be NULL. Both cannot be NULL.

A message Sink is returned. Consumes the channel specific portion of the URI and returns the rest of the URI. The channel must return an objectURI. If objectURI is null the complete url is used for the objectURI.

IChannelReceiver Reference

The receiving side of channels must expose the IChannelReceiver interface.

public interface IChannelReceiver : IChannel
{
  public Object ChannelData {get; }
  public String[] GetURLsforURI(String objectURI);
  public void StartListening(Object data);
  public void StopListening(Object data);
}

ChannelData

This read-only property returns the channel specific data. Used when Marshal is called and an ObjRef is created.

GetURLsforURI

Return an array of all the URLs for a URI. Called by ChannelServices. GetURLsforObject.

StartListening

Instruct a particular channel to start listening for requests. The data object can be used to pass specific initialization state to the channel.

StopListerning

Instruct a particular channel to stop listening for requests. The data object can be used to pass specific state to the channel.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Remoting Contexts

© 2000 Microsoft Corporation. All rights reserved.


Remoting Contexts Concepts

Contexts Concepts

Key Points

  • A boundary containing a collection of objects with the same usage rules
  • Context rules include: synchronization, thread affinity, transactions etc
  • Classes are marked with Context Attributes which provide the usage rules

Execution system

  • On the newobj IL instruction, the EE finds a compatible or creates a new Context for the object
  • Ensures the rules are enforced when entering and leaving a context
  • Provides synchronization rules for managed objects
  • Flexible synchronization model

A context is an ordered sequence of properties that define an environment for the objects resident inside it. Contexts get created during the activation process for objects that are configured to require certain automatic services such synchronization, transactions, just-in-time activation, security etc. Multiple objects can live inside a context.

Below is a brief summary of some of the context characteristics:

  • Contexts are a general mechanism that bound a set of objects.
  • A context is composed from a set of context properties.
  • Classes that can be bound to a context are called Context-bound classes.
  • Any call from an object in one context to an object in another context will go through a context proxy and be affected by the policy that the combined context properties enforce.
  • Contexts are created during object activation i.e. a new object is either placed in an existing context or a new one is created for it, but once there, the object stays in its context for life.
  • A new object's context is generally chosen based on meta-data attributes on the class. This mechanism is extensible via custom attributes. These are known as static context properties, which are compiled into the class meta-data. Dynamic (also known as configuration) context properties can be applied and configured by administrators.
  • As a general rule, contexts are freed when there are no longer any external references to objects in the context and there are no threads executing in the context.
  • There is one special kind of context property called an app domain property. An App Domain property defines the boundary that contains the set of statics for a program (set of classes). If the same class is loaded into two contexts with different App Domain properties then two copies of all of its static member variables are created.

Because all context-bound objects can be proxied across contexts, this imposes certain restrictions on how code can manipulate context-bound objects. For example, code cannot access a field on a context-bound object directly. Another example is that non-virtual calls must be made through an indirection and no inlining is allowed (to allow proxies when appropriate). This makes context-bound objects a good bit more expensive than one would like.

Classes, which are not context-bound are sometimes referred to as agile classes.

This should not be confused with agile locations (described below in the location section), although the ideas are parallel.

Agile objects are not bound to contexts and they are not proxied across contexts within the same AppDomain. Agile objects are only agile within an App Domain. Agile objects “live” in the Default Context (of which there is one per app-domain). The default context has no non-trivial properties. Any object passes across an AppDomain boundary is considered to be remoted.

Context-bound objects are very appropriate for transaction isolation / propagation and other features where the cost is acceptable.

Context Attributes and Context Properties Concepts

Types (classes) can be attributed with custom attributes. IL generator (compiler) support is needed for custom attributes. Context-bound classes can be attributed with specialized custom attributes known as Context Attributes. Context Attributes provide context properties which when added to a context can enforce policies such as Synchronization, Thread Affinity, or Transactions etc.

Many IL Generators (language compilers) support injecting custom attributes onto classes. This allows for context attributes to be attached to a type at compile time. Custom attributes are stored within the metadata of the PE.

Context Attributes can be applied to a type statically i.e. embedded within the metadata of the PE (compiled into the module) in which the type is defined. The developer set of static context properties are fixed.

Third parties can develop new Context Attributes and Properties.

Types (classes) can be attributed with custom attributes. IL generator (compiler) support is needed for custom attributes. Context-bound classes can be attributed with specialized custom attributes known as Context Attributes. Context Attributes provide context properties which when added to a context can enforce policies such as Synchronization, Thread Affinity, or Transactions etc.

IL Generators (language compilers) must support injecting custom attributes onto classes. This allows for static context attributes to be attached to a type at compile time. Custom attributes are stored within the metadata of the PE.

Client Context Chain and Server Context Chain Concepts

The server context sink chain is built by asking the first context property for a sink, proceeding to the last property. The client context sink chain is built by asking the last context property for a sink, proceeding to the first property. When a call enters a content (on the server context chain) and then makes a call out of the context (client context chain) we need to rollback the policies on the thread (e.g. Synchronization and Security).

Context Local Store Concepts

Key Points

  • Data slots per context.
  • Similar to Thread Local Store

The Context Local Store provides data slots for the context. The data slots are unique per context i.e. the state is not shared across contexts. Named slots are also supported.

Slots. Allocated explicitly.

Named Slots. Allocate by name.

Message Sink Chain Concepts

Active Context Properties achieve their functionality by contributing MessageSink objects, which perform interception on remote object method calls. These MessageSink objects can intercept both incoming calls on objects living in the context and/or outgoing calls to objects in other contexts. MessageSinks implement the IMessageSink interface.

Recall that the job of a proxy is to convert a method call invoked on it into a Message object. This Message object, which implements the IMessage interface, is passed from the client end to the server end by invoking ProcessMessage on MessageSink objects. MessageSinks are chained together in the sense that every MessageSink is responsible for calling ProcessMessage on the next MessageSink after it has performed its work. For instance, a synchronization related messageSink may cause a lock to be acquired / released and then delegate further to the downstream message sink.

From a Contexts standpoint, interception requirements lead to the following MessageSink Chains, listed in order as a call goes from the proxy to the real object:

  • Server Envoy Chain
  • Client Context Chain
  • Server Context Chain
  • Server Object Chain

At the end of each of the above chains is an infrastructure provided MessageSink object that ‘does the required work’ and hands the Message object to the next chain. For instance, at the end of the Server Context Chain (which would be common to all objects living in the context), we delegate the call to the chain for the correct Server Object chain, since the latter would be object specific. Tasks like this will be the responsibility of the infrastructure sinks.

Cross Context Call Concepts

Let us walk through a scenario where a context-bound object is called from another context. We will assume that at some point during activation, the client and server contexts have been created for us.

  1. A context-bound object reference is stored in an agile location (Wrap operation):
    • Obtain Server Object Chain by invoking GetServerObjectChain on the native context of the server object. The chain is initially created by calling GetObjectSink method on those server context properties that implement IContributeObjectSink and cached for future use.
    • Obtain Server Context Chain by invoking GetServerContextChain method on the native context of the server object. This chain is initially created by calling GetServerContextSink() method on those server context properties that implement IContributeServerContextSink interface and cached in the Context object for future use.
    • Create the Transparent Proxy and the associated Real Proxy. Also create the Server Envoy chain from those sinks in the Server Object and Server Context chains that implement IContributeEnvoySink interface. The RealProxy holds this chain.
  2. The agile location is accessed from another context to make a call to the object.
    • The call starts on the Transparent Proxy, which converts it to a Message object and forwards it to the Server Envoy chain.
    • The terminating sink obtains the Client Context Chain by calling GetClientContextChain method on the current context and delegates further processing to it. The chain is created the first time by invoking GetClientContextSink method on each property in the Dictionary returned by invoking GetPropertiesByType on the client context.
    • The terminating sink of the Client Context Chain does the work of transporting the call across the remoting boundary. In our case, it will switch to the server context do any other needed work like swizzle objRefs and ask the Server Context chain to process the message further.
    • The Message object passes through Server Context Chain and the terminating sink for that chain routes the message to the appropriate Server Object Chain.
    • At the end of the Server Object Chain is a dispatcher sink that dispatches the call to the real object

Context Relative Statics Concepts

Context Relative Statics provide for type member fields that are relative for the context. Whenever the static field is accessed on a particular context it will be unique or relative for that context only. The static is not shared between contexts. If the static field is access on another context the field will contain a different value, since the field is unique per context.

The member field is marked with a custom attribute System.ContextStatic.


Remoting Context Reference

Context Reference

public class Context
{
   public Context();
   public Int32 ContextID {get;}
   public IContextProperty[] ContextProperties {get;}
   public static Context DefaultContext {get;}
   public virtual IContextProperty GetProperty(String name);
   public virtual void SetProperty(IContextProperty prop);
   public virtual void Freeze()
   public override String ToString();

   public void DoCallBack(CrossContextDelegate deleg)

   public static LocalDataStoreSlot AllocateDataSlot()
   public static LocalDataStoreSlot AllocateNamedDataSlot(String name)
   public static LocalDataStoreSlot GetNamedDataSlot(String name)
   public static void FreeNamedDataSlot(String name)
   public static void SetData(LocalDataStoreSlot slot, Object data)
   public static Object GetData(LocalDataStoreSlot slot)

   public static bool RegisterDynamicProperty(
               IDynamicProperty prop,
               MarshalByRefObject obj,
               Context ctx);
   public static bool UnregisterDynamicProperty(
               String name,
               MarshalByRefObject obj,
               Context ctx);
}

Context

Initializes a new Context.

ContextID

Returns the Context ID for the Context.

ContextProperties

This read-only property returns an array of the context properties.

DefaultContext

This read-only property returns the default context for the AppDomain. Every thread in an app domain for which a context is not explicitly set is considered to be running in the DefaultContext.

GetProperty

Returns a specific context property by name.

SetProperty

Sets a specific context property by name.

Freeze

Freeze the context. Context properties cannot be added after the context has been frozen.

ToString

String representation of the context.

DoCallback

Used to execute code in another context. A callback may be done by calling the DoCallBack method on it by passing in a delegate. The delegate used to request a callback. The delegate must be of type CrossContextDelegate.

RegisterDynamicProperty

This allows users to register a property implementing IContributeDynamicSink with the remoting service. Based on the obj and ctx parameters, the property is asked to contribute a sink that is placed at some location in the path of remoting calls.

If multiple properties are registered, their sinks may be called in an arbitrary order that may change between calls.

Parameters

Parameters | Description
name | String name for property
property | An object that implements IContributeDynamicSink
obj | The object/proxy for which the property is registered , may be null.
ctx | The context for which the property is registered.

Remarks

If obj is non null then if it is a proxy, all calls made on the proxy are intercepted otherwise if it is a real object, all calls on the object are intercepted the ctx argument must be null

If ctx is non-null then the obj argument must be null and all calls entering and leaving the context are intercepted

If both ctx and obj are null then all calls entering and leaving all contexts are intercepted.

UnregisterDynamicProperty

The property may be unregistered, which implies that the sink will be dropped for subsequent remoting calls.

Unregister dynamic property. See RegisterDynamicProperty. Name, obj, ctx arguments should be exactly the same as a previous RegisterDynamicProperty call.

AllocateDataSlot

Allocates an un-named data slot. The slot is allocated on ALL the contexts.

AllocateNamedDataSlot

Allocates a named data slot. The slot is allocated on ALL the contexts. Named data slots are "public" and can be manipulated by anyone.

GetNamedDataSlot

Looks up a named data slot. If the name has not been used, a new slot is allocated. Named data slots are "public" and can be manipulated by anyone.

FreeNamedDataSlot

Frees a named data slot. The slot is allocated on ALL the contexts. Named data slots are "public" and can be manipulated by anyone.

GetData

Retrieves the value from the specified slot on the current context.

SetData

Sets the data in the specified slot on the current context.

ContextProperty Reference

The properties that make up the context are provided by various automatic services and are setup by Activation Services at the time the Context is created. Each property is itself a managed object. Depending upon whether properties contribute MessageSinks or not, we look upon them as being Active or Passive.

A ContextProperty is a name-value pair holding the property name and the object representing the property in a context. An array of these is returned by Context::GetContextProperties().

class ContextProperty
{
  public String Name {get;}
  public Object Property {get;}
};

Name

This read-only property returns the name of the context property.

Property

This read-only property returns the actual context property object.

CrossContextDelegate Reference

Delegate defined for context call backs. It is expected to be a function on an agile object.

delegate void CrossContextDelegate();

ContextAttribute Reference

ContextAttribute is the root for all context attributes. ContextAttribute provides the default implementations of IContextAttribute and IContextProperty. Simple Context properties can derived from ContextAttribute with the context attribute and the context property being in the class. For more specialized or more sophisticated needs the context attribute still derives from ContextAttribute and context property can be split into a separated class.

[AttributeUsage(AttributeTargets.Class)]
public class ContextAttribute
   : IContextAttribute, IContextPropery
{
   . . .
   // Default Context Attribute implementation
   . . .
};

IContextAttribute Reference

The current context that the thread is executing in and construction call message are passed to the attributes (attributes on the class, call site attributes) via IContextAttribute.IsContextOK. It must return & must return false if it wants to go out of context.

“Prototype” context properties are collected by asking the attributes for context properties.

IContextAttribute is exposed from all context attributes. It is used to determine whether a particular context is OK for the context attribute and returns a context property for a new context.

public interface IContextAttribute
{
  public bool IsContextOK(
                     Context ctx,
                     IConstructionCallMessage msg,
                  );

  public void GetPropertiesForNewContext(
                     IConstructionCallMessage msg);
}

IsContextOK

Returns whether the context parameter meets the context attribute’s requirements.

GetPropertiesForNewContext

The Attribute can add Context Properties directly to the __ContextProperties List in the IConstructionCallMessage. The context properties will be added to the new context. The default implementation in ContextAttribute will add this to the context property List. Context attributes are free to override GetPropertiesForNewContext and can implement their own behaviour e.g. add a new class that implements the context property to the List.

IContextProperty Reference

IContextProperty is exposed from all context properties. It is used to gather naming information from the context property and also to determine whether the new context is ok for the context property.

public interface IContextProperty
{
    public String Name  {get; };
    public boolean IsNewContextOK(Context newCtx);
    public void Freeze(Context newContext);
}

Name

Returns the name of the Context Property.

IsNewContextOK

Once all the context properties have been added to the new context, they are all asked whether they are OK in the new context. The context property could look at the other context properties from the parameter new context and determine whether it is ok coexisting with these other context properties. Returns false if not ok in the context.

Freeze

Called when the context is frozen.

IContributeClientContextSink Reference

The IContributeClientContextSink interface is implemented by context properties in a Context that wish to contribute an interception sink at the context boundary on the client end of a remoting call. The Client Context Chain is composed from those properties in the client context that implement IContributeClientContextSink interface and which to contribute a sink via the GetClientContextSink method call. This chain is cached for future use.

interface IContributeClientContextSink
{
  public IMessageSink GetClientContextSink(IMessageSink nextSink);
};

GetClientContextSink

Takes the first sink in the chain of sinks composed so far, chains its message sink in front of the chain formed and returns the composite sink chain.

IContributeServerContextSink Reference

The IContributeServerContextSink interface is implemented by context properties in a Context that wish to contribute an interception sink at the context boundary on the server end of a remoting call. The Server Context Chain is composed from those properties in the server context that implement IContributeServerContextSink interface and which contribute a sink via the GetServerContextSink method call. This chain is cached for future use. Thread Affinity could be an example of such a context property i.e. in this case the MessageSink would involve switching to a specific thread before processing a call further.

interface IContributeServerContextSink
{
  public IMessageSink GetServerContextSink(IMessageSink nextSink);
};

GetServerContextSink

Takes the first sink in the chain of sinks composed so far, chains its message sink in front of the chain formed and returns the composite sink chain.

IContributeObjectSink Reference

The IContributeObjectSink interface is implemented by context properties in a Context that wish to contribute an object specific interception sink on the server end of a remoting call. The Server Object Chain is composed from those context properties in the server object's context that implement IContributeObjectSink interface. [Just-In-Time activation would be an example of such a property i.e. the message sink would activate the object after the call arrives and then pass the message on to the object.]

Note: Server object sinks are per object i.e. the Context Property is free to add one sink per context-bound object.

interface IContributeObjectSink
{
  public IMessageSink GetObjectSink(
                  MarshalByRefObject obj,
                  IMessageSink nextSink);
};

GetObjectSink

Takes the object and the first sink in the chain of sinks composed so far, chains its message sink in front of the chain formed and returns the composite sink chain.

IContributeEnvoySink Reference

The IContributeEnvoySink interface is implemented by context properties in the Server Context that wish to contribute an envoy message sink on the client end. The Server Envoy Chain is composed from those context properties that implement IContributeEnvoySink interface. The Envoy Chain lives on the client end and acts as representative of the corresponding message sinks from the server context properties.

Note: Envoy object sinks are per object i.e. the Context Property is free to add one sink per context-bound object.

interface IContributeEnvoySink
{
  public IMessageSink GetEnvoySink (
                   MarshalByRefObject obj,
                   IMessageSink nextSink
                   );
};

GetEnvoySink

Takes the first sink in the chain of sinks composed so far, chain its message sink in front of the chain formed. Used as an optimization to create envoy sink when the destination is a different context in the same application domain and is used by the Wrap operation.

ContextStaticAttribute Reference

Used to mark static field to be relative to the context.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Remoting Configuration

© 2000 Microsoft Corporation. All rights reserved.


Remoting Configuration Concepts

Remoting Configuration allows for the various components of NGWS runtime Remoting to be specified external to the user code. This section detailed how Remoting configuration is used and the elements in the configuration file.

Application Startup, Remoting and Configuration

Key Points

  • Particular to Host & Channel
    • ASP+ HTTP Handler – lazy load on first inbound call
    • TCP Channel (any port) – active load – need to start listening
    • HTTP Channel (any port) – active load – need to start listening
    • 3rd Party Channel – depends on the channel
  • Read Configuration and Initialize Channels
  • Read Configuration and Register WellKnown Objects (WKO)

Read Configuration and Initialize Channels

Key Points

  • Locate Application Configuration File
  • Read Channels Configuration
  • For each channel:
  • Create a channel instance
  • Initialize from configuration
  • Register the Channel with ChannelServices

When an application is first started it performs a number of activities. One of these is to register the channels. The channel can use this opportunity to start to listen for requests on the protocols they were designed for. One of the aspects of registering the channels, is when a server object reference returned to the caller as the result of a method call, an ObjRef is created. The ObjRef is populated with the channel data of the registered channels. This allows the caller to make method calls on the remote object using the available server protocols if those protocols are also available on the caller.

Another aspects of registering the channels, is when a server object reference returned to the caller, the ObjRef is populated with the channel data of the remote applications registered channels. This allows the caller to make method calls on the remote application objects using the available remote application protocols if those protocols are also available on the caller.

Channels are registered via ChannelServices.RegisterChannel.

Read Configuration and Load WellKnown Objects (WKO)

Key Points

  • Locate Application Configuration File
  • Read WKO Configuration
  • For each WKO:
    • Load the assembly if it has not been load before
    • Load the type if it has not been loaded before
  • For each Singlecall WKO:
    • Register the type with the ObjectURI as a Singlecall
  • For each Singleton WKO:
    • Register the type with the ObjectURI as a Singleton

In order for incoming calls to reach the server objects, the inbound object identifier (URI) needs to be resolved to the real server object and the first message sink that can be used to communicate with the real server object. Remoting Services uses the entries in the identity table in order to find these items. Entries are place in the identity table when objects are registered (marshaled). Objects are registered (marshaled) when server object references are returned, as the result of a method call. Objects can also be marshaled for WellKnown objects.

Well known objects can use well known names (ObjectURI) and register these into the identity table. There are three types of well known objects:

  • Well known objects which are faulted in, created and marshaled / registered on first usage and are unregistered and discard after the call completes, and so live on for the duration of the call. These are known are Single call.
  • Well known objects which are faulted in, created and marshaled / registered on first usage and which live for a long time. These are known are Singeton
  • Well known objects which are created and marshaled / registered on the application started and which live for a long time. These are known are Singleton Registered

Well Known objects can only be published / marshaled in the appdomain in which they were created.

When context-bound objects are created a matching context is found or create that meet the context property requirements for the object. See the context section for further details.

When a server object is marshaled the server object sink chain, which leads to the object is built. The stack builder sink is the last sink in the server object chain and communicates with the server object directly. The stack builder sink is provided by the remoting infrastructure. When a server object that is context-bound is marshaled, the reference to the context in which the object was created is stored in the entry in the identity table.

When inbound calls arrive, they are dispatched on the server object. The identity table provides the URI to object, context and message sink chain mapping.


Remoting Configuration Reference

RemotingServices.ConfigureRemoting Reference

namespace System.Runtime.Remoting
{
  public class RemotingServices
  {
    public static void ConfigureRemoting(String fileName);
  }
}

ConfigureRemoting.

Read the configuration file and configure Remoting.

Remoting Configuration File Format

Assembly or Type

An Assembly or a type within can live in a remote application with respect to the current application.

Entity | Description | Example
RemoteApplication | The Application Name | MyRemoteApp1

Type

Entity | Description | Example
URI | If the URI is specified then Connect is used to obtain a proxy to a wellknown object when a new class is instanciated e.g “new SomeClass”. If a full URI (e.g. http://. . . ) is specified it is used, otherwise it is assumed the URI is relative to the URI for the Remote Application. | MyFoo=http://somehost/foo.soap or MyFoo=foo.soap

Remote Application Configuration

Applications can be remote with respect to the current application.

Entity | Description | Example
RemoteApplicationName | The Application Name | My401k
RemoteApplicationURI | The Application URI | http://www.contoso.com/My401k

Well Known Object (WKO) Configuration

Well Known objects can be published by an Application.

Entity | Description | Example
FulllTypeName | The full type name. | MyNamespace.MyType
AssemblyName | The Assembly Name | MyNamespace
ObjectURI | The Object URI on which to publish the object. | foo.soap
ObjectMode | SingleCall Singleton | SingleCall

Export

Export objects can be client activated.

Entity | Description | Example
AssemblyName | The Assembly Name | MyNamespace
FulllTypeName | The full type name. | MyNamespace.MyType

Channel Configuration

An Application registers channels.

Entity | Description | Example
ChannelFullTypeName | The channel type name | System.Remoting.Channels.TCP.TCPChannel
mimeType | The mime type the channel uses to encode messages. This specified in the Name=Value section for the channel. | text/soap or bin/bin

Transport specific Items

TCP Channel Configuration

Entity | Description | Example
ChannelTypeName | The channel type name | System.Remoting.Channels.TCP.TCPChannel
Ports | TCP Ports to listen on | 9090;9898
mimeType | The mime type the channel uses to encode messages. This specified in the Name=Value section for the channel. | text/soap or bin/bin

HTTP Channel Configuration

Entity | Description | Example
ChannelTypeName | The channel type name | System.Remoting.Channels.HTTP.HTTPChannel
Ports | TCP Ports to listen on | 8080;8081
Listener | Specified that another component is listening and dispatches HTTP messages into the HTTP Channel | ASPPlus
mimeType | The mime type the channel uses to encode messages. This specified in the Name=Value section for the channel. | text/soap or bin/bin

Lifetime

An Application registers channels.

Entity | Description | Example
LeaseTime | The default lease time for the application in seconds | 600
RenewOnCallTime | The default renew on call call time for the application in seconds. | 120
SponsorshipTimeout | The default sponsorship timeout for the application in seconds. | 120
LeaseManagerPollTime | The default lease manager poll time for the application in seconds. | 10

Remoting Configuration File Format

For PDC Remoting Configuration uses a compact file format.

--- Comments ------------------------------------------------------------------
- Comment lines begin with a “-“
--- Application Section  ------------------------------------------------------
Name#[ApplicationName]
--- Assembly Section  ---------------------------------------------------------
Assembly#[AssemblyName]#[RemoteApplicationName]#[FullTypeName[=URI]][;...]
--- RemoteApplication Section  ------------------------------------------------
RemoteApplication#[RemoteApplicationName]#[RemoteApplicationURI]
--- Channel Section  ----------------------------------------------------------
- Channel##[ChannelFullTypeName]#[Name=Value;Name=Value]
--- WellKnown Section --------------------------------------------------------
WellKnownObject#[FullTypeName]#[AssemblyName]#[ObjectURI]#[ObjectMode]
--- Export Section -----------------------------------------------------------
- These types can be client activated
Export#[AssemblyName]#[FullTypeName][;...]
--- Lifetime Section ---------------------------------------------------------
Lifetime#[Name=value;Name=value;Name=value]
--- End ----------------------------------------------------------------------

For Post PDC Remoting Configuration will be integrated into the standard XML Application Configuration file.

An application configuration file is an XML document that contains sections for various feature areas. The general format is a follows:

Application
   Feature+
      Feature Items*
      SubFeature*
         Sub feature items+

Example

<Application name="My401kApp">
   <aspplus:ASPPLUS xmlns:aspplus="http://www.microsoft.com/aspplus">
    . . .
   </aspplus:ASPPLUS>

   <urt:FusionBindings xmlns:urt="http://www.microsoft.com/urt">
   . . .
   </urt:FusionBindings>

  <urt:Assemblies xmlns:urt="http://www.microsoft.com/urt">
    <Assembly name="Calculator" version="5.0.0.0">
    <Assembly name="401kWizard" version="*">
    <Assembly name="401kTax" RemoteApplication="MyRemoteApp1" version="*">
        <Type name=EmployeePayrollRecord />
        <Type name=GovTaxRateTable URI=”GovTaxRateTable”/>
    </Assembly>
    <Assembly name="401kUtils" version="*">
      <Type name=Employee RemoteApplication="MyRemoteApp2"/>
      <Type name=Customer RemoteApplication="MyRemoteApp3"/>
    </Assembly>
    . . .
  </urt:Assembly>

  <urt:Remoting xmlns:urt="http://www.microsoft.com/urt">
    <RemoteApplications>
      <MyRemoteApp1>
          <RemoteApplicationName>My401ktax</RemoteApplicationName >
          <RemoteApplicationURI>
                      http://secure.hr.microsoft.com/My401ktax"
         </RemoteApplicationURI >
      </MyRemoteApp1>
      <MyRemoteApp2>
          <RemoteApplicationName> My401kEmployee"</ RemoteApplicationName >
          <RemoteApplicationURI>
                         http://host.microsoft.com/My401kEmployee"
          </RemoteApplicationURI >
      </MyRemoteApp2>
      <MyRemoteApp3>
          <RemoteApplicationName>My401kCustomer"</RemoteApplicationName>
          <RemoteApplicationURI>http://host.microsoft.com/My401kCustomer"</URI>
      </MyRemoteApp3>
    </RemoteApplications>

   <WellKnownObjects>
     <WellKnownObject>
        <FullTypeName>MyPackage.MyType1</FullTypeName >
       <AssemblyName> MyAssembly </AssemblyName>
       <ObjectURI> MyURIForMyType1 </ObjectURI>
      <ObjectMode> SingleCall </ObjectMode>
     </WellKnownObject>
     <WellKnownObject
       <FullTypeName> MyPackage.MyType2 </FullTypeName>
       <AssemblyName> MyAssembly </MyAssemblyName>
       <ObjectURI> MyURIForMyType2 </ObjectURI>
      <ObjectMode> Singleton </ObjectMode>
     </WellKnownObject>

   <Exports>
     <Export>
      <AssemblyName> MyNamespace </AssemblyName>
        <FullTypeName>MyPackage.MyType1</FullTypeName>
     </Export>
     <Export>
      <AssemblyName> MyNamespace </AssemblyName>
        <FullTypeName>MyPackage.MyType2</FullTypeName>
     </Export>
   </Exports>
   <Channels>
     <TCPChannel>
         <ChannelFullTypeName>
                 System.Remoting.Channels.TCP.TCPChannel
         </ChannelFullTypeName>
        <Ports> 5480;5340 </Ports>
         <MimeType>bin/bin</MimeType>
     </TCPChannel>
     <HTTPChannel>
        <Ports> listener=”System.Remoting.Channels.HTTP.ASPX.HttpListener”> 80 </Port>
       <Listener> ASPPlus</Listenen>
        <MimeType>text/xml</MimeType>
     </HTTPChannel>
    </Channels>
  </urt:Remoting>
</Application>


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Remoting Lifetime Overview

© 2000 Microsoft Corporation. All rights reserved.


Lease Based Lifetime Concepts

Distributed Garbage Collection (DGC) controls when server applications can be deleted. Traditionally DGC uses reference counts and pinging for control. This works well when there are a few clients per service, but doesn’t scale well when there are thousands of clients per service. A Leasing Distributed Garbage Collector (LDGC) associates a lease time with each service. The service is deleted when the lease’s time expires. The LDGC can subsume the function of a traditional DGC, in addition it scales well when the numbers of clients increase per server.

The Leasing Distributed Garbage Collector (LDGC) associates a lease with a remotely activated object. When the lease expires, the object is removed. A lease can specify that an object has an infinite lifetime. An infinite server is removed by being explicitly canceled by a client or administrator.

Lifetime Policies

A LDGC supports many policies in controlling the lifetime of an object. The following are typical lifetime policies:

Few long-lived remote objects with many clients

The remote object’s life is independent of the clients’ life. The lease specifies a long lifetime that can periodically be renewed by an administrator or a client. This is the most efficient use of leases in that there is very little network traffic needed for distributed garbage collection.

Many short lived remote objects with few clients

The remote object is not used very long, and needs to be removed as soon as its function is completed so as not to tie up scarce server machine resources. In this case the object has a lease with a short lifetime. The client frequently renews the lease with a short time span. When all the clients are finished with the remote object, the service is removed. This policy substitutes increased network traffic for more efficient use of the server machines resources.

Administrative Policies

A LDGC can be administered in many ways.

Central administrator. The central administrator establishes the initial value for the remote object’s lease.

Client administrator. The client specifies the initial lease time.

Sponsorship. A client sponsors the remote object by registering itself with LDGC. When the lease expires it asks one of its sponsoring clients for more time.

This administration policy is particular effective for parallel problem solving, in that the client can give multiple remote objects a problem to solve. When a remote object runs out of time, it asks the client for more. If the client already has the solution to its problem from another remote object, it doesn’t renew the object’s lease.

This policy is also useful for migrating remote objects, in that the remote object contacts the client from its current location rather then the client having to find it.

In the case where there are many clients per remote object, it is more efficient to have the remote object ask one of the clients for a lease renewal, then have all the clients pinging the remote object.

Design

Each remote object can have a lease that controls its lifetime. All leases in an Appdomain are connected to a Lease Manager. This object is the root object that keeps the leases and remote object alive. This object is also used to query and administer the leases in the appdomain. It determines when a lease’s time has expired, and if it has, informs the lease. The lease runs in the Lease Managers thread.

A lease contains its expiration time. A lease can be renewed in three ways.

  • The client calls the Leases Renewal method.
  • If the RenewOnCall property is set, then each call to the remote object renews the lease.
  • The lease calls back to a sponsor requesting a lease renewal.

Lease Manager

Each lease appdomain contains a Lease Manager that administers the leases in its domain. Among its tasks is to periodically examine the leases for time expiration. If a lease has expired it can either be cancelled by removing its reference to the lease, or renewed by invoking one or more of the lease’s sponsors. The lease manager has a Lease List with leases sorted in order of increasing time left. The leases with the least time will be on the top.

Lease

The lease consists of a collection of properties that determine its policies, and methods that renew the lease time. The lease exposes the ILease interface. The Lease contains the information that is detailed in the following sections.

Renewal Policies

RenewOnCall – Each time an invocation is made on the service, the lease time is set to the maximum of the current LeaseTime or the current time plus the RenewOnCall time. This value supports the client transparently setting the lease time. If the value is null, then the object will not be pinged on call.

SponsorshipTimeOut – When the LeaseTime is elapses a sponsor is asked to renew the lease. The sponsorshipTimeout specifies the amount of time to wait for a sponsor to reply. If the value is null, then the lease will not take sponsors.

CurrentLeaseTime – The time that the lease will expire. If the value is zero, the lease will not expire and will be infinite. Specifies that this is an infinite lease that will not expire. Shutdown of the service occurs with the use of the Cancel method. This value is set if no lease time is given in the constructor.

Sponsors

SponsorList – A list of the sponsors ISponsor callback object sorted in order of decreasing sponsorship time. When a sponsor is needed for renewing the lease’s time, one or more sponsors from the top of the list will be asked to renew the time. The top of the list represents the sponsor that previously requested the largest lease renewal time. If a sponsor doesn’t respond in by the SponsorshipTimeOut time span, it will be removed from the list.

Links to lease manager and managed object

LeaseManager – The leaseManager object.

ManagedObject – Object which lease manages.

State

State – The current state of the lease.

ILease properties

LeaseTime - Initial time for lease. If leaseTime is null, then isInfinite is set to true.

RenewOnCall – The amount of time that a call to the remote object increases the leaseTime. If RenewOnCall is not null, then a sync is created to renew the lease by the RenewOnCall amount on each call.

SponsorTimeOut – The about of time to wait for a sponsor to return with a lease renewal time. If the sponsorshipTimeOut is null, then this lease will not take sponsors.

ILease Methods

Register – registers the sponsor’s callback object with the lease. A renewal time can also be sent with the Register method.

UnRegister – removes a sponsor from the lease.

Renew – This method is used to renew a lease. The lease time is set to the maximum of the current leaseTime or the current time plus the renewal time. The leases expiration time is returned to the caller.

Obtain the lease

The lease has various methods that can be called either locally or remotely. The lease is obtained by calling the static method RemotingServices. GetLifetimeService..

public class RemotingServices
{
  public static Object GetLifetimeService(MarshalByRefObject obj);
}

The GetLifetimeService method parameter is the reference to the object.

  • If the object is local, the parameter is the local reference to the object, and the Lease object returned is the local reference to the Lease.
  • If the object is remote, the parameter is the transparent proxy to the object, and the Lease object returned is the transparent proxy to the Lease.

Object providing their own leases

Object can provide their own lease and so control their own lifetime. They do this by overriding the InitializeLifetimeService method provided on MarshalByRefObject.

public class Foo : MarshalByRefObject
{
  public override Object InitializeLifetimeService()
  {
   ILease lease = (ILease)base.InitializeLifetimeService();
   if (lease.CurrentState == LeaseState.Initial)
   {
       lease.InitialLeaseTime = TimeSpan.FromMinutes(1);
       lease.SponsorshipTimeout = TimeSpan.FromMinutes(2);
        lease.RenewOnCallTime = TimeSpan.FromSeconds(2);
   }
     return lease;
  }
}

Property sets will only work when a lease is in the initial state. A lease is in the initial state which it is created by InitializeLifetimeService. It is placed in the Active state when the object is marshaled.

When the ILease lease = (ILease)base.InitializeLifetimeService(); is called within InitializeLifetimeService, either an existing lease is returned for this object, or if no lease exists, a new lease is returned. Only if a new lease is returned can the lease properties be set. An exception will occur if an attempt is made to set an existing lease. A new lease has the state LeaseState.Initial, an existing lease has the state LeaseState.Active or LeaseState.Expired.

The only call that counts for the Lifetime service is the call to InitializeLifetimeService from the Remoting Lifetime Lease infrastructure (implemented as an AppDomain property), this the call that acivates the lease. Someone else can call InitializeLifetimeService and create a lease, but that lease will stay in its initial state until it is returned to the Remoting Lease Infrastructure. An existing lease that is not in the initial state cannot be set with new values, although it can be returned to Remoting Lifetime Lease infrastructure for use use by the new sink for RenewOnCall when an invocation on the object occurs. This allows multiple sinks for an object to point to the same lease.

No lease will be created if the leasetime is zero or a null lease is returned lease.InitialLeaseTime = TimeSpan.Zero; or return null. No sink will be created if the RenewOnCallTime is Zero, but a lease will be created. lease.RenewOnCallTime = TimeSpan.Zero;

Initializing a Lease

A lease is created when activating an object. The values for fields’ RenewOnCall, sponsorship, and leaseTime are supplied in the lease’s constructor. These values can come from multiple places:

  • In the Activation call.
  • The Configuration database.
  • The Object’s attributes.

The above list also gives the order in which the values are found.

The Activation call can also supply a list of sponsors for the lease. Additional sponsors can be added any time the lease is executing.

The leaseTime can be extended while the object’s clients are executing.

  • The client can directly invoke the Lease Class Renew method.
  • The lease can request a Renewal from a sponsor.
  • When a client invokes a method on the object, the lease will be renewed by the RenewOnCall value.

Explicit client Lease Renewal (Renew)

The client of the remote object can explicitly renew the lease. This is done as follows:

  • The client gets a reference to the lease by calling RemoteServices.GetLease(obj), where obj is a reference to the remote object.
  • RemoteServices locates the LeaseManager object in the AppDomain of the remote object. From the LeaseManager the lease for the object can be located.
  • A transparent proxy to the remote reference is returned to the client.
  • The client invokes the Renew method on the lease.

Implicit client Lease Renewal (RenewOnCall)

A client can implicitly renew a lease each time a method on the object is invoked. This is done as follows:

Initializing the object

  • During activation, when the lease is created, if the RenewOnCall’s value is not null, a RenewOnCall ContextProperty is added to the context.
  • Activation creates the object and marshals it.
  • Marshal adds a RenewOnCall sync to the helix chain using the RenewOnCall context property.

Calling the object

  • A message is sent from the client to the server.
  • Before the remote object is invoked, the RenewOnCall helix sync updates the lease time using the value in the lease’s RenewOnCall field.
  • The remote object is invoked.

Explicit Remote Object Renewal Request using Sponsorship

A Lease on a remote object can explicitly ask a sponsor to renew a lease. This is done as follows:

Registering a Sponsor

The sponsor is registered during activation or

The sponsor registers itself

  • The client gets a reference to the lease by calling RemoteServices.GetLease(obj), where obj is a reference to the remote object.
  • RemoteServices locates the LeaseManager object in the AppDomain of the remote object. From the LeaseManager the lease for the object can be located.
  • The client invokes the register method on the lease. The client specifies the sponsor call back object as a parameter of the registration method.

Invoking the sponsor

The Lease Manager has a thread that periodically examines all the leases it manages. When a leases time has expired and the lease has sponsors it performs the following:

  • It determines which sponsors to ask for time renewal. The sponsor list is maintained in decreasing order of their last renewal time. The sponsors with the highest last lease renewal time are first on the list.
  • It asks the chosen sponsor to renew the lease by invoking the Renewal method on the ISponsor interface for the sponsor. The sponsor is actually a transparent proxy on a remote object.
  • The amount of time the sponsor has to renew the lease is specified in the sponsorshipTimeOut Lease field. If a sponsor hasn’t returned with the renewal time by the sponsorshipTimeOut TimeSpan, it is removed from the sponsor list.
  • If a sponsor returns with a renewal time, the lease is renewed. If no sponsor returns with a renewal time, the lease state goes to Expired.

Terminating the Object

When a lease is in the Expired state, the LeaseManager will terminate it. The lease’s reference is removed from the LeaseManager and also the object reference is removed from the ID table. The Garbage Collector will then remove the lease, the object, and the remoting pipeline.

Once a lease is in the Expired state, no further lease messages or sponsor returns will be accepted.

Reachability of Sponsors

In order for the remote object to call back to the sponsor, it must be reachable by the remote object. The Web and firewalls can make it difficult to establish a client/server relationship from the remote object back to the client. The sponsor doesn’t need to be in the same location as the client, so it can be in a part of the network reachable by the remote object.

Automatic Lease Renewal (Not in PDC)

A possible renewal policy is to have a renewal agent rather then the client renew the lease. The client can specify a policy to keep alive the remote object for as long as the client is alive. The renewal agent will periodically renew the lease on behalf of the client.

Automatic Lease Sponsorship (Not in PDC)

A possible sponsorship policy is to have a sponsor agent automatically supply sponsors for one or more clients. The sponsor will renew the lease when asked until the clients are no longer alive.

Performance Improvements over DCOM

DCOM uses reference counting and pinging to keep a remote object alive. LDGC can support higher performance then DCOM because of the following:

  • RenewOnCall occurs on method calls to objects. DCOM requires a separate ping. The RenewOnCall renewal time is adjustable. A DCOM remote object is pinged by a client every 2 minutes, and will be collected if it doesn’t receive a ping in 6 minutes.
  • LeaseTime and lease renewal replace reference counting. Reference counting supports quick cleanup of remote objects. It is complex, in that when a reference is passed to another object, the reference count at the remote object must me incremented. It is also unreliable in that a network connection could go down, which would leave a count for a client that has disappeared. This is why ping is also needed with reference counting.
  • A lease possibly looses some efficiency in the quick collection of remote objects, but it gains efficiency in reducing the amount of administrative network traffic devoted to reference counting and pinging.
  • A lease separates the lifetime of a remote object from the lifetime of its clients. Thus it supports two scenarios: the remote object as a service, and the remote object as a part of an executing distributed application.

Cycle Elimination (Not in PDC)

LDGC as currently described will not detect cycles. That is the clients and sponsors will renew the lease, but none of the clients or sponsors are rooted in an actively executing application. With the help of the garbage collector at each network node, cycle detection can be added to LDGC.


Lease Based Lifetime Reference

MarshalByRefObject Reference

MarshalByRefObject provides some methods to work with lifetime.

public MarshalByObject
{
  public Object GetLifetimeService();
  public virtual Object InitializeLifetimeService();
}

GetLifetimeService

This method is used return a lifetime service object that is used to control the lifetime policy to the object. For the default Lifetime service this will be an object of type ILease.

Return Object to control lifetime Service.

InitializeLifetimeService

This method is used return lifetime service object. This method can be overridden to return a LifetimeService object with properties unique to this object. For the default Lifetime service this will be an object of type ILease.

Return: Object to control lifetime Serivice

ILease Reference

The ILease interface contains the publicly callable methods in the Lease class.

public interface ILease
{
    void Register(ISponsor obj, TimeSpan renewalTime);
    void Register(ISponsor obj);
    void UnRegister(ISponsor obj);
    TimeSpan Renew(TimeSpan renewalTime);

    TimeSpan RenewOnCallTime {get; set;}
    TimeSpan SponsorshipTimeout {get; set;}
    TimeSpan InitialLeaseTime {get; set;}
    TimeSpan CurrentLeaseTime {get;}
    LeaseState CurrentState {get;}
}

Register

Register a sponsor for the lease with a renewal time.

Register

Register a sponsor for the lease with no renewal time.

UnRegister

Unregister a sponsor for the leae.

Renew

Renew the lease for the specified time.

RenewOnCallTime

The renew on call time for the lease.

SponsorshipTimeout

The sponsorship timeout for the lease.

InitialLeaseTime

The initial lease time.

CurrentLeaseTime

The current lease time.

CurrentState

The current lease state.

ISponsor Reference

Each sponsor supports the ISponsor interface. It is used by the LDGC to call back to the Sponsor.

interface ISponsor
{
   TimeSpan Renewal();
}

Renewal

The LDGC asks a sponsoring client to renew the lease for the specified service. The sponsor returns the additional lease time.

LeaseState Enum Reference

Lease States.

public enum LeaseState
{
  Null = 0,
  Initial = 1,
  Active = 2,
  Renewing = 3,
  Expired = 4,
}

Null

The lease is not initialized.

Initial

Lease has been created, but not yet active.

Active

The lease is active and has not expired.

Renewing

Lease has expired and is looking for sponsorship.

Expired

Lease has expired and cannot be renewed.

LifetimeServices Reference

Used to set and get lifetime settings. Time set for an AppDomain. They must be set before the first lease is created (the first object is marshaled which creates a lease).

public class LifetimeServices
{
    public static TimeSpan LeaseTime {get;set}
    public static TimeSpan RenewOnCallTime {get;set}
    public static TimeSpan SponsorshipTimeout {get;set}
    public static TimeSpan LeaseManagerPollTime {get;set}
}

LeaseTime

Initial Lease Time span for appdomain. Default 5 minutes.

RenewOnCallTime

Initial renew on call time span for appdomain. Default 2 minutes.

SponsorshipTimeout

Initial sponsorshiptimeout for appdomain. Default 2 minutes.

LeaseManagedPollTime

Initial sponsorshiptimeout for appdomain. Default 10 seconds.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Remoting MetaData

© 2000 Microsoft Corporation. All rights reserved.


Remoting Metadata Concepts

MetadataServices Concepts

Key Points

  • Discovery
  • HTTP GET on URL returns Generated XML Schema from Runtime MetaData via Remoting MetaDataServices (SUDS)
  • Remoting MetaData converts XML Schema into MetaData via CodeGen or Reflection Emit
  • Full fidelity of Runtime Meta into XML
    • Class hierarchy, Interfaces
    • Fields, Methods, Properties
    • Delegates

Metadata Services can generate and consume XML Schema to and from NGWS runtime Runtime Metadata.

  • Transform MetaData into XML Schema
  • Transform XML Schema into MetaData using CodeGen
  • Transform XML Schema into MetaData using Reflection Emit

Abstract Assembly Concepts

  1. The contract for remoting is a TYPE (e.g., not a methodref...). A VOS type is an interface or a class.
  2. In making a remote reference, the client side must have the full remoting contract. Having the full remoting contract means:
    • if the type is a concrete class... full metadata plus code
    • if the type is an abstract class... full metadata
    • if the type is an interface... full metadata
  3. The assembly design in place today still holds:
    • types are deployed in assemblies
    • references to types are scoped by assembly
    • remoting information is assembly configuration metadata
  4. Could design the client-side remoting contract in one of several ways (not exclusive):
    • Opt1: it's just the server-side assembly... single contract; this would mean the full server-side assembly (incl code) would need to exist on client machine ("assembly light" may be a V2 feature, but not V1)
    • Opt2: developer of remote server could elect to define a collection of interfaces/abstract types specifically to serve as the client-side contract, deployed in one/more assemblies, and then to inherit from / implement them in the server; this way, client side needs no code
  5. Could deploy the client-side remoting contract in one of several ways (not exclusive):
    • Opt1: deploy to client
    • Opt2: deploy to server; "where" name to assembly causes runtime/fusion to locate client-side remoting contract assembly and download/cache
  6. Observe:
    • There's nothing special about a "client-side remoting contract assembly"... it's just an assembly.
    • This is "better" than a remoting proxy because the "c-s r c a" is actually used by both client and server side.


Remoting Metadata Reference

Remoting MetadataServices Reference

public class MetaDataServices
{
  public static void GenerateSchema(Type[] types, Stream output);
  public static void GenerateCode(Steam input, Stream output);
  public static void GenerateMetadata(Steam input, Stream output);
}

GenerateSchema

Transform MetaData into XML Schema

GenerateCode

Transform XML Schema into MetaData using CodeGen

GenerateMetadata

Transform XML Schema into MetaData using Reflection Emit


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Remoting Security

© 2000 Microsoft Corporation. All rights reserved.


Remoting Security Concepts

Related Topics

ASP+ Security

Role based Security

HTTP Channel WebSecurity

WebSecurity is based on the Internet standards for secure communication and authentication. The HTTP Channel utilizes System.Net on the sender and IIS with ASP+ for security on the receiver to provide Web Security. The credentials (Username, Password and optionally Domain) can be set on the HTTP Channel Message Sink Properties. The Remoting infrastructure passes the credentials to System.Net, which sends them to the server. HTTP The Channel Message Sink properties contain a PreAuthenticate property that by default are will send the credentials on the first message.

Authentication

Applications hosted in ASP+ can be configured for authentication using the combination of IIS and ASP+ security. The ASP+ Application vroot e.g. http://contoso.com/Cart is configured for security in IIS and ASP+. See the IIS documentation and ASP+ Security topics for details on configuring IIS and ASP+ for authentication.

Anonymous

Allows users to establish an anonymous connection, select this option. Your server logs on the user with an anonymous or guest account.

Note: You must turn off Anonymous if you want to use basic, digest or integrated windows authentication.

Basic Authentication

Basic authentication is a security mechanism using a standard HTTP mechanism in which user information is sent and received as clear text. It uses a base64 encoded string that contains the user name and password. (The term clear text indicates data is transmitted as clearly readable text characters rather than binary bit-streamed information, which is not text-formatted.) Passwords and user names are encoded but not encrypted in this type of authentication. It is important to note that Basic authentication results in the transmission of passwords across the network in an unencrypted form. A determined computer vandal equipped with a network monitoring tool could intercept user names and passwords.

Basic Authentication can be useful when the Application just needs some identifier for the calling Application.

Digest Authentication

Digest authentication sends a hash value over the network rather than the password. This method works across proxy servers and other firewalls.

A challenge-response scheme that challenges using a nonce (a server-specified data string) value. A valid response contains a checksum of the user name, the password, the given nonce value, the HTTP method, and the requested URI.

Digest authentication addresses many of the weaknesses of Basic authentication. Most notably, the password is not in clear text when you use Digest authentication. In addition, Digest authentication can work through proxy servers, unlike integrated Windows authentication.

Digest authentication is useful when the Application needs a stronger level of authentication than basic.

Cookie Authentication.

Cookie authentication is generally used to refer to a system whereby unauthenticated requests are redirected to an HTML form (using HTTP client-side redirection). The user provides credentials and submits the forms. If the application authenticates the request, the system issues a cookie that contains the credentials in some form or a key for reacquiring the identity. Subsequent requests are issued with the cookie in the request headers and they are authenticated and authorized by an ASP+ handler using whatever validation method the application desires.

It should be noted that cookie authentication is often used for personalization, where content is customized for a known user. In some of these cases, identification is the issue rather than authentication, so it’s enough merely to store the user name (in a durable cookie) and use that to access the user personalization information. For more information on ASP+ personalization services, please refer to the User Profiles specification.

Integrated Windows Authentication

Integrated Windows authentication uses a cryptographic exchange with the user’s Internet Explorer Web.

Integrated Windows authentication—formerly known as both NT LAN Manager (or NTLM) and Windows NT Challenge/Response authentication—is more secure than Basic authentication. This authentication scheme works especially well in an intranet environment where users have Windows domain accounts.

In integrated Windows authentication, the browser attempts to use the current user’s credentials from a domain logon. If those credentials are rejected, integrated Windows authentication will prompt the user for a user name and password by means of a dialog box. When integrated Windows authentication is used, the user’s password is not passed from the client to the server. If a user has logged on as a domain user on a local computer, the user won’t have to be authenticated again when accessing a network computer in that domain.

The user is not prompted for a user name and password for each HTTP request; rather, this will happen only when the cached credentials do not have sufficient permissions to access a specific page or file.

Role-based Security in Managed Applications

Role-based Security is a common application concept in which one makes Authorization decisions based on the identity and/or role(s) for the entity on whose behalf an application is executing. The definition of “roles” is typically application specific as are the conditions under which one is willing to authorize specific actions.

It is common for applications developers to make authorization decisions based on the identity and/or role(s) associated with the execution context. One often sees this in financial or business systems as a means of policy enforcement. For example, one may impose limits on the value of a transaction depending on the role of the user making the request. Clerks may be able to process transactions up to some set value, supervisors to some higher limit, and VPs yet a higher limit. Another example is business systems requiring multiple approvals to complete an action. Consider a purchasing system in which any employee may generate a purchase request, but only a purchasing agent may process that request into a Purchase Order that can be sent to the supplier.

In the past, such requirements were often handled using proprietary mechanisms. Microsoft first introduced infrastructure to support this type of “role-based” security with MTS and recently COM+1.0 services. The NGWS runtime provides similar support but does so in a way that is well integrated into the NGWS programming model and scales to the needs of distributed internet applications.

Integration with the NGWS programming model means that role-based security mechanisms:

  • Provide the ability to check the identity and/or role of the entity on whose behalf code is executing
  • Integrate well with distributed applications using NGWS runtime object remoting
  • Work consistently in both client and server execution environments
  • Expose these services to app developers via mechanisms similar to Code Access Security checks
  • Are ubiquitous, and always available
  • Are easily configured to use application defined roles

Impersonation

ASP+ Security

Windows Identity Impersonation

WindowsIdentity.Impersonate

ImpersonationContext.Undo

Secure Communication in ASP+

The HTTP Channel hosted in IIS with ASP+ supports sending and receiving secure communication using SSL (Secure Sockets Layer). See the IIS documentation for configuring SSL. Start the URLs with https in calls to Activator.GetObject, Activator.CreateInstance, items in the Remoting Configuration file or wherever URLs are used to specify a remote object, remote WebService or Remoting application.

ASP+ HttpRequest.IsSecureConnection in the HttpContext returns a boolean that indicates whether the Http connection is secure i.e. HTTPS.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Remoting Architecture

© 2000 Microsoft Corporation. All rights reserved.


Remoting High level Overview

The Runtime provides an environment where objects can make method calls and access properties and fields on other objects. Generally speaking, when the caller and callee object instances are both within the same boundary, method calls occur directly from the caller instance to the callee instance. When method calls occur between objects instances across a boundary, Base Remoting is used. These boundaries include calling between: contexts, application domains, processes and machines.

Figure 1: Contexts and Base Remoting Infrastructure Model.

Runtime types (classes) can have methods, properties and fields. The purpose of Base Remoting is to present the illusion that what is being called locally has the same type (class) as the remote instance. The application can be deployed and configured by an administrator to run local or remote.

Remoting stages

Let us discuss by way of an example how an operation on a transparent proxy is forwarded to the remote object it represents. For the purposes of discussion below we will assume that the “client” object obtained a transparent proxy, by some means, to a “server” object living in a different process on the same machine.

Figure 1: Remoting stages: outlines various stages involved in processing a call made by a client on a proxy and forwarding it to the server object.

Packaging stage

The transparent proxy has two method tables: a forwarding method table that the client’s calls are routed into and the Proxy object’s method table, which is just the regular method table for any managed object.

When a client invokes a method on transparent proxy, control reaches a forwarding thunk present in the forwarding method table. Utilizing the method signature, the forwarding thunk captures the stack frame representing the call, encapsulates it inside a Message object and delegates to the message sink stored inside the proxy object by passing it the message. If the method represents an asynchronous call a reply sink object is created and is also passed. If the method represents a one-way message, the reply sink will not be created or passed.

When the reply message arrives, the return and out parameters are propagated from the reply message to the client memory locations and the call is marked as complete.

Processing stage

Message sinks are responsible for carrying message object from their source to the destination. The message object is serialized for transportation across the boundary via a channel. This may utilize object Formatters to serialize object references inside message object depending upon how far the message is going from a remoting standpoint. The Formatters are pluggable into the channel. For example a HTTP channel can have many formatters plugged in (e.g. XML Formatter, Binary Formatter etc). These same Formatters can be plugged into another channel (e.g. TCP Channel).

Message sinks processing one-way messages may return the client thread before completely dispatching the message.

When the reply message arrives on the reply sink, return/out parameters are propagated to the client memory locations. Object references are serialized / deserialized as needed.

Dispatch stage

A dispatcher / stack builder object that appears as a message sink to its clients, accomplishes this stage. It sets up an exception handler and invokes the method on the server object. It then calls the reply sink supplied by its caller by passing it the reply message generated as a result of the call invocation. If something goes wrong, the reply message will contain information about the Exception.


Architecture

Figure 2: Contexts and Base Remoting Infrastructure Architecture.

Server side startup

When the server application is first started it performs a number of activities. One of these is to register the server channels. The server channel can use this opportunity to start to listen for requests on the protocols they were designed for. One of the aspects of registering the server channels, is when a server object reference returned to the caller as the result of a method call, an ObjRef is created. The ObjRef is populated with the channel data of the registered channels. This allows the caller to make method calls on the server object using the available server protocols if those protocols are also available on the caller. Channels are registered via ChannelServices.RegisterChannel.

In order for incoming calls to reach the server objects, the inbound object identifier (URI) needs to be resolved to the real server object and the first message sink that can be used to communicate with the real server object. Remoting Services uses the entries in the identity table in order to find these items. Entries are place in the identity table when objects are marshaled. Objects are marshaled when server object references are returned, as the result of a method call. Objects can also be marshaled for well known objects.

Well known objects can use well known names and register these into the identity table.

There are three types of well known objects:

  • Well known objects which are created and marshaled / registered on the application started and which live for a long time.
  • Well known objects which are faulted in, created and marshaled / registered on first usage and which live for a long time]
  • Well known objects which are faulted in, created and marshaled / registered on first usage and are unregistered and discard after the call completes, and so live on for the duration of the call.

When context-bound objects are created a matching context is found or create that meet the context property requirements for the object. See the context section for further details.

When a server object is marshaled the server object sink chain, which leads to the object is built. The stack builder sink is the last sink in the server object chain and communicates with the server object directly. The stack builder sink is provided by the remoting infrastructure. When a server object that is context-bound is marshaled, the reference to the context in which the object was created is stored in the entry in the identity table.

When inbound calls arrive, they are dispatched on the server object. The identity table provides the URI to object, context and message sink chain mapping.

Client side startup

The client application is first started it performs a number of activities. One of these is to register the client channels. One of the aspects of registering the client channels, is when a server object reference returned to the caller as the result of a method call, an ObjRef is created. The ObjRef is populated with the channel data of the registered channels. This allows the caller to make method calls on the server object using the available server protocols if those protocols are also available on the caller. Channels are registered via ChannelServices.RegisterChannel.

Proxies

The Transparent proxy provides the same class / type by exposing the remote object instance methods and fields. It forwards calls on the proxy to the real remote object instance. The caller is provided with the illusion that the remote instance is local. An explicit check can still be made to determine if the object reference is to a proxy by calling System.Remoting.IsTransparentProxy and passing in the object reference. See the System.Remoting package for details.

Client side Explicit – GetObject

A client obtains a proxy by making an explicit call to Connect. This is shown in the code fragment below:

Foo oFoo = (Foo)Activator.GetObject(
                             typeof(Foo),
                             ”http://store/cart/Foo”);
int ret = Foo.Baz(1, 2.3, “Zap”);

Connect uses the URL to determine whether a proxy already exists for the URL endpoint by performing a lookup in the client side identity table and using the URI (the non channel specific portion of the URL) as the key. If the proxy already exists it is returned to the caller. If no proxy exists then one is created. Creating a proxy involves finding a channel, which supports delivering messages to the URL endpoint. This resolution is performed through ChannelServices and the IChannel interface on the channel. If the channel determines that it is capable of delivering messages it will return a message sink to RemotingServices. RemotingServices then creates the proxy using the metadata of the runtime type / class. The proxy holds references to the first envoy sink chain and channel message sink.

When calls are made on the proxy, the last sink in the envoy chain (which is an infrastructure provided sink) will find the current context on the current thread and call through the client context sink chain.

The last sink the client context sink chain will use the identity table and call the channel sink. The client context sink chain is build by RemotingServices from the context properties of the caller objects context.

Marshaling method parameters

When a method call occurs on the transparent proxy, a message object is generated by the proxy and passed to the first sink in the message sink chain. The message object provides access to the call parameters, method signature, method name, and the destination URI. Access is provided via the IMessage and IMethodCallMessage interfaces. The stack is walked to provide the call parameters.

Message Sink

Message Sinks provide an extensibility mechanism for base remoting. The Message Channel, which delivers messages to the destination, appears as a Message Sink and exposes the IMessageSink interface. Message Sinks can forward the incoming message to another message sink, transform the message or generate a new message. Examples of Messages Sinks are Message Channels, Stack Builder sinks, Debugging sinks, Security sinks and Synchronization sinks. Message Sinks can be composed into a Message Sink chain, with messages being passed through the Message Sink chain.

Message Channel

The source and target Message Channels provide a virtual Message Channel between the source and target. The virtual Message Channel receives messages into the source Channel and delivers the messages into target Message Channel. The virtual Message Channel serializes the message into an appropriate form when transmitting from the source to the target.

Serializing Messages

Serialization is the process of converting a graph of objects, located in memory, into a linear sequence of bytes. That sequence of bytes can be sent elsewhere (for example, to a remote computer) and deserialized, thereby making an exact clone in that remote memory of the original graph of objects. The NGWS runtime Serialization services are located in the System.Serialization package.

Dispatching method calls on to the target object

The target Channel deserializes the message into a Message object. It performs a lookup in the Identity table using the URI in the Message object to find the target object and the first Message Sink in the target Message Sink chain. The Message Channel sends the message to the first Message Sink in the chain. The last Message Sink in the chain is the Stack Builder sink. The Stack Builder sink converts the Message Object into a stack frame and makes the method call on the target object. When the method call returns, the stack has the return values from the call.

Sending returning values back to the caller

The Stack Builder sink converts return values into a message and sends it to the first message sink in the reply sink chain. The last sink in the reply sink chain is the target channel. The reply message is serialized and returned back to the source channel.

Propagating return values back onto the original stack

The source Message channel deserializes the reply message into a Message object and sends the message to the next reply Message Sink in the chain. The last Message Sink in the reply sink chain propagates the return values to the caller stack. [Implementation note: the last Message Sink in the reply sink chain is the original message created by the proxy, which was sent to the first message sink when the call started. The call unwinds and the proxy returns to the caller.]


Remoting Feature Summary

Remoting provide two levels:

  1. Send or Receive Messages to or from endpoints on particular transports e.g. XML Documents over HTTP e.g. SOAP, binary over high speed TCP.
  2. Distributed Object Model, Distributed Identity, Activation, Lifetime. (built on top of 1).


Types and Objects

Key Points

NGWS Messages
  1. Type Based
    • Preserves and Extends the base Runtime Object Model
    • MetaData and Language support
  1. Objects live within a Remoting Boundary i.e. AppDomain / Context
  1. Object passed across a Remoting Boundary can be Marshal By Value - make a copy when passed across an AppDomain boundary
  1. Objects passed across a Remoting Boundary can be Marshal By Reference - hand out a proxy – transported as an OBJREF. The object remains in the AppDomain in which it was created
  1. Value Types passed across a Remoting Boundary are marshal by value
  1. Public Instance Methods are executed remote (virtual and non virtual)
  1. Public Instance Fields access are remote
  1. Public Instance Properties access are remote
  1. Public Static Methods are executed local
  1. Public Static Fields access are local
  1. Public Static Properties access are local
  1. Interfaces
  1. Parameters
    • In, Out, ByRef (In/Out), VarArgs
  1. Parameter Types
    • Primitives (Int, Float, String etc)
    • Value Types
    • Objects with embedded object references (graphs of Objects)
    • Typed References
  1. Return Types
    • Primitives (Int, Float, String etc)
    • Value Types
    • Objects
    • Typed References


Calls, Parameter Capture, Dispatch and Return

Key Points

NGWS Messages
  1. Capture support - RemotingPorxy
    • In EE Infrastructure
    • Generated at Execution Time from Type MetaData
    • Message wraps Stack Frame of parameter
    • Delayed parameter access until message serialized in channel
    • ProvidesType Safety from MetaData
    • MetaData can be in own PE (DLL) or external PE (DLL)
    • Build your own proxy - RealProxy extensibility
  1. Dispatch support
    • In EE Infrastructure
    • Transform Message into a Stack Frame and dispatch on Object (StackBuilder)
  1. Dispatch support
    • Stack, In EE
    • Transform Message into a Stack and dispatch on Object (StackBuilder)
    • Public Instance Method
    • Public Instance Property
    • Public Instance Field
  1. Messages
    • Access to parameters
    • Access to call details (method name, signature)
    • Can be used to base out of band info
    • Dictionary access to entries
    • Fast path property access to well known entries
  1. Interception – Message Sinks
    • Access to the request call message
    • Access to the response return message
    • Synchronous path
    • Asynchronous path
  1. Interception – Dynamic Properties
    • Add and remove at execution time
    • Between Proxy and Channel
    • Between Dispatch (StackBuilder) and Real Object
    • Outbound from AppDomain
    • Inbound into AppDomain
    • Outbound from Context
    • Inbound into Context


Extensibility

Key Points

NGWS Messages
  1. Envoys - server context property can project a client side envoy which can add items in the message which can be picked up by the server sinks
  1. 3rd party transports – channels
  1. 3rd party formatters
  1. 3rd party context attributes and properties
  1. 3rd party proxies


Registration and Resolution

Key Points

NGWS Messages
  1. Activator.GetObject (RS.Connect)
    • Client 1st - a local representative (proxy) is provided immediately
    • Messages sent only when 1st method call occurs.
    • Optionally set Properties via IDictionary
  2. To any well known object:
    • Faultin Singlecall
    • Faultin Singleton
    • Registered Singleton
  1. Activator.CreateInstance (was Type.CreateInstance)
    • Optionally pass in URL at callsite
    • Optionally Look up URL in Application Configuration file
    • Create a new instance of the object on Server
    • Once the object has been created on the server, a local representative (proxy) is provided.
  1. IL newobj instruction
    • Look up in Application Configuration file for URL
    • Create a new instance of the object on Server
    • Once the object has been created on the server, a local proxy is provided.
  1. Compiler support for CallSite attributes
    • Optionally pass in URL at callsite
    • Optionally Look up URL in Application Configuration file
    • Create a new instance of the object on Server
    • Once the object has been created on the server, a local representative (proxy) is provided.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Communication Patterns

Key Points

NGWS Messages
  1. Request Response
  1. Oneway


Communication Style

Key Points

NGWS Messages
  1. Synchronous
  1. Asynchronous on Client – delegate BeginInvoke and EndInvoke, IAsyncResult, AsyncCallback


Hosting

Key Points

NGWS Messages
  1. DLL Applications
    • Hosted inside ASP+ “DLLHOST equiv”
  1. EXE Applications - Launched from Command Line
    • “Manual” – start channels to listen
    • “Automatic” (post PDC)


Activation

Key Points

NGWS Messages
  1. Objects are activated in an Application
  2. Application and URL / Machine can be specified by callsite
  3. Application and URL / Machine can be specified inside Local Configuration File for a given Type
  4. Each AppDomain exposes a well known Activator which can activator a specified type in its AppDomain
  5. The Application Property which participates in activated reads the configuration file and replaces the local Activator with the Activator in the Remote AppDomain


Remoting Lifetime

Key Points

NGWS Messages
  1. Well Known - Faultin Singlecall
    • Each method call gets its own object instance for the duration of the call
  1. Well Known - Faultin Singleton / Activated
    • Object faulted in on first use
    • Lease Based
    • Keeping Objects alive for remote clients
    • Cleanup objects when clients no longer exist.
  1. Registered Singleton
    • Object registered
  1. Objects can manually Disconnect themselves from Remote Clients and any further calls will return with an Exception ObjectNotFound


Boundaries

Key Points

NGWS Messages
  1. AppDomain
    • Type Isolation
    • Static Isolation
    • No sharing
  1. Object Reference Management
    • Marshal and Unmarshal across boundaries
    • Distributed Identity across Boundaries
  1. Context
    • Context Attributes and Properties enforces rules on types.
    • 3rd parties can build then own context attributes and polices
    • Synchronization
    • Thread Affinity


Remoting Encoding

Key Points

NGWS Messages
  1. Serialization Support in COR BCL
  1. System.Serialization.Formatters.Soap.SoapFormatter
  1. System.Serialization.Formatters.Binary.BinaryFormatter
  1. 3rd parties can write Formatters
  1. Formatters Plug into Channels


Transports

Key Points

NGWS Messages
  1. Channels
    • A channel provides point to point communication of messages out of and into AppDomains
    • Pluggable
    • Sender and Receiver can be indepdent
    • No assumption of what is at the other end (could be a PERL script!)
  1. Pluggable Channels
  1. Pluggable Formatters
  1. HTTP Channel
    • Client – System.Net
    • Server – System.Net
    • Server - ASP+ Listener
    • SOAP Formatter
    • POST, M-POST
  1. TCP channel
    • Client – System.Net
    • Server – System.Net
    • Binary Formatter
  1. 3rd parties can write own Channels


Discovery

Key Points

NGWS Messages
  1. HTTP GET root -> HTML Simple Description
  1. SDL Formatter

    NGWS Types + Annotation <-> XSD Schema

    |

    V

    MD & IL Emit

  1. SDL dynamically generated


Remoting Security Overview

Key Points

NGWS Messages
  1. Client System.Net security
    • SSL
    • Digest
    • Basic Auth
    • NTLM Challenge Response
  1. ASP+ Security


Remoting Administration

Key Points

NGWS Messages
  1. ASP+ Configuration File +
  2. Channels for an Application
  3. Security for an Application / Types
  4. Remote Types and Application
    • URL / Machine name
    • Application


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Role-based Security in NGWS

© 2000 Microsoft Corporation. All rights reserved.


Introduction to Role-based Security in NGWS

This specification discusses requirements for Role-based Security mechanisms in the NGWS runtime the supporting infrastructure provided to address those requirements. Role-based Security is a common application concept in which one makes Authorization decisions based on the identity and/or role(s) for the entityon whose behalf an application is executing. The definition of “roles” is typically application specific as are the conditions under which one is willing to authorize specific actions.


Relevant Documents

NGWS Runtime Application Domains

NGWS Runtime Security Permissions

NGWS Runtime Security Policy Specification


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Role-based Security Requirements

© 2000 Microsoft Corporation. All rights reserved.


Role-based Security in Managed Applications

It is common for applications developers to make authorization decisions based on the identity and/or role(s) associated with the execution context. One often sees this in financial or business systems as a means of policy enforcement. For example, one may impose limits on the value of a transaction depending on the role of the user making the request. Clerks may be able to process transactions up to some set value, supervisors to some higher limit, and VPs yet a higher limit. Another example is business systems requiring multiple approvals to complete an action. Consider a purchasing system in which any employee may generate a purchase request, but only a purchasing agent may process that request into a Purchase Order that can be sent to the supplier.

In the past, such requirements were often handled using proprietary mechanisms. Microsoft first introduced infrastructure to support this type of “role-based” security with MTS and recently COM+1.0 services. The NGWS runtime provides similar support but does so in a way that is well integrated into the NGWS runtime programming model and scales to the needs of distributed internet applications.

Integration with the NGWS runtime programming model means that role-based security mechanisms:

  • Provide the ability to check the identity and/or role of the entity on whose behalf code is executing
  • Integrate well with distributed applications using NGWS runtime object remoting
  • Work consistently in both client and server execution environments
  • Expose these services to app developers via mechanisms similar to Code Access Security checks
  • Are ubiquitous, and always available
  • Are easily configured to use application defined roles

Considerations for Server Applications

Business logic operating on a server will often perform role-based checks as described above. From a programming standpoint, the NGWS runtime mechanisms for performing role-based security checks should appear identical whether one is running in a typical client or server environment. However, the server environment, as typified by the NGWS ASP+ and NGWS runtime remoting Well Known Object functionality, does impose important requirements on the infrastructure.

First, in the server environment one is most often dealing with an incoming request from a remote system. These requests are typically processed by one or more “handlers” prior to calling the application code. These handlers may perform multiple functions such as request routing, logging, authentication, etc. It is important to note that from a role check perspective, it isn’t the OS account on whose behalf the handlers are running that’s of interest, but rather the entity on whose behalf the external request is being made. Hence, the system must provide a straightforward means of linking Authentication actions and the server execution context.

Second, it is common in the server environment to use process/thread pooling with asynchronous call handling for efficiency. The result is that multiple processes/threads may be involved in responding to a given request. Once one has determined the entity on whose behalf the request is being made, it is necessary to bind this information to the call context so that it may be passed on to the thread currently processing the request.


COM+1.0 Interoperability

MTS (and subsequently COM+1.0) introduced role-based security functionality to the Microsoft platform. This allowed application developers to make authorization decisions based on roles associated with the NT account on whose behalf code was executing. These checks could either be directly coded by the application developer, via the IsCallerInRole() method, or administratively configured to restrict access to COM interfaces or methods.

COM+1.0 additionally supports functionality to determine the “callers” in the call chain, where the caller refers to the NT account on whose behalf application code was running. For each caller, one can determine the SID, AccountName, Authentication Service, Authentication Level, and Impersonation Level. COM+1.0 also removed several restrictions on what code could perform role-based security checks.

MTS is available on Windows NT 4 systems with the NT 4 Option Pack installed. COM+1.0 services are available on Windows2000. Within these environments it should be possible to use these role-based security mechanisms from inside managed code so long as the managed code is exposed to MTS/COM+1.0 as a COM type. This makes it feasible to extend existing applications while taking advantage of the NGWS runtime.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Role-based Security Functionality

© 2000 Microsoft Corporation. All rights reserved.


Encapsulating Identity Information

In order to support application identity/role checks we need a way to expose this information to managed code. This requires there be managed objects representing this information. We use two concepts—Principal and Identity—when considering this information. A Principal represents the security context (role membership) under which the code is running while an Identity represents the identity of the user associated with that security context.

The IPrincipal Interface defines a property for accessing an associated Identity object and a method for checking whether the entity identified by the Identity object is a member of a given role. A Principal object is required to implement the IPrincipal Interface. The Principal object definer may implement additional properties or methods as desired.

The Identity of the principal is exposed via an Identity object, which implements the IIdentity Interface. This IIdentity interface defines properties for accessing a name and a “type”. Where the type information indicates the Authentication mechanism used to generate the identity information (i.e., NTLM, Kerberos V5, Basic Auth, ….).

The definer of an Identity object may implement additional properties or methods beyond those in IIdentity as desired. An example is the WindowsIdentity object that provides identity information based on a Windows NT token with methods to allow the impersonation/revert operations possible under NT.

In general, one will require authentication Credentials in order to determine the correct identity to use. Examples of Credentials include:

  • a name and password pair
  • a Public Key Certificate and proof of Private Key possession (i.e., the result of a challenge-response protocol)
  • a Kerberos ticket

The appropriate Credentials will depend on the type of authentication supported in a given application environment. It should be noted that to avoid a potential security vulnerability, the Credential information is not stored as part of an Identity object.

As should be clear from the above discussion, there will always be an Identity object associated with any valid Principal object. Throughout the remainder of this document, references to a Principal object implies there is an Identity object accessible via the Principal.Identity property.

Relationship to an OS Account Token

The NGWS runtime makes no assumption about the relationship between an Identity object and the OS account on whose behalf a given process is running (i.e., process token). This is done to support scaling to heterogeneous environments and Internet distributed services in which no centrally administered trust environment is present.

It is possible the Identity object will reflect the same OS account as that associated with the process token. In practice, this means there is a WindowsIdentity object that maps to the NT process token. This is determined by the application being run and its associated authentication services.


Managing Identity and Role Information

There are number of scenarios we must be concerned with that impact the creation and management of identity/role information. This section discusses the requirements and design for handling each of these. We briefly summarize the situations of concern below. The subsequent sections expand on how these are handled.

  1. Unmanaged code has determined the Principal & Identity objects useful for this application and wants to set them prior to calling into the managed application code. In this case, the unmanaged code must be able to set the desired reference to the Principal object for the current thread of execution prior to calling the managed code entry point.
  2. The unmanaged code developer know that the entry into managed code is always to some highly trusted (i.e., an ASP+ handler) managed object that sets the correct Principal object reference. In this case, we should not construct default Principal and Identity objects.
  3. The managed application code knows it handles requestor authentication and will create the correct Principal & Identity objects, attaching the Principal object to the current execution thread prior to any use of such objects. Again, we should not construct default Principal and Identity objects.
  4. The application relies on a worker thread pool. A thread will pick up an asynchronous completion event and then transition into managed application code in the correct AppDomain. In general, the application will want the Principal object associated with the asynchronous call to be attached to new worker thread.
  5. The host doesn’t know what Principal & Identity info the app may want nor if the app performs its own authentication services. In this case, we must be prepared to compute appropriate default Principal and Identity objects and attach the Principal to the thread.
  6. Managed code creates a new Context in the current AppDomain which needs to have the appropriate Principal and Identity objects associated with it.
  7. Managed code creates a new execution thread which needs to have the appropriate Principal and Identity objects associated with it.
  8. Managed code does a remote object call to another AppDomain in the same process, an AppDomain executing in a different process, or an AppDomain executing on a different machine. The appropriate Principal and Identity objects needs to be available to the called code.
  9. The managed code application is designed to extend an existing COM+1.0 application and compatibility with COM+1.0 security semantics is required.

Creating the Initial Principal and Identity Objects

To support role-based Security, the NGWS runtime infrastructure must provide a means for managed code to inquire about the identity and/or role of entity on whose behalf the application is acting. The Principal object, exposing the identity and role information through public properties and methods, supports this. Effectively, this means we must know what Principal object should be associated with each execution thread transitioning into a given AppDomain.

One option for determining this is to have the calling code supply this information just prior to transitioning into managed code. If the code is sufficiently trusted this should be allowed. For example, we support scenarios where the trusted host has access to authentication information prior to calling into managed code within an AppDomain.

A second option is that we simply create default objects representing the identity and role information based on an AppDomain specific policy. There are two options that must be supported.

  • Create Principal and Identity objects for the unauthenticated entity
  • Create Principal and Identity objects that reflect the OS token associated with the current execution thread. We would map the associated OS groups into the roles.

We default to the first option since it is generally dangerous to give potentially hostile code access to information about OS account names, roles, etc. In a typical situation involving mobile Internet code, this could allow a user’s log-on name to be communicated back to the code’s site opening up a potential attack. But there are also situations where an application has legitimate requirements to access such information. Typically, these applications would be more trusted and reside on the local machine.

The NGWS runtime will support these options through a global AppDomain policy setting. A Trusted Host may override the default policy specifying use of Principal and Identity objects for the unauthenticated entity. The implementation defers construction of default Principal and Identity objects until the first attempt to reference them. Constructing these objects can be expensive, especially if it requires accessing an NT account token. Many applications will never check identity or role information, so we only do this work when absolutely necessary. This means that no special handling is required for scenarios where managed application code will supply the Principal and Identity prior to making and role checks. The act of attaching the Principal to the execution thread automatically overrides the default AppDomain policy.

Irrespective of how the Principal and Identity objects are created, we need to associate them with the executing code. Maintaining the Principal and Identity objects as an attribute of the AppDomain isn’t adequate to meet all needs. Rather, these objects need to be bound to an NGWS runtime execution Context within the AppDomain. Asynchronous execution models, such as implemented by ASP+, drive this requirement. The runtime supports this model by allowing certain context properties to flow between the threads serially assigned to processing a request is based on CallContexts. By attaching a Principal object reference to the CallContext we fit into this general architecture. Note that it isn’t necessary to attach the Identity object to the CallContext since the Principal object contains a reference to it. This design doesn’t present any unusual problems since a default context is created with an AppDomain, so there is always one available to accept the Principal object.

Context Transitions

A given AppDomain may contain multiple contexts. This raises the issue of how to handle the Principal object when creating a new context within managed code. One could insert a reference to caller’s Principal object on transition into a callee’s context, or create a new Principal object for each context. A single Principal object referred to by multiple contexts has several advantages:

  • Minimizes the number of Principal objects to maintain
  • Insures the Principal object reference remains valid when flowing across contexts
  • Avoids any special case handling by an application to maintain the desired Principal object when transitioning across contexts. This could be a fairly common problem. Consider an application defined Authentication services that uses a transaction context when accessing a database of known entitys. This would authenticate the entity; create the desired Principal and Identity objects; and then return. But the return would cause a context transition potentially reverting to a different Principal object.

However, it also has disadvantages:

  • May require resolving a reference that crosses context boundaries for each Principal object access.
  • Makes it more difficult for applications to create context-dependent identity/role information, though there are no clear requirements for this capability.

The advantages of maintaining a single Principal object outweigh the potential disadvantages. There is limited overhead in references to an object across contexts if it is declared marshal-by-value. Given that Principal object references will typically be infrequent, this approach should be more efficient than cloning the Principal object on each context creation and managing changes to multiple contexts when an application needs to change the valid Principal object.

We also need to properly handle the Principal object when creating a new thread within an AppDomain. When a new thread is created, a CallContext will be created for the new thread. We will automatically copy the Principal object reference from the creating thread to the new threads CallContext. This provides the new thread with the required Principal and Identity objects. There may be certain pathological cases where it is difficult to determine the Principal object of the creator when a new thread is started. In such cases, we fall back to the default AppDomain policy for Principal and Identity object creation.

Replacing the Principal Object in Managed Code

It must be possible for managed application code to replace the Principal object for a given thread. This is necessary in those case where an application supplies managed code authentication services. This capability is security critical, since the Principal object affects subsequent authorization actions based on identity/role information. Hence, any code attaching Principal objects to the current Context must be trusted by the application calling it. We use Code Access Security to provide a means of expressing this trust and restricting access to manipulation of the Principal object reference in the CallContext. Only Assemblies granted the System.Security.Permissions.SecurityPermission(ControlPrincipal) permission will be able to perform this operation. This permission does not impact the ability to create Principal objects or the ability to perform role-based security checks.

Principal object lifetimes are handled in exactly the same manner as any other managed object. Once all references to the object are released, they are subject to garbage collection. This does mean an application may need to free Principal object references when replacing the active Principal objects.

Impersonation and Revert Operations

A natural question to ask is the relationship between the Identity object and impersonation/revert operations under Windows NT. There is no required coupling between an Identity object and the NT process token under which a thread is currently executing. If the Identity object is a WindowsIdentity object, then it is possible the Identity object will represent an impersonatable NT token.

Given a WindowsIdentity object, and sufficient NT privilege, a thread can use this object to impersonate the referenced account. This simply requires calling the Impersonate method on the WindowsIdentity. Doing an impersonation does not change the Principal object associated with the current CallContext. Reverting to self is equally simple and also does not alter the Principal object for the current CallContext.

Object Remoting

Object remoting introduces additional considerations on the management of Principal objects. Within the NGWS runtime, object remoting is generally used to describe the process of calling a method on a class when the code is not running in the current AppDomain. Hence, it includes calls that are to:

  • Code executing on a different machine
  • Code executing in a new process on the same machine
  • Code executing in the same process but a different AppDomain

When doing object remoting, the called code may be interested in performing authorization actions based on the entity identity/roles associated with the calling code. A simplistic approach to addressing this requirement is to simply “flow” the Principal object associated with the caller’s Context to the callee’s Context. Unfortunately, other considerations make this unworkable as a general solution.

Remoting to a Different Machine

Consider the case where the call is between objects on different machines. Copying of the Principal object to the callee’s machine raises both a trust issue and usability issue. The Principal object contains information about a entity’s identity and roles, but these were generated by trusted code on the caller’s system based on some form of authentication Credentials. The Principal object doesn’t provide the callee with access to the authentication Credentials and even if it did there is no guarantee it would resolve these credentials to the same identity and roles. Hence, there is no reason for the callee to believe the information contained in the Principal object. To make this approach viable would require cross-machine trust between the NGWS runtimes, an infrastructure for understanding cross-machine Authentication policy, and a secure channel for passing the Principal object between the systems. The full span of required mechanisms are not availablein the NGWS runtime V1 product.

There are also potential usability issues. If the callee’s Identity object (accessible via the Principal object) represents an abstraction of a Windows OS token, there is no way to preserve the original semantics when copying since Windows security tokens are machine dependent. Other specialized application-defined Identity objects may exhibit similar problems.

In addition, the NGWS runtime is required to support interoperability with existing systems. Hence, objects on a remote system may be called by unmanaged code. This only requires the callee to properly format and transmit a message that is understood by the caller. For such cases, it is unreasonable to force applications to support a complex infrastructure for transmitting trusted Principal objects between machines.

In consequence, Principal objects are not be transmitted between machines. Rather, we require the appropriate entity Credentials (as defined by the called application) be transmitted so the correct Principal and Identity objects can be constructed for the caller’s Context. In some cases, the remoting infrastructure may be able to automatically determine the correct Credentials to use. Examples include Kerberos delegation or PublicKey/Certificate-based authentication when only a single valid entity Certificate exists. In other cases, this may require user interface code to request entity Credentials (name/password, correct Certificate to use, and so forth).

Remoting to a Different Process

In this case, one is concerned about remoting to an object running in a different process but on the same machine. It is similar to the cross machine case in that both processes must be running, the AppDomains created and the caller and callee objects instantiated prior to the call being made. This means the caller already has a Context with a Principal object at the time of the remote call.

But since one is communicating between two processes on the same machine the trust and usability issues associated with the cross-machine case are largely mitigated. Since each process may be running a different version of the NGWS runtime, they may still have different policies in effect.

In consideration of these factors, we use exactly the same model as the remote machine case. Specifically, this approach:

  • Supports a consistent distributed application programming and configuration model.
  • Simplifies the requirements on unmanaged code doing remote object calls via messaging.
  • Avoids any special case handling when the two processes are running different versions of the NGWS runtime.
  • Avoids any special case handling if the two processes are running under different security policies that could affect identity and/or role computation.
  • Obviates the need for runtime code and policy settings to control resolution when both a called Context Principal object exists and a callee Context Principal object is presented.

Remoting within the Same Process

This case further simplifies the potential issues in passing identity/role information since we know the Process is running under a single security policy, there is only one version of the NGWS runtime, and the Principal object can easily be referenced across the two AppDomains. In addition, it would be unusual to expect code designed to run in-process to perform its own Authentication services.

Given these factors, it isn’t necessary to use the same approach as for the cross-Process and cross-Machine situations. Rather we will use a very straightforward mechanism:

  • When creating a new in-process AppDomain, no special handling is required. If sufficiently trusted, the code creating the AppDomain may set the policy controlling the type of Principal and Identity objects created by default.
  • When making the remote call to a different AppDomain, the remoting infrastructure will copy a reference to the Principal object associated with the caller’s context to the callees context. Since we are in the same process, the callee can access the Principal object via this reference without complex marshalling.

This is the only behavior supported and no application specific configuration is provided.

COM+1.0 Interoperability

The NGWS runtime supports managed code interoperability with COM+1.0 security services as discussed in the subsequent paragraphs. Interoperability with the older MTS functionality, only available on NT 4, is not directly supported from managed code though it is accessible using unmanaged code interoperability. This decision was made due to the more limited deployment of MTS along with a number of functional limitations (i.e., the inability to use security services from inside the caller’s process).

COM+1.0 security services can only be used by managed code on Windows2000 systems. Using these services requires the managed application code be exposed as COM objects and registered with the COM+1.0 registration database. COM+1.0 then transparently tracks the ‘context’ associated with the code and maintains the mapping between the process token account and application-defined roles.

To make managed code visible to COM+1.0 services requires it be marked with the ComEmulate custom attribute. This insures all calls into this code are made using COM-compatible mechanisms. Developers should be aware that calls into other managed code objects not marked ComEmulate are opaque to COM+1.0 services. Basically, they are treated the same as COM component calls into non-COM DLLs.


Using Role-based Security in NGWS

The preceding section discussed the requirements and design of the role-based security infrastructure in NGWS. This section discusses the publicly exposed interfaces to that functionality.


Managing Principal and Identity Objects

Setting Identity/Role object Policy

Trusted code creating an AppDomain may set the AppDomain policy for controlling construction of default Principal and Identity objects. This is global to the AppDomain and will apply to all execution threads. An unmanaged, Trusted Host, inherently has the right to set this policy. Managed code creating an AppDomain must have the System.Security.Permissions.SecurityPermission(ControlPrincipal) permission to set this policy.

Setting the policy is done using a method on the AppDomain:

AppDomain.SetPrincipalPolicy(PrincipalPolicy policy)

Where, PrincipalPolicy is an enum with the following values:

Enum PrincipalPolicy {
   UnauthenticatedPrincipal;   // create Generic Identity and Principal objects representing the unauthenticated entity
   WindowsPrincipal;         // create WindowsIdentity and WindowsPrincipal objects representing the NT token for the execution thread

}

Additional policy values may be allowed in future releases.

Supplying Principal and Identity Objects to an AppDomain

Unmanaged, Trusted Hosts, creating an AppDomain may also supply a Principal object, and associated Identity object, to attach to the CallContext for a given thread. A Trusted Host can construct these objects using the NGWS runtime interoperability facilities.

We define a Method on the AppDomain for this purpose:

AppDomain.SetThreadPrincipal (IPrincipal principal)

Managed code creating a new in-proc AppDomain is not allowed to supply initial Principal and Identity objects. The rules of assigning an initial Principal object in this case are described in the section entitled Managing Identity and Role Information.

Constructing Default Principal and Identity Objects

Replacing a Principal Object

Once running inside an AppDomain, authorized managed code may replace the active Principal object for the current thread. To do this, the code:

  • Creates the new Principal object and associated Identity object, usually after performing an Authentication action.
  • Creates a new System.Security.Permissions.SecurityPermission(ControlPrincipal) and does an Assert of this permission. In general, it will need to Assert this permission since other code in the call chain is unlikely to have this permission granted. If the code has not been granted the SecurityPermission, with ControlPrincipal and Assert enabled, then the code will be unable to replace the Principal object.
  • Finally, the code uses a property on the Thread object to attach the new Principal object to the CallContext.

    Thread.CurrentPrincipal = principal;

    If the code is not authorized to perform this operation, then this call will fail and a SecurityException will be generated.

Modifying Principal and Identity Objects

Principal and Identity objects in the NGWS runtime are not modifiable after construction. The identity, role, and other information they represent are bound into them at creation time and can not be changed by application code.

Principal and Identity Object Destruction

Principal and Identity objects can not be destroyed by application code. They are managed objects and will be automatically garbage collected by the NGWS runtime when no longer referenced.


Performing Principal and Identity Demands

Directly Referencing the Principal Object

There are cases where direct access to the Principal object is required. We provide a mechanism to do this easily. This allows one, for example, to access the method for performing impersonation given a WindowsIdentity or to access to behaviors specific in an application defined Principal.

It is also possible to use this mechanism to perform identity/role checks. For example:

If (Thread.CurrentPrincipal.IsInRole(“Administrator”)) {}

This test will succeed if “Administrator” is a valid role within the Principal object.

Similarly, an identity may be checked by

If (Thread.CurrentPrincipal.Identity.Name == “Bob”) {}

The test will succeed if “Bob” matches the name held in the Identity object.

Using the PrincipalPermission Object

A mechanism for checking the identity and/or roles consistent with those for resource and code identity permissions is also supported. This allows one to leverage the defined language constructs for both declarative and imperative security actions.

This functionality builds on the Code Access Security permission objects and the IPermission abstract Interface specification defining a “PrincipalPermission” object to enable this functionality. The PrincipalPermission exposes a constructor that accepts an identity string and a role string. For example:

   String id = “Bob”;
   String role = “Administrator”;
   PrincipalPermission principalPerm = new PrincipalPermission(id, role);

To match the active Principal object, and associated Identity object, both the specified identity and role must match. A NULL identity string is interpreted as a request to match any identity. A NULL role string will match any role. By implication, PrincipalPermission(NULL, NULL) will match the identity and roles in any Principal object.

It is also possible to construct a PrincipalPermission that only checks for whether the Identity represents and authenticated or unauthenticated entity. In this case, the name and roles are ignored.

To determine if the active Principal object matches the identity and/or role for a given PrincipalPermission, one does a “Demand” on the permission object. This is simply a check against the active Principal, there is no dependence on code in the call chain and no stack walk is required. Hence, the Demand() and DemandImmediate() methods on the PrincipalPermission always return the same result.

As with other permissions, one may intersect and/or union the PrincipalPermission objects. An intersection is somewhat non-sensical since we expect individual PrincipalPermission objects to be orthogonal. But, we define it for completeness. The result if intersecting PrincipalPermission objects is the identity and/or role that are common to the set , where intersecting non-comparable entities returns the empty string. For example:

   String id1 = “Bob”;
   String role1 = “Administrator”;
   PrincipalPermission PrincipalPerm1 = new PrincipalPermission(id1, role1);
   String id2 = “Louise”;
   String role2 = “Administrator”;
   PrincipalPermission PrincipalPerm2 = new PrincipalPermission(id2, role2);

PrincipalPerm1.Intersect(PrincipalPerm2) will return an object equivalent to that created by

   String id3 = “”; // i.e., no identity can simultaneously represent both Bob and Louise
   String role3 = “Administrator”;
   PrincipalPermission PrincipalPerm3 = new PrincipalPermission(id3, role3);

In effect, PrincipalPerm3.Demand() would only succeed if we had an unauthenticated principal (with name = “”) allowed to act in the Administrator role.

A union will create a permission object representing the criteria of a given set of individual PrincipalPermission objects and is useful for compactly representing a set of conditions to test. Given the above example,

(PrincipalPerm1.Union(PrincipalPerm2)).Demand()

will succeed if the Principal object represents:

  • Bob in the role of Administrator

    or

  • Louise in the role of Administrator

Subset operations will determine if a PrincipalPermission instance is a strict subset of another. Effectively this means the identities and roles must exactly match or one must be NULL

Declarative Demands

Consistent with Code Access Security, it is possible to place Declarative demands for identity and/or role compliance on any Class, Method, or Event (see NGWS Runtime Security Permissions). These are interpreted as requirements that must be satisfied for the associated method(s) to be called. If they are not met, then a call attempt will result in a SecurityException.

The Declarative syntax is language specific, but will indicate the type of check required. It is only valid to request a runtime ‘Demand’ of this permission, i.e., the SecurityAction enum value is SecurityAction.Demand.

Use of LinkDemand or InheritanceDemand is not valid since these checks are only done once, on the first reference between a caller and callee. This is useful for code evidence related permissions, since these don’t change after loading. But, there is nothing to prevent the same code path from being executed multiple times on a behalf of a different entity as reflected in the Principal and Identity objects. Also Assert, Deny, and PermitOnly actions aren’t valid, as these aren’t defined for an object implementing the IPermission interface.

Declarations may affect multiple Methods within an Assembly:

  • Class declarations apply to all Methods in the Class without a specific declaration
  • Property and Event declarations apply to their associated Methods (e.g., get).
  • A PrincipalPermission declaration on a specific method will override a Class declaration.

Imperative Demands

Imperative demands for a given identity or role can be easily incorporated into managed code. Doing this requires the code create a new PrincipalPermission object specifying the identity and role of interest. For example, checking for an identity of “Bob” and a role of “Administrator” can be done by:

   String id = “Bob”;
   String role = “Administrator”;
   PrincipalPermission PrincipalPerm = new PrincipalPermission(id, role);
   try {
      PrincipalPerm.Demand();
      // code here executes when identity and role match
  }
 catch (SecurityException) {
      // error handling code
 }


Using COM+1.0 Security

NGWS managed types marked with the ComEmulate custom attribute and registered with COM+1.0 services may use COM+1.0 security services. Information on using and administering COM+1.0 security services are provided in the Platform SDK and COM+1.0 documentation.


Security Policy Implications

Security Policy controls which managed code is allowed to replace the active Principal object associated with a given thread. To replace the Principal object requires the Assembly be granted the System.Security.Permissions.SecurityPermission(ControlPrincipal) permission.

Of course, such code may also need additional permissions such as FileIO read access to access an authentication database of known entities. In addition, one would generally grant such code the Assert permission since one wouldn’t want to grant all code in the call chain permission to change the active Principal object.

Security policy does not control the rights to perform identity or role checks. There will always be a Principal object present and any application code may check its identity/role information. This treatment is similar to that for Code Identity permissions.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Public Interfaces and Objects

© 2000 Microsoft Corporation. All rights reserved.


Interfaces

IIdentity Interface

namespace System.Security.Principal
{
    public interface IIdentity
    {
      // the entity identity
      String Name { get; }
      // the authentication type
      String Type { get; }
      // is this an authenticated identity
      Bool IsAuthenticated  {get; }
   }

}

Notes:

  • Name will typically be set to the empty string for an unauthenticated entity, but need not be.
  • Type should be set to the empty string or a string representing an a type of authentication understood by the application (Basic auth, NTLM, Kerberos, Passport, …)

IPrincipal Interface

namespace System.Security.Principal
{
    public interface IPrincipal
    {
        // the associated identity object
        IIdentity Identity  { get; }
        // dteremine if the principal represents a given role
        bool      IsInRole  (String Role);

    }
}


Default Principal and Identity Object Classes

This section defines default Principal and identity objects shipped with the NGWS runtime. Applications may define their own objects to deal with specific needs. These objects may be referenced across multiple threads within an AppDomain, and so must be thread safe.

GenericPrincipal Object

namespace System.Security.Principal

public class GenericPrincipal : IPrincipal
{
   // Constructors
   public GenericPrincipal (IIdentity identity, String [] roles) {}

      // IPrincipal Properties
   public IIdentity Identity  { get; }

   // IPrincipal Method
   public virtual bool IsInRole(String Role);
}

GenericIdentity Object

namespace System.Security.Principal

public class GenericIdentity : IIdentity
{
   // Constructors.
   public GenericIdentity (String name)
   public GenericIdentity (String name, String type)

   // IIdentity Properties
   public String Name { get; }
   public String Type { get; }
   public bool IsAuthenticated {get;}
}

Notes:

  • If NULL parameters are supplied to the constructor, then construction will fail.
  • Unless isAuthenticated parameter is supplied to the a constructor, supplying a non-empty name string to the constructor implies IsAuthenticated is true. If the name string is “” then IsAuthenticated is set to false.

WindowsPrincipal Object

namespace System.Security.Principal

public class WindowsPrincipal : IPrincipal
{
    // Public Constructor
    public WindowsPrincipal (WindowsIdentity ntIdentity) {}

    // IPrincipal Property
    public IIdentity Identity { get;}

    // IPrincipal Method
    public virtual bool IsInRole (String Role);

    // Populate the list of NT groups in which this entity is present
    private  void PopulateGroups();
 }

Notes:

  • PopulateGroups will be called by the constructor
  • If NULL identity supplied in the constructor, then construction will fail.

WindowsIdentity Object

namespace System.Security.Principal

public class WindowsIdentity : IIdentity
{

    // IIdentity Properties
    public   String    Name       { get; }
    public   String    Type       { get; }
   bool IsAuthenticated {get;}

   // WindowsIdentity Properties
   public int Token { get;}
   public bool IsGuest {get;}
   public bool IsSystem {get;}

   // Public Constructors, Token must be a valid Windows Token
    public WindowsIdentity (int Token) {}
    public WindowsIdentity (int Token, String type) {}
    public WindowsIdentity (int Token, String type, WindowsAccountType  acctType) {}
    public WindowsIdentity (int Token, String type,  WindowsAccountType acctType, bool isAuthenticated) {}

    // Try to impersonate based on the Windows Token
    public virtual WindowsImpersonationContext  Impersonate();
    public virtual WindowsImpersonationContext  Impersonate(int Token);

   // Static method to get a WindowsIdentity for the current
   // execution token
   public static WindowsIdentity GetCurrent();

}

Where:

Enum WindowsAccountType {
   Guest;
   Normal;
   System;
}

public class WindowsImpersonationContext
{

    // Revert from the impersonation associated with this object
    public void Undo();
}

Notes:

  • The identity name should be retrieved based on the token using LookupAccountSID, this reduces probability of creating a mis-match between the name string and account name.
  • The WindowsAccount type enum in the constructors is used to indicate that the supplied token represents a special account. This can either be ‘Guest’ for a Windows guest account or ‘System’ for a system account. The properties IsGuest and IsSystem are set accordingly. If it is a guest or system account, and the authenticated parameter is not explicitly supplied, then authenticated is assumed false. For Normal accounts it is assumed authentication has succeeded.
  • If the Token is invalid, or any other parameters are Null, then construction should fail.


PrincipalPermission Object

namespace System.Security.Permission

public Interface IPermission
{
    public IPermission Copy();
    public IPermission Intersect(IPermission target);
    public IPermission Union(IPermission target);
    public boolean IsSubsetOf(IPermission target);
    public void Demand(); // throws SecurityException
    public void DemandImmediate(); // throws SecurityException
}

public class PrincipalPermission : IPermission
{
   // Public Constructors
   public PrincipalPermission( String name, String role) {}

   public PrincipalPermission(bool Authenticated) {}
   // permission will match authenticated or unauthenticated
      identity based on value of Authenticated regardless of
      Identity.Name value.

    // IPermission Methods
    public IPermission Copy();
    public IPermission Intersect(IPermission target);
    public IPermission Union(IPermission target);
    public boolean IsSubsetOf(IPermission target);
    public void Demand();
    public void DemandImmediate();

}


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Runtime Code Access Security Administration specification

NGWS Runtime Security Specification


Introduction to Code Access Security Administration

The NGWS runtime includes an extensible security system. This security mechanism determines whether a particular managed assembly is allowed to run and what degree of trust to access protected resources it is given.

This document shows how to administer the NGWS runtime security mechanism using the CASPol administrative tool. For this purpose it will be shown how to use CASPol to complete common administrative tasks. This document also includes a section on the schema of the security configuration files.


Relevant Documents to Code Access Security Administration

NGWS Runtime Security Permissions

NGWS Runtime Security Policy Specification


Overview of the code access security model

The NGWS runtime includes an extensible code access security (CAS) model. The goal of code access security (CAS) is to prevent code from accessing protected resources it has no right to access (such as undesired access to files or the registry). In order to satisfy this goal the CAS policy system grants code specific amounts of trust based on evidence about the code. A CAS policy is expressed as a tree-hierarchy of code groups (see diagram below). Each code group is defined by one and only one membership condition as well as an associated permission set. A membership condition ascertains whether some piece of code has the necessary credentials (evidence) to belong to the respective code group. We support the following types of standard membership conditions: Zone, Site, Strong name, Publisher, URL, Hash value, Skip Verification, code’s application directory and the all code membership condition (see NGWS Runtime Security Policy specification). For instance, in the diagram below the leftmost code group in the bottom row has the membership condition that code should possess the strong name MS.office.

Each code group has a named set of permissions associated with it. A named permission set contains at least one permission and a name and description for the permission set. Each permission expresses a specific amount of trust to access a particular protected resource. We support permissions for FileIO, UI access, registry access, Reflection, File Dialogue access, Isolated Storage, Security related settings, and access to environment variables (see permissions specification). We also allow developers to define their own permissions. A named permission set therefore represents an expression of trust to access some protected resources. In the example below the code group defined by the site evidence ‘localweb’ has the ‘MyPermissions’ permission set associated with it. This means that all code to which this code group applies will get trust to read and write to c:\temp and do registry operations.

As mentioned before, code groups are hierarchically arranged allowing for ever-decreasing sub-divisions among code groups. For instance, in the code group hierarchy diagram above the code group defined by the membership condition Zone:intranet, has the child code groups defined by Site:localweb and publisher:corpadmin. Each of these code groups has a named permission set associated with it. This means that a piece of code that comes from the intranet zone will get the permissions granted that are part of the permission set associated with the intranet zone code group. If that same piece of code also comes from the site local web, then it also gets the permissions that are part of the permission set associated with the site:localweb code group. In general, if code belongs to more than one code group, all permissions that are part of the permission sets of the code groups the code belongs to, are unioned together. This yields the effective permissions granted to code.

There are currently two tiers of policy we support - the machine policy and the user policy. The machine policy is the code group hierarchy that applies to the whole machine and it is typically set by the machine admin. A user policy is a code group hierarchy that represents the per-user security settings, it may be modified by the user. The overall policy applying to a specific piece of code is determined by the policy at each of the two tiers. The permissions granted to code by the machine policy setting are intersected with the permissions granted by the user level policy settings. This means that the machine policy represents a “high water mark” for the user policy. For instance, suppose the machine policy for some code only grants code read access to c:\temp but the user level policy grants both read and write access to that directory. In this case the effective permissions the code receives is just read access to c:\temp, which represents the intersection between the permissions granted by machine and user level policy.

For more in-depth information concerning the policy system, please refer to the NGWS Runtime Security Policy specification.

In the following section it will be shown how code access security gets administered using the CASPol tool.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Administering code access security using CASPol

© 2000 Microsoft Corporation. All rights reserved.


Introduction to CASPol

CASPol is a command line tool that is used to administer code access security (CAS). Security policy changes should always be made using this tool. This insures that the security configuration files will not be inadvertently corrupted. CASPol also includes several features that allow administrators to catch, reverse and analyze unintended policy changes.

In the following sections I will show how most standard security administration tasks are completed using CASPol.

Referencing security objects

To facilitate references to code groups in a hierarchy, the policy list (-list command, see below) will show indented objects with labels (1, 1.1, 1.1.1, etc.) for reference purposes. Subsequent command line operations targeting code groups use these labels to refer to specific code groups.

Named permission sets are referenced by their name. When a policy is listed the list of code groups followed by a list of named permission sets available in that policy will be shown.

CASPol users can also switch between the user and machine policies (using the –machine and –user options, see below).

Default Behavior

If a user without machine administrative rights calls CASPol, by default all CASPol options refer to the user level policy, unless the CASPol user explicitly refers to the machine policy level (see ”Viewing Policy”).

If an administrator calls CASPol, by default all CASPol options refer to the machine’s policy, unless the administrator explicitly specifies the user policy level as target (See below).


Running CASPol

CASPol is installed as part of the runtime SDK and the PATH is automatically set to point to the directory in which the CASPol executable is located. Therefore, typing ‘caspol’ at any command line will start up the program. Typing ‘caspol’ without any options will show the available options their syntax.

If the executable needs to be directly accessed please go to %SDKDIR%\Bin, where SDKDir stands for the directory into which the sdk has been installed.


Viewing Policy

In order to view the policy (code group hierarchy and list of known permission sets) for a policy level the –list option is used.

Caspol –list

This will show the code group hierarchy and known named permission sets at the default policy level (for administrators that is the machine policy level, for others that is the user policy level

It is possible to explicitly defer an option to either the machine or user policy level. For this simply prefix an option with –machine or –user. This can be used to list the policy of either the machine or user level, as can be seen below:

Caspol –machine –list
Caspol –user –list

If the policy of both machine and user level should be listed, instead of –machine or –user, one can simply use –all, as shown below:

Caspol –all –list

This will list both the machine and the user policy.

Generally the output format for the –list option is:

Microsoft (R) COM+ Caspol [Version] - the policy manipulation tool
Copyright (c) Microsoft Corp 1999-1999. All rights reserved.

Security is [ON|OFF]
Execution Checking is [ON|OFF]
Policy Change Prompt is [ON|OFF]

Level = [Machine|User]

Code Groups:

1. [(FirstMatchCodeGroup)] MemberShipConditionName [– Value] : PermSetName
[(FirstMAtchCodeGroup)] MemberShipCondName [– Value] : PermSetName
……
…..

Named Permission Sets:

1. Permission Set:Pset_name,(Description)
<Pset Xml>

2. Permission Set:Pset name,(Description)
<Pset Xml>

As can be seen code groups are labeled by a reference number. This number is used in CASPol options to refer back to specific code groups. Both the membership condition name as well (if present) as a membership condition value are listed, followed by the name of the permission set associated with that code group. If the code group merges the permissions of its child code groups using the First Match logic this is indicated by (FirstMatchCodeGroup). The default merge logic is unioning the permissions granted by child code groups (for more information on this please refer to the Policy Specification)

Following the code group hierarchy is a list of the named permissions known at that policy level.


Viewing Code Groups

It is possible to only list the code groups belonging to a policy. For this one simply uses the –listgroups option.

Caspol –listgroups

This will list the code groups at the default policy level. As with the –list option, -machine,-user,-all can be used to determine for which policy level code groups should be listed. For instance, the command below lists the code groups at the user policy level.

Caspol –user -listgroups

The command below will list the code groups at both the machine and user policy level.

Caspol –all -listgroups


Viewing permission sets

It is also possible to view just the permission sets known at a policy level. This is done using the –listpset option, which functions similar to the –listgroups option (see “Viewing Code groups”).

Caspol –listpset

The above command will list the permission sets at the default policy level. Using the –user/-machine/-all options as prefix to the –listpset option enables the CASPol user to choose for which policy the known permission sets should be dumped. For instance the command below will show the permission sets known both at the machine and at the user policy level.

Caspol –all -listpset


Adding a named permission set to a policy

Before a named permission set can be associated with a code group it needs to be known at that policy level. A new named permission set can be added to a policy using the option –addpset. The permission set needs to be created beforehand as an xml file (see section which is then imported using –addpset (see below). For instance, the command below imports the MyPermissions.xml permission set file as the MyPermissions permission set:

Casol –addpset MyPermissions.xml

The following sections will present more detail about how to actually author and import a named permission set.

Creating a named permission set file

In order to add a new permission set using –addpset, an xml file representing the permission set must be authored. As the name indicates, every permission set consists of a set of permissions. Consequently the permission set xml file is simply the xml representation of a number of specific permissions. In general the xml format for standard permissions is the following:

<PermissionSet class=”System.Security.NamedPermissionSet” version=”1”>

   <Permission class=”System.Security.Permissions.PermissionName, mscorlib,[Ver=XXXX.XX.XXXX.XX], SN=03689116d3a4ae33” version=”…”>
      [permission xml]
   </Permission>

   <Permission class=”System.Security.Permissions.PermissionName, mscorlib,[Ver=XXXX.XX.XXXX.XX], SN=03689116d3a4ae33” version=”…”>
      [permission xml]
   </Permission>

   …
   <Name>[Name of permission set]</Name>
   <Description>[description of the permission set]</Description>
</PermissionSet>

As can be seen, the permission set file always starts with the <PermissionSet class=”System.Security.NamedPermissionSet” version=”…”> tag, and ends with the </PermissionSet> tag. The <Name> tag simply indicates the name of the permission set (that is the name the permission set will be referred to when it is to be modified or deleted). The content between the <Description> tags should be used to comment on the intended purpose of the permission set (i.e. what permissions are granted from this permission set). Specifying the assembly version (“ver=”) is optional. “SN=” denotes the compact strong name originator of the mscorlib assembly. For the prerelease the version=”..” attribute is ignored. It will be used for forward compatibility and versioning.

The <Permission> tags sandwich the actual XML serialization of specific permissions. The Runtime security system ships with a set of standard permission types that can be used to create specific permissions to be included in named permission sets. For a list of these standard permission sets and their XML format consult ”Schema for the security configuration” of this document.

Below is an example of a small permission set file, representing the FullFileIO permission set.

<PermissionSet class=”System.Security.NamedPermissionSet” version=”1”>
<Permission class=”System.Security.Permissions.FileIOPermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
      <Unrestricted/>
   </Permission>
   <Name>FullFileIO</Name>
   <Description>Gives Full File io</Description>
</PermissionSet>

Once a permission set file has been created, it can be added to a policy. The following CASPol command will add the FullFileIO permission set to the user policy level (also see “Importing a named permission set file” below)

CASPol –user –addpset FullFileIO.xml

Using Custom Permissions

Custom permissions are permissions that are not shipped as standard ingredients of the runtime security system. It is possible to use custom permissions and include them in permission set files. Custom permissions are surrounded by the permission tags, and a proper permission class+assembly name needs to be specified:

<Permission class=”CustomPermissionClassNamespaceandName, name, [Ver=”XXXX.XX.XXXX.XX], SN=compactStrongNameReprOfAssembly ” version=”…”>
      [permission xml]
</Permission>

The major difference is that class=”” will point to a custom permission class as opposed to a permission class that is shipped with the Runtime. The permission XML in-between the <Permission> tags is completely up to the custom permission developer, since it is entirely processed by the class that implements the custom permission.

The out-of-the-box permission sets

There are a number of permission sets that are shipped by default with every policy. In particular, every policy already includes the following permission sets:

  • Nothing : Denies all resources, including the right to execute
  • FullTrust: Allows full access to all resources
  • Internet: Default rights given to internet applications
  • Everything: Allows unrestricted access to all resources covered by built in permissions
  • Execution: Permits execution
  • SkipVerification: Grants right to bypass the type safety verification process that takes place as part of Just In Time compilation (JIT)
  • LocalIntranet: Default rights given to applications on your local intranet

These permission sets can be associated with code groups and represent very commonly used permission sets. For some administrative needs it is therefore possible to reuse these permission sets as opposed to having to create a new one.

With the exception of Internet and LocalIntranet, no standard permission set can be changed. Nothing, FullTrust, Everything, Execution and SkipVerification are therefore immutable standard permission sets.

Importing a named permission set file

There are two ways to add a new permission set to a policy using –addpset.

  1. If the name of the permission set is contained in the permission set file (within the <Name> tags), then a permission set name does not need to be explicitly defined and the following syntax is used: Caspol –addpset PsetFile.xml. This will be the most common way of importing a named permission set. Example:
    Caspol –machine –addpset MyPset.xml
  2. If, however, a permission set file does not contain a name sandwiched between <Name> tags then a name for this permission set must be explicitly defined. Such a case may occur when permission sets are distributed by third parties who want to avoid name conflicts and leave it up to the administrator to choose a unique name for the permission set. In these instances the following syntax is to be used: Caspol –addpset PsetFile.xml PermissionSetName. Example:
    Caspol –addpset MyUnnamedPset.xml FileReadRestrictions


Removing a Permission Set

A permission set can only be removed if and only if

  1. the permission set is not one of the standard immutable out-of-box permission sets
  2. the permission set is not currently used at any code group in the policy level from which it is to be deleted

If neither of the above conditions obtain then a permission set can be removed from a policy. In order to remove a permission set from the default policy level just type:

Caspol –rempset PsetName

As can be seen, permission sets are always referred to by their name. Using the –machine and –user options it is possible to explicitly state from which policy level a given permission set is to be deleted. For instance, the command below deletes the MyFilePset from the machine policy:

Caspol –machine –rempset MyFilePset

The xml file from which the permission set may have been imported will not be deleted. All that –rempset does is to exclude a certain permission set from a policy.


Changing a permission set

It is possible to change the content of a particular permission set using CASPol. This is done by importing a new permission set xml file that replaces the old permission set. No out-of-the-box permission set with the exception of Internet and LocalIntranet can be altered, though. For the PDC release of caspol no permission sets that are currently in use by a code group can be altered.

In order to change the permission set, either alter an existing permission set xml file or author a new one .Then use the following syntax to replace the old permission set with the new one:

Caspol –chgpset pset_file pset_name

The command below shows how the FilePset permission set in the user policy gets changed to the content of the NewPset file.

Caspol –user –chgpset NewPset.xml FilePset


Adding a code group to a code group hierarchy

Both the machine and user policy are expressed as a hierarchy of code groups. Every code group has a membership condition as well as a permission set associated with it. So when adding a code group to either the machine or user code group hierarchy both a membership condition as well as a named permission set must be defined. The label of the parent code group under which the new code group is to be added must also be defined. Optionally, it is also possible to set a couple of flags on the code group. Below is a summary of the syntax:

 Caspol -addgroup parent_label mship  pset_name  [-exclusive {on|off}][-levelfinal {on|off}]

The parent label is the label of the parent code group to which the new code group should be attached. To find out what code group is identified by what label, simply do a –list or a –listgroups.

The pset_name simply refers to the named permission set that is to be associated with the new code group . Before a named permission set can be associated with a code group, it needs to be known at the policy level at which the code group should be inserted. For instance, if a MyPset permission set is to be associated with a new code group in the user policy, the MyPset permission set must have been added to the user policy previously. The only time when a permission set does not need to be added beforehand is if one of the standard (out-of-the-box) permission sets is used.

Mship stands for the membership condition for the new code group. A membership condition is basically what defines whether particular assemblies are or are not covered by a code group. For instance, a common code group is Zone, with value Internet. So code that has evidence certifying that the code has originated on the Internet will meet this membership condition. Below is the general syntax that mship can take in the –addgroup option:

Argument | Description
-all | All code group, all code applies to this code group
-pub { -cert file.crt | -file file.exe | –hex hexstring } | Software publisher denoted by certificate file, the signature on a file, or a hex string
-strong -file file_name {name | -noname} {version | -noversion} | Strong name as designated by a file, name and version.
-zone zonename | Zone of origin of code. Zonename can be one of the following values: “Internet”, ”Intranet”, ”Trusted”, ”Untrusted”, “MyComputer”
-site website | Site of origin.
-hash hashAlg {-hex hashValue | -file assembly_file } | An Assembly’s hash can be used as a code group membership condition. Either the hash value or the assembly file needs to be specified.
-appdir | If –appdir is specified as membership condition the URL evidence of code is compared with the AppDir evidence of that code. If both evidence values are the same this membership condition succeeds.
-skipverification | This membership condition tests whether code requests to skip verification.
-url urlname | url of origin

Finally, it is also possible to set a couple of exclusive flags on the new code group. See explanation below:

Argument | Description
-exclusive {on|off} | When this flag is set only the permission set of the added code group applies when some code fits the membership condition of the added code group.
-levelfinal {on|off} | When this flag is set (typically at the machine policy level) then no policy level below the level in which the added code group occurs is considered. For instance if this flag is set for some code group at the machine level and some code matches this code group’s membership condition, then the user level policy for this code is not calculated and applied.

Common code group addition tasks

Below are examples and explanations for some of the most common code group addition tasks:

Adding a code group that targets code from the Internet

In order to make a code group applicable to code from the Internet, one simply uses the ‘–Zone’ option, with value ‘Internet’ as membership value. In the example below, the ‘Nothing’ permission set is associated with code from the Internet and inserted as a child code group to the root of the code group hierarchy.

Caspol –addgroup 1 –zone Internet Nothing

Adding a code group that targets code from the Intranet

The solution is similar to the one above. By using ‘-zone’ with value ‘Intranet’ one can generate a code group that targets code from the Intranet. In the example below a code from the Intranet is given the ‘Everything’ permission set.

Caspol –addgroup 1.1. –zone Intranet Everything

Adding a code group that only applies to a certain software publisher

Files from a software publisher need to be signed appropriately in order for this membership condition to work. The ‘–pub’ membership condition can be used to set a code group to apply only to a certain software publisher. The membership condition can either be constructed on the basis of an actual certificate file or a signed exe file. Suppose the certificate file for BuggySoft is handy (BuggySoft.cer) then the command below adds a code group in the machine policy with permission set ‘Nothing’ for code published by BuggySoft. The code group is added as a child code group of the root.

Caspol –machine –addgroup 1 –pub –cert buggysoft.cer Nothing

Adding a code group that applies to code from a particular web site

In order to add a code group that applies only to code from a particular web site one should use the ‘–site’ membership condition. It should be noted though that due to the possibility of DNS name spoofing this is not a safe way to ascertain the identity of a piece of code. Where possible one should use strong name or publisher membership conditions. Below is an example that gives code coming from the ‘www.Microsoft.com’ Intranet permissions.

Caspol –addgroup 1 –site www.microsoft.com Intranet

Adding a code group that overrides any other permissions granted at that policy level

For this purpose one simply sets the ‘–exclusive’ flag for the new code group. In the example below a code group is added that grants the ‘Everything’ permission set if the zone is Trusted, overriding any other permissions granted by any other code group.

Caspol –addgroup –zone Trusted Everything –exclusive On

Adding a specific code group both to the user and machine policy level

CASPol does not support a way in which an –addgroup commands adds a code group to the user and machine level at the same time. Each such addition has to be done separately, already since differing code group labels and conflict in the availability of certain permission sets may cause confusion.


Changing a code group

Using the –chggroup option it is possible to change the membership condition, associated permission set and exclusive flags. It is possible to change only one,some or all of these code group attributes. The –chggroup option works in a similar fashion to the –addgroup option introduced above. The syntax for the –chggroup option is:

Caspol –chggroup label {mship |pset_name | [-exclusive {on|off}][-levelfinal{on|off}] }

The syntax and meaning of mship,pset_name,exclusive and levelfinal are exactly the same as for the –addgroup option (see “Adding a code group to a code group hierarchy”). Below are some examples that illustrate the use of this option.

This example shows how the code group at label 1.2.1. gets the Zone membership condition Internet:

Caspol –chggroup 1.2.1. –zone Internet

The following example shows how the code group 1.3. gets the Internet permission set associated with it and the group exlusive flag turned off

Caspol –chggroup 1.3. Internet –exclusive off

It is very important to note that changes to a code group can have wide repercussions for security. Use this option with caution.


Deleting a code group from a code group hierarchy

Extreme caution should be exercised when deleting code groups since this may have strong effects on security. When deleting code groups with child code groups, it should be noted that all the child code groups are deleted as well. It should also be noted that at this point there is no way to copy part of the code group hierarchy to another part or another code group hierarchy. So deleting a parent code group most definitely destroys any security behavior that was gleaned from the child hierarchy of the deleted code group.

Code groups can be deleted using the –remgroup option. Below is the syntax for this option:

Caspol –remgroup label

The example below shows the deletion of the code group labeled 1.1.2.

Caspol –remgroup 1.1.2.


Analyzing the security settings that apply to an assembly

When an assembly does not seem to run properly it may very well be that it has not been granted sufficient permissions to execute in the way envisioned by the author of the assembly. It is possible to use CASPol to localize problems with an assembly in respect to the policy system. CASPol offers two analysis features: showing the code groups that apply to a particular assembly and showing the permissions that apply to a particular assembly.

Showing the code groups that apply to a particular assembly

Sometimes unintended security settings apply to an assembly because it does (or does not) belong to some unconsidered code group. Such side effects may occur when code groups are added or deleted. In order to make it easier to find such problems in the user and machine code group hierarchy CASPol implements a feature that lists all the code groups an assembly belongs to. The syntax is as follows:

Caspol –resolvegroup assembly-file

Assembly-file is simply the name of the managed code that should be tested for code group membership. The example below shows how you can find the code groups that CASPol itself belongs to (CASPol is itself a managed application)

Caspol –resolvegroup caspol.exe

As all other CASPol options the –resolvegroup option will apply to the default policy level , unless it is explicitly specified whether just the code group matches at the machine policy level (-machine), user policy level (-user) or both (-all) should be shown. In the example below, the code group matches for CASPol at the user policy level are shown.

Caspol –user –resolvegroup caspol.exe

The below example shows the code group matches for CASPol at both the machine and user policy levels:

Caspol –all –resolvegroup caspol.exe

Showing the permission set that applies to an assembly

Code gets granted a set of permissions based on evidence about the code. Each policy levels grants a set of permissions to code. The effective permission set that the security policy grants to code is determined as the intersection between the permission sets granted by the machine and user policy. CASPol can be used to view the permission set that an assembly would be granted by a particular policy level, CASPol can also be used to view the permission set that results from intersecting the permission set granted by the machine policy to code with the permission set granted by the user policy to code. To show the permission set for a particular assembly the following syntax is used:

Caspol –resolveperm assembly-file

So for instance to show the permissions that apply to CASPol at the user policy level, the following command should be used:

Caspol –user –resolveperm caspol.exe

To show the permission set that is granted to CASPol by the overall (machine and user) policy, simply type:

Caspol –all –resolveperm caspol.exe

Common security policy problems with an assembly and how to fix them

How to analyze problems with an assembly that does not run

There are a wide variety of reasons why an assembly may not run. It may be the case that it does not have sufficient permissions to run properly. In order to find that out, one should first determine what permissions policy grants that assembly:

Caspol –all –resolveperm assembly_file

The above command can be used to find out the permissions granted to the assembly by the policy system. Check whether the assembly has the right to execute by checking whether the execution flag is set in the SecurityPermission. Furthermore, if the assembly needs to access a protected resource (such as disk access, create windows, write into the registry etc) one should check whether the appropriate permissions are set.

If it turns out that insufficient permissions are set for that assembly, one should check the permissions granted at the machine policy to the assembly and the permissions granted from the user policy level using

caspol –machine –resolveperm assembly-file

and

caspol –user –resolveperm assembly-file

Note: since the permissions granted from the machine and user policy are intersected, a permission must be set at both levels in order for an assembly to receive it.

If some permissions were insufficient one can try to introduce a new code group using ‘–addgroup’ that grants the assembly the permissions needed. It may also be the case that the assembly should have been covered by a code group whose membership condition is too restrictive, or whose associated permission set does not contain the permissions needed for the assembly to function. To find out which code groups the assembly belongs to type

Caspol –all –resolvegroups assembly-file

One can then attempt to modify either the associated permission set or the membership condition of code groups so that the assembly has the appropriate permissions granted to it. Such modification is done using the ‘–chggroup’ option.

Analyzing an assembly that accesses protected resources or is not supposed to run

If an assembly runs, but it should not, or if an assembly accesses resources (such as creating dialogue windows) but it shouldn’t, then the assembly has probably been granted too many permissions. As in the case above, using ‘–resolveperm’ one should find out what permissions the assembly is granted. Then, using ‘–resolvegroups’ one can find out what code groups it belongs to and change either the membership conditions or associated permission sets of the code groups the assembly belongs to in order to limit the permissions it is granted.


Undoing a policy change

It may turn out that a policy change has undesired side effects, or that the change accidentally was done at the wrong policy level (i.e. machine instead of user policy). CASPol allows for the recovery of the last machine or user policy before a change has been made. The syntax is as follows:

Caspol –recover

The –machine and –user options are used to explicitly state what policy should be recovered. For instance, the below command undoes the last change to the user policy

Caspol –user -recover

Note that this option only undoes the last change. CASPol does not cache any changes before that. That means if you call this option twice you The policy recovery will have been annulled.


Returning to the default policy settings

In some dire cases it may be necessary to reset a policy to the out-of-the-box defaults after some hard to track policy changes created undesirable side effects. In such cases use the following syntax to reset a policy:

Caspol -reset

Note that the reset can be undone using the –recover option if no further policy changes were done after the –reset call. The –machine and –user options can be used to specify which policy level should be reset. For instance the below command resets the machine policy:

Caspol –machine -reset


Dealing with CASPol's self protection mechanism

CASPol implements a self-protection mechanism that does not allow policy changes that would disrupt CASPol from functioning. There may be two scenarios in which this self-protection mechanism can legitimately be overridden:

  • an absolutely crucial policy fix, the administrator is aware that CASPol may not function properly afterwards, but the fix is essential to patch a security hole (we hope such a scenario will be extremely rare)
  • sometimes policy changes are a matter of more than one call to CASPol. It may be the case that intermediate policy states before the series of policy changes is complete, are policy states in which CASPol would not function properly. Yet, at the end of the policy changes, policy again grants all the necessary permissions to CASPol.

In order to override CASPol’s self-protection feature use –force to prefix the policy change option that would otherwise be rejected by CASPol:

Caspol –force –…

In the example below the user policy’s root code group is changed to have the Nothing permission set associated with it.

Caspol –force –user –chggroup 1 Nothing

It should be noted that this option is extremely dangerous. It may cause CASPol to fail functioning, in which case –recovery can not be applied because CASPol may not be able to run.

Recovering from a –force policy change that causes CASPol not to function anymore

It is possible to do the manual equivalent of a –recovery operation. The backed up machine and user policy are written to security.cfg.old files. Simply delete the security.cfg of the policy level at which you made the crashing change and rename the security.cfg.old into security.cfg. These files can be located using the Windows Explorer™ search function.


Suppressing the policy change warning

Every time a policy changing option is called CASPol warns of the specific policy change that is about to take effect. For users who are highly familiar with this tool such a warning may become unnecessary and it can be switched off:

Caspol –polchgprompt off

If you want to switch it back on again just type:

Caspol –polchgprompt on

Before turning this feature off, consider that it is often much easier to quickly double-check and confirm a policy change as opposed to having to do after-the-fact policy fixing.


Turning security off

It is possible to completely switch security off. This may be of advantage in environments where the administrator can have full trust in the code that is running.

Security can be switched off in the following manner:

Caspol –security off

It can be turned on in the following way:

Caspol –security on

WARNING: Turning security off is a machine wide operation, terminating security checks for all managed code and running for all users on that machine. Although the –list option will show security turned off, there are no other clear indications for other users that security has been switched off.


Editing the security configuration files manually

We strongly recommend that whenever possible the security administration is done using CASPol. This insures that policy changes don’t corrupt the configuration files. Since CASPol does not currently support administrators in accessing arbitrary user’s configuration files, some hand editing may become necessary. The configuration files are stored at the following locations:

Machine policy configuration
Windows 2000 | %WINDIR%\complus\vxx.xx\security.cfg
Windows NT | %WINDIR%\complus\vxx.xx\security.cfg
Windows 9x | %WINDIR%\complus\vxx.xx\security.cfg
User policy configuration
Windows 2000 | %USERPROFILE%\complus\vxx.xx\security.cfg
Windows NT | %USERPROFILE%\complus\vxx.xx\security.cfg
Windows 9x | %WINDIR%\username\complus\vxx.xx\security.cfg

NOTE:The security.cfg file for a particular policy will only be persisted on disk if the policy has been changed from the default using CASPol (The default policies are hardwired). So, for instance if a user only ever used CASPol to list his or her user policy but did not make any changes to it, there will be no security.cfg file persisted for that user. In that case the administrator will need to author a new policy file. When the security.cfg files are created they are appropriately ACLed so that only people with machine administrative rights can modify the machine’s security.cfg.

When calling –reset on a default policy, CASPol automatically saves the default policy to disk. Although the administrator cannot call –reset on arbitrary user’s policy, he or she can call –reset on his own user policy (if it is still the default policy), and use the persisted file as a starting point for authoring the particular user policy he/she had in mind.

Example: Administrator wants to change the default user policy of user Joe Doe

  1. (presuming the admin’s user policy hasn’t been changed), admin uses caspol –user –reset to generate a default user security.cfg on disk at admin’s user policy level directoy (see user policy level configuration table above).
  2. admin copies this file to the user’s policy configuration directory of Joe Doe (admin may need to create that directory)
  3. admin uses an editor to modify the file


Using abbreviated CASPol options to administer security

Instead of typing out the name of CASPol options it is also possible to use abbreviations. This should significantly reduce typing time, and we therefore recommend that administrators become familiar with the following abbreviations:

Option | Valid abbreviation
-machine | -m
-user | -u
-list | -l
-listgroups | -lg
-listpset | -lp
-addpset | -ap
-chgpset | -cp
-rempset | -rp
-remgroup | -rg
-chggroup | -cg
-addgroup | -ag
-resolvegroup | -rsg
-resolveperm | -rsp
-security | -s
-execution | -e
-help | -h
-polchgprompt | -pp
-recover | -r
-reset | -rs

For instance, in order to add a code group to the user policy the following would need to be typed if CASPol did not support abbreviations:

Caspol –user –addgroup …

However, using abbreviations you can simply type:

Caspol –u –ag …


Summary of the CASPol syntax

Below is the summary of the syntax that governs the CASPol command line tool:

CASPol   [-user]
   [-machine]
   [-list]
   [-listgroups]
   [-listpset]
   [-addgroup parent_label mship pset_name [-exclusive {on|off}][-
   levelfinal {on|off}]]
    [-chggroup label {mship | pset_name | -exclusive {on|off}| -
   levelfinal {on|off}]
   [-remgroup label]
    [-addpset { psfile pset_name | psfile } ]
   [-chgpset psfile pset_name]
    [-rempset pset_name]
   [-resolvegroup assembly-file [-all]]
   [-resolveperm assembly-file [-all]]
   [-security { on | off }]
   [-execution { on | off }]
   [-polchgprompt { on | off }]
   [-recover]
    [-reset]
   [-force]
   [-help]


Schema for the security configuration

As an administrator it may be the case that you need to know about the underlying schema of the configuration files. Check the table below for cases in which full or partial knowledge of the schema is necessary to complete certain administrative tasks:

Adminstrative Task | Knowledge of schema required
Changing, adding permission sets using CASPol | You should know the permission schema (what goes between <Permission..> tags), and the Permission set schema (what goes between the <PermissionSet..> tags)
Manually editing security configuration files | You should know the complete configuration schema as described here
All other security related administrative tasks (using CASPol) | No knowledge of the schema is required, but could be helpful when reading the output of the policy analysis and list options. Use this chapter as a reference when output involving permissions seems unclear.

When the user or machine policy is changed using CASPol the policy configuration is persisted in a security.cfg file. This file is in XML format and represents the serialization of a security policy.

In this section the different element types (<elementtype>),their attributes and values, and the way the elements need to be arranged in order to express a permission, permission set, code group, policy and finally configuration file will be introduced. The security configuration schema will be presented from the bottom up, starting with the most atomic part: permissions.

Important Note on notation. Symbols in bold are part of the meta-language (describing the syntax of the schema):

| means inclusive ‘or’ (expressions part of a | clause can be combined in full or in subsets) v means exclusive ‘or’, differentiating expressions that cannot appear together,

* means multiple instances of that expression are allowed,

[] contains a description of a of value to be inserted at that spot.

An expression marked with % is optional

There is no particular order in which tag elements at the same hierarchy level must appear. For instance, instead of

<…>
   <Read>..</Read>
   <Write>..</Write>
<…>

<…>
   <Write>..</Write>
   <Read>..</Read>
<..>

would work also.


Permission schema

General schema for permissions:

<Permission class=”[class name, assemblyName]” version=”[version number such as “1”]”>
   […permission specific XML…]
</Permission>

Explanation: [class name] indicates the name of the class as URI. For instance for all standard permissions the class name will be System.Security.Permissions.nameofstandardpermission. The assemblyName consists of a name, optional assembly version and the assembly’s compact strong name originator. Since every standard permission is included in the mscorlib assembly all stadard permissions have the following schema:

<Permission class=”System.Security.Permissions.NameOfStandardPermission, mscorlib, [ver=”XXX.XX.XXXX.XX”], SN=03689116d3a4ae33” version=”1”>
      …Permission XML…
</Permission>

The assembly version for mscorlib can be omitted, this way the latest version of the permission class is being used.

Each permission has permission specific xml elements (see sections on various permissions). The version attribute value is used to indicate the version of the xml configuration syntax with respect to NGWS frameworks and runtime releases. This will facilitate backwards compatibility.

This document will now lay out the schema for the standard permissions as well as for custom permissions. For details on the permissions please see the Security Permissions specification.

Schema for the EnvironmentPermission

This permission controls the access to environment variables.

Schema:

<Permission class=”System.Security.Permissions.EnvironmentPermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
   (<Read> [list of environment variables] </Read> |
   <Write> [list of environment variables] </Write>)v<Unrestricted>
</Permission>

Explanation: The read tag is optional and is followed by a list of case insensitive names of environment variable. Variables are separated by semicolon. The write tag is optional and also followed by a variable list demarcated by semicolons. Instead of the read/write tags the <unrestricted/> flag may be set also.

Example: Permission to read Files and Path variable, permission to write ACCESS variable

<Permission class=”System.Security.Permissions.EnvironmentPermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
   <Read>”PATH”;FILES</Read>
   <Write>ACCESS;”My;var”</Write>
</Permission>

Schema for FileDialogPermission

This schema controls the extent to which a file dialog may be used.

Schema:

<Permission class=”System.Security.Permissions.FileDialogPermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
      <Unrestricted/>
</Permission>

FileIOPermission schema

This permission determines the extent file input/output is allowed.

Schema:

<Permission class=”System.Security.Permissions.FileIOPermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
      (
      <Read>[list of files or folders]<Read/> |
      <Write>[list of files or folders] <Write/> |
      <Append/>[list of files or folders] </Append>
      ) v <Unrestricted/>
</Permission>

Explanation: lists of files and folders are semicolon separated path names, or path+ file names. The read/write/append tags must not appear in that order, a subset of these tags may be used (such as only the read tag). The unrestricted tag cannot appear in combination with the read/write/append tags.

Example: permission to read from c:\temp and to read the file D:\MyFiles\document.doc to which also append rights have been given.

<Permission class=”System.Security.Permissions.FileIOPermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
      <Read>C:\Temp;D:\MyFiles\document.doc<Read/>
      <Append/>D:\MyFiles\document.doc</Append>
</Permission>

unrestricted file access

<Permission class=”System.Security.Permissions.FileIOPermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
      <Unrestricted/>
</Permission>

IsolatedStorageFilePermission schema

This permission determines how the isolated store may be used.

Schema:

<Permission class=”System.Security.Permissions.IsolatedStorageFilePermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
(
<UserQuota>[number of bytes]</UserQuota> |
<MachineQuota>[number of bytes]</MachineQuota> |
<Expiry>[no of days after which permission expires]</Expiry>|
(<DomainUser/>v<AssemblyUser/>v<AssemblyMachine/>v<AdministerUser/>v<AdministerMachine/>v<None/>v<Unrestricted/>) |
<Permanent/>
) v <Unrestricted/>
</Permission>

Explanation: This permission sets specific amounts of trust for the isolated storage

Example:

<Permission class=”System.Security.Permissions.IsostorePermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
      <DomainUser/>
      <UserQuota>157000</UserQuota>
      <Expiry>40</Expiry>
      <Permanent>True</Permanent>
</Permission>

SocketPermission schema

This permission controls access to Sockets

Schema:

<Permission class=”System.Security.Permissions.SocketPermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
      <Transport>(TCPvUDP)</Transport>
      <Hostname>[ip address or dns name]</Hostname>
      <Port>[port number or wildcard denoting all ports]</Port>
</Permission>

ReflectionPermission schema

This permission permits the ability to read the name and type information of non-public members of types, as well as to find subclasses and superclasses and what module and assembly the class is in.

Schema:

<Permission class=”System.Security.Permissions.ReflectionPermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
      (<TypeInformation/>|<MemberAccess/>|<ReflectionEmit/>)
      v<Unrestricted>
</Permission>

Example:

<Permission class="System.Security.Permissions.ReflectionPermission, mscorlib, SN=03689116d3a4ae33” version="1">
      <TypeInformation/>
      <MemberAccess/>
      <ReflectionEmit/>
</Permission>

RegistryPermission Schema

This permission is used to separately set read, write and append rights to specific keys/values. As with the FileIOPermission read/Create or write tags cannot appear along with the unrestricted tag.

Schema:

<Permission class=”System.Security.Permissions.RegistryPermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
      (
      <Read>[list of keys and values]</Read> |
      <Create>[list of keys and values]</Create> |
      <Write>[list of keys and values]</Write>
      ) v <Unrestricted/>

</Permission>

Example:

<Permission class=”System.Security.Permissions.RegistryPermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
      <Read> some keys,values separated by semicolon</Read>
      <Append></Append>
      <Write></Write>

</Permission>

SecurityPermission Schema

This permission is used to set a variety of simple permission flags used by the security system. The execution flag must be granted for an assembly to be to run. The assertion flag is used to grant an assembly the ability to assert any permission it has been granted (again, for more details see the permissions specification). The unmanaged code flag is used to indicate that an assembly can call unmanaged code. The skip verification flag allows code to run without passing verification. And finally the thread control flag allows code to do thread operations.

Schema:

<Permission class=”System.Security.Permissions.SecurityPermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
      (
      <Execution/> |
      <Assertion/> |
      <UnmanagedCode/> |
      <SkipVerification/> |
      <ControlThread/>|
      <ControlEvidence/>|
      <ControlPolicy/>|
      <ControlDomainPolicy/>|
      <ControlPrincipal/>|
      <SerializationFormatter/>
      )v<Unrestricted/>
</Permission>

Example: The permission below allows code to execute, bypass verification and to call unmanaged code.

<Permission class=”System.Security.Permissions.SecurityPermission, mscorlib, SN=03689116d3a4ae33” , version=”1”>
      <Execution/>
      <UnmanagedCode/>
      <SkipVerification/>
</Permission>

UIPermission Schema

This permission defines access to various aspects of the Windows UI: use of display windows, use of clipboards (and events – not implemented yet). For use of windows as well as the clipboard various gradations of trust can be set by a variety of exclusive flags (viz. if Nowindows tag appears Allwindows tag cannot appear also in the permission).

Schema:

<Permission class=”System.Security.Permissions.SecurityPermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
      (
      <AllWindows/> v
      <SafeTopLevelWindows/> v
      <SafeSubWindows/> v
      <NoWindows/>
      ) |
      (
      <AllClipboard/> v
      <OwnClipboard/> v
      <NoClipboard/> v
      )v<Unrestricted/>
</Permission>

Example: Code gets granted the permission to use safe top level windows and any clipboard access.

<Permission class=”System.Security.Permissions.SecurityPermission, mscorlib, SN=03689116d3a4ae33” version=”1”>
      <SafeTopLevelWindows/>
      <AllClipboard/>
      <VersionTag>XMLNotationV1</VersionTag>
</Permission>

Custom Permission Schemas

Custom permissions can be defined to control access to resources not covered by the standard permission (or to allow for finer granularity in the control of protected resources addressed by standard permissions). Developers need to define custom permission objects that must support various APIs (such as de/encoding into XML – for more details see Permission specification).

Schema:

<Permission class=”[link to custom permission class], [name, [ver=XXXX.XX.XXXX.XX], SN=[compact strong name originator of custom permission assembly]  “ version=”[version number such as 1]” >
      […Custom permission specific XML – freely definable by custom permission developer… ]
</Permission>

Explanation: class=”..” must contain the name and namespace of the class that implements the custom permission, furthermore the assembly name needs to be given (as containing a name, strong name originator and possibly the assembly version). The only other requirement on a custom permission schema is that it is valid XML, that the custom permission class must be indicated as shown.

Note that custom permission assembly needs to be properly registered with and loaded into the Fusion cache.

Example:

<Permission class=”MyApp.Permissions.MyDBPermission, mylib, SN=44555jhw8whhs” version=”1”>
      <DB Name=”PayRecords” Access=”Read”/>
</Permission>


Permission Set Schema

Individual permissions are always part of some named permission set. Conversely a named permission set is formed from a set of permissions.

Schema for named Permission sets:

<PermissionSet class=”System.Security.NamedPermissionSet>
   (
   <Permission class=”…”…>
      [permission xml as seen in 5.1.]
   </Permission>
   )*
   <Name>[Name of permission set]</Name>
   <Description>[description of the permission set]</Description>
</PermissionSet>

Explanation: As indicated in the schema definition above a Permission set can include an arbitrary (non-zero) number of permissions. The only restriction is that the same permission type (viz. FileIOPermission) may not appear twice in the same permission set. Following the permissions are the name and the description of the permission set.

Example:

<PermissionSet class=”System.Security.NamedPermissionSet>
<Permission   class=”System.Security.Permissions.SecurityPermission”>
      <SafeTopLevelWindows/>
      <AllClipboard/>
      <VersionTag>XMLNotationV1</VersionTag>
</Permission>
<Permission class=”System.Security.Permissions.FileIOPermission”>
      <Unrestricted/>
      <VersionTag>XMLNotationV1</VersionTag>
</Permission>
<Permission   class=”System.Security.Permissions.SecurityPermission”>
      <Execution/>
      <VersionTag>XMLNotationV1</VersionTag>
</Permission>
<Name>MyPermissionSet</Name>
<Description>Allows clipboard and some Windows UI as well as full file IO and execurion of code</Description>
</PermissionSet>


Policy Schema Overview

A security policy is expressed in terms of a hierarchy of code groups. Each code group is defined in terms of one membership condition and has a permission set associated with it. Corresponding to this model the policy schema contains nested code group schemas. Below I will first introduce the code group schema. I will then show how that schema fits into the policy schema.

Code group schema

Schema:

Reminder: Expressions marked with ‘%’ are optional

<CodeGroup class=”System.Security.Policy.CodeGroup”/>
   <IMembershipCondition class=”[class name of evidence class]”>
      [evidence type specific xml]
   </ImemebershipCondition>
   <PermissionSet> class=”System.Security.NamedPermissionSet”>
      <Name>[name of the permission set]</Name>
   </PermissionSet>
   (<Exclusive/>|<Levelfinal/>)%

   (

      (
      <CodeGroup …>
         [Code group xml possibly including other childcodegroups]
      )*
   )%
</CodeGroup>

Explanation: Each code group is defined by a membership condition (testing membership for some particular type of evidence about code). The IMembershipCondition expresses this. The class name that gets inserted in the IMembershipCondition tag is the name of the class implementing the membership condition. Following this tag are some membership condition specific xml elements (for instance if the evidence type used in the membership condition is zone then there would be a <zone> tag taking the zone value). More on different membership conditions below. A code group may have child code groups, indicated by another <CodeGroup..> , which itself could of course have child code groups (the above schema definition is recursive).

Example: Below is a code group defined in terms of evidence type zone with zone value Internet. It has the named permissions set ‘ForInternet’ associated with it, and contains one child code group.

<CodeGroup class="System.Security.Policy.CodeGroup">
<IMembershipCondition   class="System.Security.Policy.ZoneMembershipCondition">
                          <Zone>Trusted</Zone>
      </IMembershipCondition>
<PermissionSet class="System.Security.NamedPermissionSet">
          <Name>ForInternet</Name>
      </PermissionSet>

      <CodeGroup class="System.Security.Policy.CodeGroup">
            <IMembershipCondition       class="System.Security.Policy.ZoneMembershipCondition">
                       <Zone>Untrusted</Zone>
                  </IMembershipCondition>

<PermissionSet class="System.Security.NamedPermissionSet">
                       <Name>Nothing</Name>
                  </PermissionSet>

             </CodeGroup>
      </CodeGroup>

I will now discuss the different forms of standard membership conditions and, correspondingly, the difference in schema (for the <IMembershipCondition> tag)

We are currently supporting the following types of standard evidence that have corresponding standard membership conditions:

  • Publisher (encrypted software publisher signature)
  • Site (Site of origin of assembly)
  • Strong Name (Strong[encrypted] name of assembly)
  • URL (URL of origin)
  • Zone (Zone of origin)
  • Hash (Hash algorithm)
  • Application Directory
  • Skip verification (membership condition tests whether code requests to skip verification)
  • All Code (applying to all code)

Site Membership Condition Schema:

<IMembershipCondition class=”System.Security.Policy.SiteMembershipCondition>
   <Site>[name of site of origin]</Site>
   …
</ImembershipCondition>

Publisher Membership Condition Schema:

<IMembershipCondition class=”System.Security.Policy.PublisherMembershipCondition”>
   <x509Certificate>[certificate value]</x509Certificate>
   …
</ImembershipCondition>

Strong Name Membership Condition Schema:

<IMembershipCondition class=”System.Security.Policy.StrongNameMembershipCondition”>
   <StrongName>[strong name value]</StrongName>
   …
</ImembershipCondition>

URL Membership Condition Schema:

<IMembershipCondition class=”System.Security.Policy.URLMembershipCondition”>
   <Url>[Url of origin]</Url>
   …
</ImembershipCondition>

Zone Membership Condition schema:

<IMembershipCondition class=”System.Security.Policy.ZoneMembershipCondition”>
   <Zone>[name of site of origin]</Zone>
   …
</ImembershipCondition>

Policy Schema

A policy is comprised of a hierarchy of code groups. Since each code group has a named permission set associated with it there must also be a list of named permission set definitions as part of a policy. The below schema reflects this model.

Schema:

<Policy>
   <PolicyLevel class=”System.Security.Policy.PolicyLEvel” version=”1”>
      <NamedPermissionSets>
         (
         <PermissionSet …>
            [xml of permission set definition as in 5.2.]
         </PermissionSet>
         )*
      </NamedPermissionSets>
      <CodeGroups>
         (
         <CodeGroup class=…>
         …     //each code group referencing one permission set             by the permission set’s name
         </CodeGroup>
         )*
      </CodeGroups>
   </PolicyLevel>
</Policy>

Explanation: Each policy contains a non-empty list of named permission set definitions (sandwiched between the <NamedPermissionSets> tags) and a non-empty code group hierarchy (sandwiched between <CodeGroups>).


Configuration Schema

A configuration file consists of a security policy and may (in future versions) also contain other configuration information. We therefore use a configuration file schema that would allow either for the addition of non-security configuration data or for the addition of non-policy security configuration information:

Schema:

<Configuration>
   //room for non-security config info
   <Security>
      //room for non-policy security config info
      <Policy>
         [Policy xml as seen above,5.3]
      </Policy>
   </Security>
</Configuration>

This represents the complete configuration schema, which comprises the security.cfg files that are used to configure the runtime security system


Schema versioning

Permissions include the version attributes which is used to indicate which runtime version the schema was intended to use. This allows the schema to be easily distinguished and used in later versions of the runtime. If this tag happens to miss from a permission then the default assumption is that the permission is of the newest version. For the pre-release the version attributes are ignored.


System.Serialization Model: Update

This section describes the Runtime’s System.Serialization package as well as some of the infrastructure the Runtime provides for serialization.


Overview of System.Serialization

This document specifies the NGWS Runtime Serialization services. Serialization is the process of converting a graph of objects, into a linear sequence of bytes. That sequence of bytes can be sent elsewhere (for example, to a remote computer) and Deserialized, thereby making a clone in that remote memory of the original graph of objects.

This document describes the NGWS runtime Serialization service and specifies the top-level design and details the APIs involved.


Some Background

Before we get to the meat here, let’s get some background information out there so we’re all on the same page.

What is Serialization?

Let’s clearly define what we mean by serialization. By Serialization we mean converting an object, or a connected graph of objects, stored within computer memory, and conventionally drawn on paper in two dimensions, into a linear sequence of bytes. That string of bytes contains all of the important information that was held in the objects we started with.

We can go on to use that sequence of bytes in several ways. For example:

  • Send it to another process (on the same machine) and use it to construct arguments to a method that is run in that other process. In this case, the sequence of bytes is copied from one location in the machine’s physical memory to another – it never leaves the machine
  • Send it to the clipboard, to be browsed or included in another application. As above, the sequence of bytes is transferred to another location in memory on the current machine.
  • Send it 'down the wire' to another machine and use it to create a clone on that machine of the original object graph. As above, we might then use this object graph as an argument to a method call. In this case, however, the sequence of bytes is actually transferred outside of the originating machine.
  • Send it to a file on-disk, so that it can be reused later. (See, for example, the classic “Scribble tutorial” in “Using Visual C++” issued as part of the Visual C++ 6.0, and earlier).

What’s an object graph?

An object graph is a set of objects with some set of references to each other. The interesting question here is what problems does this bring for Serialization? The most obvious problem is how to represent the links between the objects in the Serialized stream. After all, the value held in the field of the in-memory object which links to another object is essentially a 32-bit address, which has meaning only in the owner address space (and may even change, ‘beneath our feet’, due to garbage collection). Serialization needs to allocate each object in the stream a number. So, for example, we have assigned arbitrary, small numbers to objects below, and shown the class of each:

Then we can represent this graph of objects with a serialized stream like this:

Dog, 3, ref 4, ref 7, ref 1 || Cat, 4 || Cat 7 || Mouse, 1, ref 9, ref 2 || Horse, 9, ref 4 || Duck, 2

(To simplify the diagram, we’ve not shown the value of each object’s scalar fields, just the object references). It doesn’t matter what order we stream out the objects, nor what numbers we assign to the objects. What does matter is that no two objects are assigned the same number. The object numbers only have significance within a serialized stream. They are simply a way to represent the graph topology, and allow us to reconstruct a copy of that graph, perhaps on another computer. Of course, this is trivial to do when we can see the whole graph on paper. But an algorithm that visits objects one-at-a-time clearly has to keep track of which objects it has already ‘seen’, using an internal list for example. Without due care, we might otherwise serialize two copies of the Cat object on the left, and later Deserialize two copies of that object, one linked from object Dog,3 and the other copy linked from object Horse, 9.

Note that we have chosen to represent one complete object at a t ime in the serialized stream. This corresponds to a breadth-first traversal of the object graph. An alternative, depth-first traversal, would serialize each object, embedded into the stream, at the point the graph walk encountered its object reference in the parent object. So we’d end up with a stream like this, using a slightly different, but obvious syntax:

Dog, 3, < Cat 4>, <Cat 7>, <Mouse, 1, <Horse, 9, ref 4>, Duck, 2>>

In many cases, this technique avoids the need to insert explicit references into the stream (but not always – see the two links to Cat, 4, required to avoid duplicating objects at Deserializationn time).


How it works

Serialization is the process of converting a graph of objects, located in memory, into a linear sequence of bytes. That sequence of bytes can be sent elsewhere (for example, to a remote computer) and Deserialized, thereby making an exact clone in that remote memory of the original graph of objects.

Because the runtime metadata 'knows' all there is to know about each object's layout in memory, its field and property definitions, you can serialize objects automatically, without having to write code to serialize each field.

The serialized stream might be encoded using XML, or a compact binary representation. The format is decided by the the Formatter object that you call. Pluggable formatters allow the developer to serialize objects in with the two supplied formats (binary and SOAP) or create their own. Below is an example of how to perform default Serialization of a graph of objects, whose root is the arraylist l. This code serializes the graph to a FileStream:

class SerializeExample{

public static void Main(String[] args)
{
   ArrayList l = new ArrayList();
   for (int x=0; x< 100; x++) {
      l.Add (x);
   }                                     // create the object graph
   FileStream s = File.Create("foo.bin");      // create the filestream
   BinaryFormatter b = new BinaryFormatter();   // create the BinaryFormatter
   b.Serialize(s, l);                  // serialize the graph to the stream
} // end main
} // end class

This next code then goes on to creates a clone of the graph by deserializing it. The root of the clone graph is called p:

using System;
using System.IO;
using System.Collections;
using System.Serialization;
using System.Serialization.Formatters.Binary;

class DeSerialize{

public static void Main(String[] args)
{
   FileStream s = File.Open("foo.bin");      // open the filestream
   BinaryFormatter b = new BinaryFormatter();   // create the formatter
     ArrayList p = (ArrayList) b.Deserialize(s);   // deserialize
   p.ToString();                        // print out the new object graph
} // end Main
} // end Class DeSerialize

Serializetion can take place with any stream, not just a FileStream (see MemoryStream, NetStream, etc.)

Serialization makes use of several classes, as follows:

  • Formatter -- Responsible for writing object data in some specified format to the output stream. This class is also responsible for driving the deserialization operation.
  • ObjectIDGenerator -- ObjectIDGenerator generates IDs for objects. It keeps track of objects already 'seen' so that if you ask for the ID of an object, it knows whether to return its existing ID, or generate (and remember) a new ID.
  • ObjectManager -- Keeps track of objects as they are being deserialized. In this way, deserialization can query the ObjectManager to know whether a reference to an object, in the serialized stream, refers to an object that has already been deserialized (a backward reference), or to an object that has not yet been deserialized (a forward reference).

Each of these components is 'pluggable' -- the programmer can provide alternatives.

If a class author wishes to take special action when objects of that class are serialized and deserialized, he can choose to implement the ISerializable interface. This allows full control. For example, the author may define his own, custom SerializationInfo, to describe parts of the object, or synthesized fields that capture the object state for serialization. If a class implements ISerializable, that interface will always be called in preference to default serialization. The ISerializable interface is only contains one method:

   void GetObjectData (SerializationInfo info, StreamingContext context);
And an implied constructor that may be private.
   private <TypeName> (SerializationInfo info, StreamingContext)

An important note on deserialization is that we return the fields in the same order in which they are returned from Reflection. Reflection makes no guarantee that it will follow metadata ordering.


Scenarios in Serialization

Why would you want to use serialization? There are a couple of very interesting scenarios that rely on serialization as componant to it happening.


Persistence

To Persist an object, or collection of objects, means to make them somehow survive beyond the life of the process that holds them in its memory. Conventionally, the objects are saved onto a hard disk, within a file, a storage stream, or a database. (They might also be Persisted into NVRAM, but that’s an uncommon case with regular PC used in the office). However, within the runtime, we will use the terms Persist and Persistence in the stronger sense of saving the objects to disk, and retrieving them afterwards, under the protection of transactions. In this situation, the objects are most likely stored in relational databases, whose table layout does not parallel their layout in memory.

[Outside of the runtime, the word “persist” is commonly used for the simple act of writing data to a durable medium, such as a disk, irrespective of whether that operation is transactioned or not. Wherever there might be a doubt as to what we mean in this spec, we’ll talk about “Transacted Persistence”]

Implicit to Transacted Persistence is the notion that an object retains its unique identity, and that there is only one instance of that object being acted upon at any one moment in time. For example, there can be one and only one object representing the customer called “Guadeloupe McPherson”, of the bank called “Mohavi Mutual” at any instant – it is crucially important that his SSN is 123-45-6789 and his current bank balance is $4,321. If somehow, it were possible to construct two separate instances of “Guadeloupe McPherson” we open the flood-gates of unsynchronized access to data. This contrasts with simple Serialization, where we may store a Windows form into a file, which others can copy into their address space as many times as desired.


Flat File Persistence

Suppose we have built a simple, desktop, single-user application, such as a 2D drafting package. (It might help to think of the “Scribble” tutorial that ships with Visual C++). Suppose too that we have built our application using OO techniques, and represent the graphical objects as a graph of in-memory objects, with one object representing the root of the entire picture.

What we would like, is a painless way to save the entire picture to a disk file, so that it will be there for us tomorrow when we reboot our PC. Before the runtime, each object would have to implement the Serialize (and corresponding Deserialize) method. However, with the runtime, there is really no need to impose this burden on the application programmer. After all, the runtime maintains metadata at runtime that allows serialization code to ‘discover’ the types and values of all fields and properties that make up any object.

So long as the runtime provides a default text formatter (and the current hot favorite is to generate and XML stream), then serializing an entire object graph to a file should be as simple as:

ofstream      ofs = new FileStream (“C:\\MyApp.Dat”);
TextFormatter fmt = new TextFormatter;
fmt.Serialize (obj, ofs);

The obj argument represents the root of the object graph. (Note – this syntax is invented for illustration only. The actual APIs are described earlier – see Section Error! Reference source not found.).

Similarly, in order to start up our App the next day, ideally we need only code a few lines, like this:

ifstream      ifs = new FileStream (“C:\\MyApp.Dat”);
TextFormatter fmt = new TextFormatter;
Obj = fmt.Deserialize (ifs);

Note that the whole point of saving our object graph to a file is so that we can later restore that graph into memory as if no time had intervened between. In order to achieve this aim, it’s clear that we must serialize the entire state of all the objects. We must therefore include every field and every property of each object, regardless of their accessibility – public, protected or private.

It is likely that we will provide at least two Serializer/Formatters – one for text, and one for binary data. In addition, our aim is that ISVs and customers should be given enough information, and supported APIs to enable them to build their own Serializer/Formatters. Therefore, it is likely that each Formatter may prefix each stream with some data that identifies who wrote that stream. This would enable checking when the file came to be Deserialized. Such header information is at the discretion of each Formatter.


Remoting-By-Val

In disributed computing, we may want to execute a method, not within the current thread, but within a thread in a separate process on the same machine, or even a thread in a process on another machine. We speak of this as Remoting the method invocation.

Suppose the method to be Remoted takes, as one of its arguments, an object which lies at the root of a graph of objects. If all objects in the graph are marked as remote-by-val, then we have to Serialize a copy of graph and pass to the remote process.

For efficiency, remoting is investigating whether an object’s fields can be Serialized, without also Serializing a specifier for each field. However, the runtime makes no guarantee that fields will be laid out in memory, in the same order, from one instantiation of an object to the next; even more so if these instantiations lie on different platforms, such as Windows NT versus WinCE.

Is there any way to achieve this? Discussions to-date have thrown up a few possible approaches. For example, we might Serialize the fields in some canonical order – alphabetical by the source code, in strict order of definition in the source code, in order of decreasing field size, etc. Various options have various performance tradeoffs – all of those mentioned so far would likely require a “scatter-gather” approach – where Serialization gathers fields into canonical order, and Deserialization scatters them from that canonical order back to the in-memory layout selected by the runtime for the new object. (Again, we mention these possible approaches here, simply as background to explain the requirement).


Remoting-By-Ref

In some Remoting situations, certain objects may be marked as Remoted-by-ref. In this case, the original object remains where it is. On the remote node, we establish a transparent proxy – it behaves, for all the world, like the real object. However, any accesses of its fields are passed back to the original object.

Serialization itself does not ‘know’ how to encode information that enables Remoting to create that proxy object. This is accomplished through the Remoting service using a Surrogate for that Remote-by-ref object.

Note that in a graph of objects, some may be specified as Remote-by-val whilst others are specified as Remote-by-ref. As soon as Serialization encounters a Remote-by-ref object, it does not walk any subtree that object may have.


Transacted Persistence

Let’s first examine briefly what makes Transacted Persistence such a complex topic. (For more detailed information, see the spec “NGWS Runtime Services: Object Persistence and Serialization”)

Consider a very simple, and imaginary, Sales Order Processing System. A Salesman gathers orders at a Customer site. Later, these are fed into the company computer, which creates objects in memory, representing the situation, as shown below:

This shows a graph of objects, representing a Customer, linked to a Salesman, and to three Orders. The objects have been created by processing the Salesman’s orders, connecting with company databases to check Customer details (billing address, delivery address, credit limit, discount levels, etc), similarly to check stock details (numberof items in stock, price, weights for transportation, etc). The result is to dispatch these orders to the shipping department, with billing information to the Accounts department, etc.

All of this processing has been done under transactional control, with all that entails – locks held on database records to ensure isolation, two-phase commit protocol across multiple resource managers, write-ahead logging to ensure atomicity and recoverability, etc, etc.

The fields of the objects have been populated with values held in various company databases. For example, all fields of the Salesman object may come from one record in the company’s Salesman database. Of course, as the orders are processed, the system may update a TotalOrderValue field of that object, to reflect his monthly commission payment – and this information must be saved when the transaction completes. On the other hand, the Customer object may derive from several database tables – one for Customers, plus another for Addresses. So, populating the Customer object may involve extracting various columns from a Customer row, plus various colums from an Address row, into a buffer, which is then Deserialized into the object.

Conversely, when the transaction completes, serialization of the object graph may be a complex process – selecting some fields of an object to update certain columns of a database table row, with other fields sent to a different destination. Some fields may even be copied twice, if the database schema has been denormalized for performance reasons. Moreover, the backing databases may not all be from the same vendor – for example, a mixture of SQL Server, Oracle, and legacy C-ISAM.

The system must also be sure not to construct two instances of the same Customer in memory – that’s to say, two copies of the object representing that particular customer. Allowing this to happen blows apart the guarantees required for Enterprise transactions, opening the field to lost updates, database inconsistencies, etc, etc. Similarly, when a transaction commits, the object graph must be either destroyed, or at least marked ‘stale’ to safeguard against multiple conflicting versions of the truth.

This situation for transacted persistence becomes even more involved once we consider offline order processing, where disconnected object graphs are operated upon by a remote computer, with the changes only eventually brought back into a transaction and reconciled.

It should be clear from this much-simplified description of transacted persistence, that the requirements it imposes upon the runtime’s simple Serialization service are profound. The solution proposed is to layer the two – so that transacted persistence achieves what it needs to do, by providing its own Formatters and Serializers, but whose structure and interfaces, comply with the model followed by other, simpler, consumers of the Serialization service. Of course, transacted persistence may freely use some base services provided by the layer beneath (eg controlled and efficient access to the private fields of an object), but attempts to-date suggest it is impractical to provide a general-purpose Serialization service that fully meets the requirements of all conceivable scenarios for customer transacted persistence.


Basic Serialization

Class authors need to be aware of serialization. The serialization services assume that a type is NOT Serializable unless type is specifically marked as Serializable. In the most simple case marking a class as Serializable is the all the class author will have to do. For slightly more complex classes with state that is invalid to serialize, we provide support for marking those fields and properties as transient. For the handful of classes that need to participate in their own serialization an Deserialization we provide an ISerializable interface.


The Serializable Attribute

A class must be marked with the Serializable bit to be Serialized. An exception is thrown during serialization if the bit is not set of any class involved in the graph being serialized.

In C#, this bit is set with a reserved custom attribute

[Serializable] public class MyClass {}

Classes with this addribute have all fields (even private ones) serialized.


The NotSerialized Attribute

There are fields and properties on some types that it does not make since to serialize. Either for performance or correctness reasons the class author needs to tell the serialization service to “skip” these members. This is done by using the NotSerialized custom attribute. If the NotSerialized attribute is set the class author is saying this field or property within a class should not be serialized. In C# this bit is set with the transient keyword

[Serializable] public class MyClass
{
   [NotSerialized] int _cashSize;
}


Custom Serialization

Developers may want a way to customize exactly how data from a given object is serialized. For that, the developer should implement the ISerializable interface on the given object. This may be useful for example when some of the data associated with your object may not be valid after deserialization (pointers, hashcodes, etc.) or when you want to create some data (through calculations or some other means) that allows you to reconstruct the full state of the object during deserialization. Implementing ISerializable involves implementing the GetObjectData method on your object and adding a constructor that takes a SerializationInfo and a StreamingContext as shown below.


Implement ISerializable

Having the existence of an interface imply the existence of a constructor is an unusual design pattern, but we felt that it was needed for several reasons. Originally, we had a SetObjectData method on the interface but, like any interface method it was public. Because this method is designed to replace the entire internal state of the object, we didn't feel that it was valid to continue to leave this as a public method. This left us in the state of having the existence of the interface imply the existence of either a method or a constructor. Because, by default, no constructor is ever run on the object, we felt that it was more consistent to have this method be a constructor. The system ignores any visibility restrictions on the constructor, so the developer is free to mark it internal, protected, or even private and thus prevent external users from calling it.

When GetObjectData is called during serialization, the developer is responsible for returning a SerializationInfo. A SerializationInfo is a PropertyBag containing the type of the object being serialized and name/object pairs for the values being serialized. This structure is passed back to a Formatter which emits the data out onto the wire in the method required by its particular format. Developers are free to serialize as few or as many fields as they feel are appropriate, but the data transmitted must be sufficient to reconstitute the entire state of the object. If the base object of the current class implements ISerializable, it is usually correct to call that object's ISerializable.GetObjectData and add any additional fields required for serializing the derived class to the returned SerializationInfo.

During deserialization (the call to the constructor), the developer receives a SerializationInfo which has been populated with the type of the object and the name/object pairs which were transmitted over the stream. From this information, the developer is responsible for completely reconstituting the state of the object. If the base class also implements ISerializable, the developer has the responsiblity for calling the base class' constructor. The serialization infrastructure will delay calling this constructor until the entire SerializationInfo has been completed. If the SerializationInfo which is transmitted references objects A, B, and C, the SerializationInfo passed to the constructor will have been populated with references to objects A, B, and C. However, we do not guarantee that any of the objects referenced by A, B, or C have been completed. This makes it unsafe to call any code on A,B,or C that might require objects which they reference. For some objects, this may include code as simple as GetHashCode(). If your code requires you to do any execution based on the value of data contained in the objects referenced, it is usually best to cache the SerializationInfo and then implement IDeserializationEventListener (detailed below).

using System;
using System.Serialization;

public class ISerializableExample : ISerializable {

    int i, j, k;
    RuntimeTypeHandle th;

    private ISerializableExample(SerializationInfo info, StreamingContext context) {
        //Restore our scalar values.
        i = si.GetValue("i").ToInt32();
        j = si.GetValue("j").ToInt32();
        k = si.GetValue("k").ToInt32();

        //Deserialize the type and then get the TypeHandle from that.
        Type t = (Type)(si.GetValue("TypeObj").ToObject());
        th = t.TypeHandle;
    }

    public SerializationInfo GetObjectData(SerializationInfo si, StreamingContext context) {
        //SerializationInfo is essentially a property bag.

        //Add our three scalar values;
        si.AddValue("i", i);
        si.AddValue("j", j);
        si.AddValue("k", k);

        //TypeHandles are pointers to internal data structures.  They can't
        //be serialized directly, but we can get the type object and serialize that.
        Type t = Type.GetTypeFromHandle(th);
        si.AddValue("TypeObj", t);
        return si;
    }
}


IDeserializationEventListener

If an object implements IDeserializationEventListener, the serialization infrastructure will call that class' OnDeserialization method as soon as the entire graph has been deserialized and all fixups completed. By delaying this callback until as close to the end of the serialization as process, we provide a reasonable opportunity for objects that need to do fixups based on the state of their children. One good example of this is a hashtable, which can make no guarantees about the state of its children until the very end of the graph. By implementing IDeserializationEventListener, the Hashtable can delay running any code on its children (the objects contained within the table) until the entire graph has been deserialized.

//Exerpted from the System.NewCollections implementation of Hashtable.
public class Hashtable : ISerializable, IDeserializationEventListener {
        internal Hashtable(SerializationInfo info, StreamingContext context) {
//We can't do anything with the keys and values until the entire graph has been deserialized
//and we have a resonable estimate that GetHashCode is not going to fail.  For the time being,
//we'll just cache this.  The graph is not valid until OnDeserialization has been called.
            m_siInfo = info;
        }
        public virtual void OnDeserialization(Object sender, DeserializationEvent dsEvent) {
            if (m_siInfo==null) {
                throw new SerializationException();
            }
            loadFactor = (float)m_siInfo.GetValue(LoadFactorName);
            version    = (int)m_siInfo.GetValue(VersionName);
            comparer   = (IComparer)m_siInfo.GetValue(ComparerName);
            hcp        = (IHashCodeProvider)m_siInfo.GetValue(HashCodeProviderName);
            int hashsize = (int)m_siInfo.GetValue(HashSizeName);
            loadsize   = (int)(loadFactor*hashsize);

            buckets = new bucket[hashsize];

            Object [] serKeys = (Object[])m_siInfo.GetValue(KeysName).ToObject();
            Object [] serValues = (Object[])m_siInfo.GetValue(ValuesName).ToObject();
            if (serKeys==null || serValues==null) {
                throw new SerializationException();
            }
            if (serKeys.Length!=serValues.Length) {
                throw new SerializationException();
            }
            for (int i=0; i<serKeys.Length; i++) {
                if (serKeys[i]==null) {
                    throw new SerializationException();
                }
                Insert(serKeys[i], serValues[i], true);
            }

            m_siInfo=null;
        }
}


Custom Formatters

As seen a few times before, the Formatters are the classes that actualy format the information in the SerializationInfo into something that can be written to a stream. The Runtime provides two such formatters, the BinaryFormatter, and the SOAPFormatter. Since formatters are pluggable, developers can build their own if need be. The basic picture is as follows:

The IFormatter interface must be implemented by a developer wishing to write their own formatter. The interface is simple and consists of two methods and three properties:

public interface IFormatter:
{
   //Properties
 SerializationBinder Binder { get; set; }
 StreamingContext Context { get; set; }
 ISurrogateSelector SurrogateSelector { get; set; }
   //Methods
 object Deserialize(Stream serializationStream);
 void Serialize(Stream serializationStream, object graph);
}

By implementing the simple interface above, a custom formatter can be easily written. The PseudoMLFormatter (included in SDK) is a great example of a custom formatter.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Versioning in Serialization

Need info here.


Security in Serialization

Security is always a concern, so in the context of serialization, we’ll hilight two issues that should be specificly considered.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Permissions

Need info here.


Issues to consider

The serialization engine handles both the public and private state of the objects passed to it. When serializing an object to a stream, one must remember that the stream now also containes the public and private data of the object. If the private data is sensative, the stream should be treated with particular care (for instance, it should not be transmitted over the wire, or persisted to disk without some form of encryption).


Other Security Notes

Note that fields within a class have traditionally been declared non-public in order to improve data encapsulation. That’s to say, it was an issue of good software engineering, rather than confidIf the programmer chose to persist that information into a file, then the only security issue that really enters the picture, is whether the caller (or the installed software component itself) has CreateFile privilege for the target drive and folder.entiality of customer data (such as account balances, or salary amounts) that prompted a programmer to use private or protected fields.

However, in the wider world where one programmer writes a component, which is used by another department or another company, the notion of non-public fields provides a natural route to protect confidential information (so long as access to non-public fields is controlled by some security permission). For this reason, Serialization or Deserialization of non-public fields, will require a “Serialization” security permission.

Note that with this security model, programmers need to be aware that all properties and public fields, could be serialized and ‘extracted’ from the address space in which they reside, without that programmer’s knowledge, and without any special security permissions.


Shared Assembly Names

This specification describes the support for shared assembly names in the NGWS runtime. Simple, friendly names for assemblies are easy and convenient to use and are sufficient in a number of cases. However, there are a number of situations where developers must be able to guarantee that the names they choose for their assemblies will be globally unique and that the names they choose cannot be spoofed either accidentally or maliciously. The runtime solves the problem of name uniqueness and name security using standard public-key cryptography. Names secured with a key-pair are termed shared names.


Overview of Shared Assembly Names

Part of an assembly's identity is a simple, friendly textual name (like My401kApp). These names are given to the assembly by the developer when the assembly is authored. These names are referred to as simple names. Simple names are not guaranteed to be unique - there is nothing in the tools or in the runtime to prevent collisions.

Simple names for assemblies are easy and convenient to use and are sufficient in a number of cases. In particular, individual developers or small development shops have no need for a more sophisticated naming scheme and we shouldn't complicate their lives for no obvious benefit. However, there are a number of situations where developers must be able to guarantee that the names they choose for their assemblies will be globally unique. In COM, uniqueness was guaranteed by assigning a unique GUID to each component. We are not carrying this notion forward into the runtime because GUIDs have a number of deficiencies that we would like to correct. In particular, while GUIDs are unique when they are generated, nothing prevents another developer from later reusing your GUID and "substituting" her code for yours.

The runtime solves this problem of name uniqueness and name security using standard public-key cryptography. Names secured with a key-pair are termed shared names.

In particular, shared names satisfy the following requirements:

  1. Shared names guarantee name uniqueness by relying on unique key-pairs
  2. Shared names prevent others from "taking over your namespace". Because only you have your private key, no one can generate the same name that you can. The 401k assembly generated with my public key has a different name than the 401k assembly generated with your private key.
  3. Shared names provide some notion of identity. If the runtime crypto checks pass, you can be guaranteed that the assembly comes from the person you thought it did (assuming his private key wasn't compromised, of course). Note however that shared names in and of themselves do not provide a level of trust. Trust is accomplished through full digital signatures complete with certificates.

In addition, the runtime's security team also has the following requirements for shared names:

  1. Shared names can not tie the runtime to using Authenticode technology
  2. Shared names can not require that only signed code can be granted permissions. The security team is investigating several scenarios where it is useful to grant permissions to unsigned code. For example, you may want to grant permissions to code that was retrieved through a secure channel.


Assigning and Referencing Shared Names

In general, the process of giving an assembly a shared name and later referencing it by that name works as follows:

  1. Alice wishes to give her new assembly a shared name. She encrypts the hash of the file containing her assembly's manifest with her private key. This "shared name blob" is stored in the PE file containing the assembly manifest.
  2. When referring to Alice's assembly, Bob includes a "token" that represents Alice's public key in the reference in his assembly's manifest. We store a "token" of the public key rather than the key itself solely for space reasons.
  3. When Alice's shared named assembly is deployed, the Runtime verifies the shared name signature.
  4. When binding by shared name at runtime, the runtime compares the key stored in the Bob's manifest with the key Alice used to generate her shared name. If the crypto checks pass and the bind succeeds, Bob can determine that the bits he is getting haven't been tampered with and that they do in fact come from Alice.

As mentioned above, this scheme doesn't make any statements about trust.


Digital Signatures and Shared Names

Assemblies can carry full PKCS standard compliant Digital Signatures. These signatures include a certificate that establishes trust. It is important to note that shared names do not require code to be signed in this way. In fact the keys used to secure the assembly's name doesn't even have to be the same key used to sign the assembly itself.


Cryptographic wrapper APIs

To facilitate the implementation of shared names across tools, and the runtime, we define simple wrapper APIs to expose the needed cryptographic functionality. For the V1 release, these APIs are unmanaged wrappers for Microsoft CryptoAPI functionality (see the Microsoft Platform SDK). In particular, we abstract out much of the complexity associated with using Cryptographic Service Providers (CSPs), key management, and digital signature/verification operations. This also represents a set of APIs that may be ported to other platforms, not supporting CryptoAPI, for future shared name support.

Shared name generation requires support for public/private key pair generation, hashing, digital signature, and signature verification on development platforms. Runtime platforms will require hashing and signature verification only. The APIs are factored to allow tailoring of support for development and runtime systems.

Applications using these wrappers should be aware that APIs supporting key generation and digital signature generation should not be further wrapped to expose them as COM, or other scriptable, interfaces. This would open the possibility for Web scripts to create valid digital signatures on behalf of the user without their consent. Unmanaged, hostile, code can also do this, but it can do so by calling CryptoAPI directly. These wrapper APIs do not increase this potential vulnerability.

To simplify use of the shared name functionality, we wish to abstract out the complexity associated with using CryptoAPI directly. To do this, we make the following simplifying assumptions:

  1. A pre-designated CSP will be used for all crypto operations. The default will be “Microsoft Base Cryptographic Provider v1.0”.
  2. The RSA signature algorithm will be used.
  3. The SHA-1 hash algorithm will be used.

There will be no provision in the APIs for overriding these settings. We will however, store this information in a known location (Registry on Win32 systems at HKLM\Software\Microsoft\StrongName). It will be possible to change the default settings, but we will not provide UI to support this operation.

The CryptoAPI wrapper functions will be provided as unmanaged C-code interfaces only. Functions will return TRUE if successful, or FALSE if an error occurred. Extended error information is available by calling the function StrongNameErrorInfo() after an error, which returns a DWORD. If CryptoAPI functionality is not available, then we will return the error

(0x80131413) SN_CRYPTOAPI_CALL_FAILED.

If the specified Cryptographic Service Provider (CSP) can’t be located then we return the error

(0x80131414) SN_CSP_NOT_FOUND.

For all other failures, this value will contain the DWORD returned by CryptoAPI using GetLastError(). Future versions may extend or alter the error codes returned.


Shared Name Cryptographic Key Management

A set of helper APIs are provided to support key generation and management by compiler vendors. These are designed to provide the flexibility needed to address common development scenarios while still being easy to use.

Using these APIs, it is possible for compilers to:

  • Generate shared name cryptographic key pairs and retrieve a representation (portable key blob) that can be easily stored and moved across development platforms.
  • Associate a shared name key cryptographic pair with a project and use it for shared name generation. The key pair need not be permanently installed into the CSP used for shared name generation so that use of distinct key pairs for each project can be easily accommodated.
  • Install a key pair into the CSP used for shared name generation such that it can easily be referred to for shared name generation with any project.

In addition, the design provides assurance that shared name key operations will not impact the use of cryptographic functionality by other applications on the user’s machine.

Typical scenarios supported include:

  • Creating a shared name cryptographic key pair that may be associated with one or more projects. This is expected to be the common mode of operation. The key pair is generated using the StrongNameGenerateKey()API and the portable key blob returned. The key pair will generally not be left installed in the shared name CSP. The portable key blob, and its associated container name (or friendly name), can be stored as a file in the source control system or in the project data. If stored as a file, a standard naming convention of “<container name>.snk” is recommended. This will make it easy to locate and move test keys across development environments. When the test key is no longer needed, the key blob may simply be discarded.
  • Creating a shared name cryptographic key pair and storing it on the users machine. To do this one would follow the procedure outlined above. However, at key generation time the key generation flag parameter can be set to SN_LEAVE_KEY. This will leave a copy of the key pair installed in the shared name CSP. The development environment should store the container name for this key so it can be easily referred. Note that the key can be left installed in the CSP for default use and also stored in portable form.
  • Changing the default shared name key cryptographic pair on a user’s machine. This assumes the development environment has a portable key blob it wishes to make the default key for shared key operations. The StrongNameKeyInstall() API is used to load the portable key blob in the shared name CSP. The development environment should keep track of the container name for this key. If no default key was installed prior to this operation, then we simply install the new key pair. However, if a default key was previously installed, StrongNameKeyDelete() should be called with the container name of the old default key prior to installing the new default key. This will prevent the accumulation of a large number of key pairs on the user’s machine and minimize the probability of container name clashes.
  • Creating a deployment shared name cryptographic key pair. There is no difference in the basic operations for creating a deployment and test shared name key. But, as a rule, one would not want to keep deployment keys in a portable form accessible to large numbers of developers. It is recommended that a single machine be used for “real” signing and that the deployment keys be left installed in the CSP. If the portable key blob is stored for backup purposes, it should be isolated and physically protected. For organizations that desire high assurance no one will forge names in their shared name namespace, they may wish to use a hardware CSP (available from several third parties). In most cases, these devices will not export the key pair, but the provided APIs can still be used to generate the key(s) desired.

In using these APIs, the following cautionary issues should be considered.

  • To insure the portable key blobs can be easily transferred between development machines, they are not encrypted upon export from the CSP. This means anyone will be able to install and use the key pair. This is not a serious concern for test keys, but is an issue for deployment keys.
  • We do not enforce uniqueness on key container names. Hence, it is possible for developers to create multiple key pairs using the same container name. This could lead problems since container names are the easiest to refer to key pairs. For example, one developer could create a shared name using one set of keys while a second developer could pick up the different key pair when creating a shared name reference. In such a case, it may not be immediately obvious why the shared name reference isn’t properly resolved. Tools UI can easily help in preventing such problems.
  • The two primary objectives of shared names are to (1) create unique names and (2) prevent name spoofing. The cryptographic keys are required only to make the latter very difficult. A developer’s private key is used to prove they have the right to create new names in their private namespace. It is not used to prove identity or any other properties of the developer. Given this intent, one should make a distinction between a shared name key pair and key pairs used for digital identity or key exchange purposes. By convention, the key pairs used for creating shared names are stored in files with a .snk extension.

Key Management APIs

The following function will create a new key pair for shared name use and export it to the calling program.

   BOOL StrongNameKeyGen(
      // [in] desired key container name, must be a non-empty string
      LPCWSTR         szKeyContainer,
      // [in] flag parameter, leave key installed is only option
      DWORD            dwFlags,
      // [out] public/private key blob
      BYTE            **ppbKeyBlob,
      ULONG            *pcbKeyBlob)

The container name is a user defined string. CryptoAPI does not specify a size limit on this string, but individual CSPs may. It should generally be a short descriptive name. We allow user defined names so that multiple shared name keypairs (e.g., project1 vs project2, test vs deploy) can be managed. The dwFlags parameter is used to control optional actions. The only defined flag for this version is SN_LEAVE_KEY (value is 1). If set, then the key pair will be left installed in the shared name CSP.

This function will attempt to create a new signing keypair using the supplied container name. The CSP to use, and signing algorithm, are determined by the default settings described earlier.

The generated key will be marked exportable (if supported by the CSP) and the public/private keys exported in an unencrypted keyblob. This is returned to the caller. The keyblob may be stored in any way convenient for the calling application. If modified, it will no longer be usable.

To simplify use of this, and other functions, we will allocate the buffer to hold the protable key blob. The calling program should call StrongNameFreeBuffer() when the buffer is no longer needed to insure its properly de-allocated.

The following function may be used to install a portable key blob into a machine CSP. Its purpose is to support installation of default shared name keys.

   BOOL StrongNameKeyInstall(
      // [in] desired key container name, must be a non-empty string
      LPCWSTR        szKeyContainer,
      // [in] public/private key blob
      BYTE            *pbKeyBlob,
      ULONG            cbKeyBlob)

This function attempts to install the key pair provided in the key blob into the shared name CSP using the provided container name. If the container name already exists or an invalid key blob is supplied, an error is returned.

The following function may be used to delete a shared name key pair from the CSP. This will only be required if the SN_LEAVE_KEY option was selected at key generation time, or the key was subsequently installed loaded into the CSP. UI should make it very clear to the user that this will result in permanent deletion of the keys. If the corresponding portable key blob was not stored, they will be unable to create any future shared names in the associated namespace.

   BOOL StrongNameKeyDelete(
      // [in] desired key container name
      LPCWSTR         szContainer)

This function will acquire the shared name CSP context for the given key container name with the flag CRYPT_DELETEKEYSET.

The following function is used to free memory allocated by the shared name functions.

VOID StrongNameFreeBuffer(
   // [in] pointer to memory to be freed
   BYTE            *pbMemory)

pbMemory should only refer to memory allocated by the shared name functions when return key blobs to the calling program.


Defining a Shared Name

To securely define a shared name namespace, the PE file containing the Assembly metadata is signed with the private key corresponding to the public key that defines the namespace. This operation requires several steps.

The development environment is expected to perform the following operations in sequence in generating a shared name.

  1. The Shared Name public key blob is retrieved. Space is reserved in the metadata section to hold the key blob and it is inserted. The shared name public key blob should also be stored by the project system so that it can be used when creating references in other assemblies.
  2. The size of the shared name signature blob is retrieved. Space is allocated in the PE file to hold the signature and information in the NGWS runtime file header set to point to this space.
  3. The PE file is saved to disk.
  4. The Shared Name signature blob is then generated. The calling program provides the name of the PE file and the shared name keys to use. The signature function will compute the signature and, optionally, insert it into the defined section in the PE file.

The below APIs support these operations. In each function, the shared name key pair may be provided as a portable key blob, or a container name reference to a key pair already installed in the shared name CSP. The shared name CSP, hash algorithm, and signing algorithms based on the default machine settings.

This method below supports retrieval of the public key blob.

   BOOL StrongNameGetPublicKey (
      // [in] desired key container name
      LPCWSTR         szContainer,
      // [in] public/private key blob (optional)
      BYTE            *pbKeyBlob,
      ULONG            cbKeyBlob
      // [out] public key blob
      BYTE            **ppbPublicKeyBlob,
      ULONG            *pcbPublicKeyBlob)

This function will attempt to acquire the shared name CSP context with the provided szContainer name. If pbKeyBlob is NULL, then it is assumed the key pair is already installed in the CSP so the container must exist and it must contain a public key pair associated with a signing algorithm.

If pbKeyBlob is non-NULL, then the CSP is acquired with CRYPT_NEWKEYSET. In this case, an error is returned if the container name exists. If we acquire the CSP successfully, then the key pair in key blob will be imported. In this case, the key pair will be deleted before the function returns.

We will allocate the buffer to hold the public key blob and return a reference to it in ppbPublicKeyBlob. This buffer should be freed by calling StrongNameFreeBuffer().

The returned PublicKeyBlob with be encoded as:

   Struct {
      ALG_ID   SigAlgID;     // strong name signature algorithm
      ALG_ID   HashAlgID;    // strong name hash algorithm
      ULONG      cbPublicKey; // length of the key in bytes
      BTYE[]   PublicKey;    // variable length byte array
                              containing the key value in format
                              output by CryptoAPI
   } PublicKeyBlob;

The following method supports the signature operations. It can be used to retrieve the size of signature blob so that space in the PE file can be allocated. It is then used to generate the signature. The function will optionally write the signature directly into the PE file or return the signature blob to the caller.

     BOOL StrongNameSignatureGeneration(
      // [in] valid path to the PE file for the Assembly
      LPCWSTR         szFilePath,
      // [in] name of key, if NULL will default to StrongNameKey
      LPCWSTR        szKeyContainer,
      // [in] public/private key blob (optional)
      BYTE            *pbKeyBlob,
      ULONG            cbKeyBlob
   // [in/out] signature blob
      BYTE             **ppbSignatureBlob,
      ULONG          *pcbSignatureBlob)

This function will attempt to acquire the shared name CSP context with the provided szContainer name. If pbKeyBlob is NULL, then it is assumed the key pair is already installed in the CSP so the container must exist and it must contain a public key pair associated with a signing algorithm.

If pbKeyBlob is non-NULL, then the CSP is acquired with CRYPT_NEWKEYSET. In this case, the container name should not exist. If we acquire the CSP successfully,then the key pair in key blob will be imported. In this case, the key pair will be deleted before the function returns.

If szFilePath is NULL, it indicates that only the size of the signature blob should be returned. This is returned in the location pointed at by *pcbSignatureBlob.

If szFilePath is non_NULL, then we then hash the PE file and sign the hash signed using the private key. Hash computation excludes some sections of the PE image (section reserved for the signature blob, file CRC, and so on) that will change once the signature blob is added to the file.

If ppbSignatureBlob is NULL, this function will insert the signature blob into a reserved section in the PE and perform any required PE fix-ups (e.g., CRC). If the caller wishes to handle the signature blob and its insertion into the PE file, then ppbSignatureBlob should be a valid pointer to a pointer variable.

If the signature blob is returned to the calling program, the buffer holding it will be allocated by this function. When no longer needed, it should be freed by calling StrongNameFreeBuffer().

The following method returns the size of the signature blob given only the public key. This is to support the “Deferred Signing” and “Test Signing” approaches described below.

SNAPI StrongNameSignatureSize(
 BYTE    *pbPublicKeyBlob, // [in] public key blob
 ULONG    cbPublicKeyBlob,
 DWORD   *pcbSize);        // [out] size of the signature in bytes


Creating the token for a shared name

We use a fixed size “token” to refer to a shared name namespace. This is computed using a hash function from the public key associated with the namespace. This hash operation is used to:

  • reduce the size of the token relative to the key size
  • provide high assurance of a unique token (statistically unique)
  • and make it extremely difficult to forge a shared name.

Note that by using this approach, forging a shared name requires finding a public/private key pair that will generate a known token. This is extremely difficult given modern message digest algorithms.

The SHA-1 has function has the desired properties and is readily available since we assume its use in generating the shared name. A drawback to SHA-1 is that it generates a 160-bit value. This is much smaller than typical RSA signature keys (512 bits or larger), but is larger than the key size typically used for some other algorithms such as ECC. To reduce the size of the token, the low-order 64-bits of the SHA-1 hash will be used as the token value. This has adequate strength for our needs. This token size should not be assumed to always be 64-bits as it may be necessary to change this in the future.

To create a token, the public key blob is normally extracted from the referenced Assembly manifest in a PE file (the PublicKeyBlob structure in the metadata). For some scenarios, it is also desirable to generate the token from a public key blob structure. Both methods are supported.

   BOOL StrongNameTokenFromAssembly(
      // [in] valid path to the PE file for the Assembly
      LPCWSTR         szFilePath,
      // [out] strong name token
      BYTE             **ppbStrongNameToken,
      ULONG          *pcbStrongNameToken)

   BOOL StrongNameTokenFromPublicKey(
      // [in] public key blob
      BYTE            *pbPublicKeyBlob,
      ULONG            cbPublicKeyBlob)
      // [out] strong name token
      BYTE              **ppbStrongNameToken,
      ULONG           *pcbStrongNameToken)

We return both the token and its size for future flexibility. The 64-bit token size may need to be increased at some point in the future, so tools should not hard code in an assumed token size.

As with other functions, the buffer to hold the token will be allocated by the function. It should be freed when no longer needed by calling StrongNameFreeBuffer().


Verifying a shared name

A shared name is verified by using the shared name public key to verify the shared name signature blob (see 1.2.2). This operation requires a reference to the PE file containing the Assembly manifest.

If this function returns TRUE, then the signature is verified, otherwise it returns FALSE and the shared name should not be respected.

   BOOL StrongNameSignatureVerification(
      // [in] valid path to the PE file for the Assembly
      LPCWSTR               szFilePath)

SzFilePath must be valid reference to a PE file and it must contain: 1) a PublicKeyBlob structure in the metadata; and 2) a shared name signature blob in the section of the PE file reserved for this purpose.

The function computes the hash of the PE file using the same algorithm used in StrongNameSignatureGeneration(). Verification involves operations on this hash, the signature, and public key that depend upon the algorithm being used.


Creating Assemblies with Shared Names

Tools assign shared names to assemblies using the crypto helper API's and the assembly metadata API's. The basic steps are as follows:

  1. A developer indicates that he'd like to give the new assembly a shared name. Presumably, this is either done through a dialog in the IDE or as a command line switch to the compiler/linker.
  2. The tool should allow the developer to designate the key pair to use for the shared name. This may be:
    • an existing key pair stored in a file using the portable key blob format
    • the default key pair installed on the machine, the tool should know the container name of this key
    • a new key generated by a call the the Crypto helper API StrongNameKeyGen
  3. The tool must reserve space in the PE containing the manifest for the shared name signature and set the StrongNameSignature field in the COR Header to point to this space. The size required for the signature can be determined using StrongNameSignatureGeneration.
  4. When generating the manifest, the tool passes in the public key associated with the shared name as a parameter to IMetadataAssemblyEmit::DefineAssembly (the parameter is called Originator).
  5. After the assembly has been built, the tool needs to generate the shared name and store it in the PE file containing the manifest. This is done by calling StrongNameSignatureGeneration. This function will insert the shared name signature blob into the StrongNameHash section reserved in the PE by default.


Referencing Assemblies with Shared Names

References to assemblies with shared names (AssemblyRefs) contain a "token" that represents the public key used to generate the shared name in the referenced assembly. This token is used at bind time to help locate the desired assembly.

A development tool obtains the token for a public key using either StrongNameTokenFromAssembly or StrongNameTokenFromPublicKey.

This token is then passed as the originator parameter to IMetadataAssemblyEmit::DefineAssemblyRef when the AssemblyRef is defined.


Runtime Verification of Shared Names

The NGWS runtime verifies the shared name signatures at the following points (using StrongNameSignatureVerification):

If an assembly is deployed to the global assembly cache, the signature is verified when the assembly is placed in the cache.

If an assembly is deployed to the application directory (or a subdirectory thereof), the signature is verified when security policy is evaluated IF there is any policy defined based on the shared name corresponding to the public key stored in the assembly manifest. Also, for Beta2 we are considering adding a configuration setting that will force shared names to always get verified when an assembly is loaded.


Binding to Assemblies with Shared Names

Since verification is done a cache-load time, binding to an assembly with a shared name only requires a key equivalence check. The shared name token recorded as part of the referenced is included in the cache lookup.


Key Management during Development

The process described above for generating shared names assumes that the developer has access to the key pair that will be used to generate the shared name. This is not always the case. In many situations, an organization will have a closely guarded key pair and will not be willing to let developers have copies of this key pair to use on a daily basis. Of course, the public key will be available, but the private key is often kept in safe keeping and only a few select people have access to it.

A similar problem exists for Authenticode signing, but in that case the entire signing process can be done after everything is built and right before shipment. This is not the case with shared name signing – each assembly reference contains the (token of the) public key used to give the target assembly a shared name. This requires that at least the public key be available during the development process.

There are numerous solutions to this problem, two of which are described below. Internally at Microsoft, we use “Deferred Signing”.


Deferred Signing

Deferred signing refers to the process of reserving space in the PE file for the shared name signature at build time, but the actual signing is deferred until some later stage - typically right before the assembly is shipped. The process works as follows:

  1. The developer must get the public key portion of the key pair from the organization in the company that will do the eventual signing. Typically this key is in the form of an “snk” file as described above.
  2. The key is given to the compiler with a command line switch. For example, in C# this switch is /a.keyfile: In addition the compiler is typically given a switch indicating the assembly should be deferred signed.
  3. The compiler inserts the public key into the manifest using IMetadataAssemblyEmit::DefineAssembly and reserves space in the PE for the signature. Again, the real public key must be stored while the assembly is built so that other assemblies that reference this one can get the public key to store in their references. Given just the public key, the compiler can determine the size of the signature using StrongNameSignatureSize.
  4. Because the assembly doesn’t have a valid signature, the verification of that signature must be turned off. This can be done for all assemblies on the machine (using SN –Vr*), or for particular assemblies (using SN –Vr).
  5. At some later point, the assembly is handed off to the organization’s signing authority for “real” signing (using SN –R).


Test Signing

Test signing refers to the process of signing your assembly with a test key during development, then replacing this signature with one done with the real key prior to deployment. The test key is a non-secure key that can freely be shared – the signature based on the test key won’t be part of the assembly when it is deployed. As with deferred signing, test signing requires access to the real public key at build time. Test signing works as follows:

  1. The developer must get the public key portion of the key pair from the organization in the company that will do the eventual signing. Typically this key is in the form of an “snk” file as described above. In addition to the real public key, the developer needs a test key pair. This is typically generated with SN (the –k option).
  2. The key is given to the compiler with a command line switch. For example, in C# this switch is /a.keyfile: In addition the compiler (or other tool that is building the assembly – link Alink.exe) is also given a switch that specifies the test key pair to use for signing.
  3. The compiler inserts the public key into the manifest using IMetadataAssemblyEmit::DefineAssembly and calls StrongNameSignatureGeneration with the test key to emit the test signature. Again, the real public key must be stored while the assembly is built so that other assemblies that reference this one can get the public key to store in their references.
  4. When verifying the shared name signature, the NGWS runtime must use the test public key because the signature in the assembly will not verify using the public key in the manifest (it’s the real one, not the test one). SN contains an option (-V) for registering the test public key as a “fallback” key. When such a key is registered, the NGWS runtime will use this key to verify the signature if verification fails using the key in the manifest.
  5. At some later point, the assembly is handed off to the organization’s signing authority for “real” signing (using SN –R). Real signing replaces the test signature with a signature generated using the real key pair.


System.Reflection.Emit Overview

This document describes a set of managed Class Library APIs in the System.Reflection.Emit namespace. The APIs allow a compiler or tool to emit metadata and IL and optionally generate a PE file on disk. The primary users of these APIs are script engines and compilers.

The Reflection Emit APIs provide the following services:

  • Ability to define assemblies at runtime and then run and/or save them to disk.
  • Ability to define modules in existing assemblies or in new assemblies at runtime and then run and/or save them to disk.
  • Ability to define types at runtime, create instances of these types, and invoke the type’s methods.
  • Ability to define symbolic information for defined modules for use by tools such as debuggers and code profilers.

The Metadata Emit interfaces described in Metadata Emit/Import – Interface Specification provide an alternative set of unmanaged APIs for defining metadata. The Reflection Emit APIs provide stronger semantic error checking and higher level of abstraction of the metadata than the Metadata Emit interfaces.


Related Specifications

  • The Virtual Object System (VOS) specification, which defines the object model that underlies the runtime, its externalization in metadata, and its implications for the runtime.
  • A Guide for Compiler Writers, which describes information useful to a compiler writer.
  • The Metadata Emit/Import – Interface Specification, which defines unmanaged APIs for importing and exporting metadata. These APIs are intended for use by compilers, loaders, and other low-level tools that require fast access to metadata.
  • The Metadata Validation specification, which defines the conditions that ensure the metadata is well-formed.
  • The System.Reflection interfaces, which are managed Class Library APIs for obtaining runtime information about assemblies, modules, types, etc. Most classes and interfaces in the Reflection Emit namespace extend classes and interfaces in the Reflection namespace.
  • The Runtime Signature specification, which defines the binary encoding for variable types and signatures.
  • The Common Language Subset specification, which places a number of modeling restrictions on the metadata. The metadata design accommodates but does not explicitly enforce CLS rules.
  • The Unmanaged Symbolic Information Interfaces specification, which describes the unmanaged interfaces for reading and writing symbolic information. The managed versions of these interfaces are defined in the System.Diagnostics.SymbolStore namespace.


Reflection Emit Abstractions

The Reflection Emit abstractions correspond closely to the object model that underlies the runtime as described in the Virtual Object System (VOS) specification,. In the following sections, the abstractions are described in alphabetical order.

AssemblyBuilder. An assembly is a configured set of loadable code modules and other resources that together implement a unit of functionality. Assemblies themselves have names that form the basis of the VOS naming hierarchy; see Assembly Overview for details.

This document classifies assemblies as static or dynamic. A static assembly is an assembly that is loaded from disk. A dynamic assembly is an assembly that is created using the Reflection Emit APIs.

The AssemblyBuilder class is used to define and represent a dynamic assembly. The AssemblyBuilder class inherits from the Assembly class.

ConstructorBuilder. A constructor describes an operation that instantiates a type. ConstructorBuilder is used to define and represent a constructor. The ConstructorBuilder class inherits from the ConstructorInfo class.

CustomAttributeBuilder. The CustomAttributeBuilder class is a helper class used to define custom attributes.

EnumBuilder. EnumBuilder is a helper class used to define and represent an enumeration type. The EnumBuilder class inherits from the Type class.

EventBuilder. Events of an object type specify named state transitions in which subscribers can register or unregister interest via accessor methods. The EventBuilder class is used to define and represent an event. The EventBuilder class inherits from the EventInfo class.

FieldBuilder. A field is a named subdivision of a value. A type can have field members. A module or a type can also reference fields defined in the .sdata section of a PE file. The FieldBuilder class is used to define and represent a field. The FieldBuilder class inherits from the FieldInfo class.

ILGenerator. The runtime JIT compiler translates instructions in the Intermediate Language (IL) to native code. ILGenerator is used to define and represent IL.

Label. The Label class is an opaque representation of a label used by the ILGenerator class. Labels correspond to specific locations in the IL code.

LocalBuilder. The LocalBuilder class represents a local variable declared within the body of a method.

MethodBuilder. A method describes an operation that can be performed on values of a type. MethodBuilder is used to define and represent a method. The MethodBuilder class inherits from the MethodInfo class.

ModuleBuilder. A module is a compilation unit, an executable unit, a development unit, a deployment unit, or a runtime unit. Modules can have global fields. A module created using the Reflection Emit APIs is called a dynamic module for the duration of the run in which the module was created. The term dynamic module does not apply if the module is saved to disk and then loaded from disk. Dynamic modules that can be saved to disk as a PE file are called persistable dynamic modules. Dynamic modules that exist only in memory only are called transient dynamic modules. The ModuleBuilder class is used to define and represent a module. The ModuleBuilder class inherits from the Module class.

ParameterBuilder. Parameters are locations that represent arguments of methods or constructors. The ParameterBuilder class is used to define and represent a parameter. The ParameterBuilder inherits from the ParameterInfo class.

PropertyBuilder. Properties of an object type specify named values that are accessible via accessor methods that read and write the value. The PropertyBuilder class is used to define and represent a property. The PropertyBuilder class does not inherit from the PropertyInfo class. This will be fixed in a future release.

TypeBuilder. Types describe values. The type defines the allowable values and the allowable operations supported by the values of the type. Types can have members that are types, methods, fields, properties, or events. The TypeBuilder class is used to define and represent a type. The TypeBuilder class inherits from the Type class.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Reflection Emit Scenarios

© 2000 Microsoft Corporation. All rights reserved.


Reflection Emit Core Scenarios

This section describes the core scenarios supported by Reflection Emit.

  1. Dynamic module defined in the caller’s static assembly. In this scenario, a dynamic module is defined in a static assembly that contains the code that creates the dynamic module. The dynamic module created this way alters the runtime representation of the assembly’s manifest. Since the disk representation of the manifest cannot be altered, the dynamic module is transient and cannot be persisted. While the dynamic module can add new COMTypes to the runtime representation of the manifest, it cannot override COMTypes that were originally defined in the manifest.
  2. Dynamic module defined in a static assembly that is not the caller’s assembly. In this scenario, a dynamic module is defined in a static assembly that is different from the one that contains the code that creates the dynamic module. A dynamic module created this way is transient and the restrictions that apply to the previous scenario also apply to this scenario.
  3. Dynamic module defined in a dynamic assembly. In this scenario, a dynamic module is defined in a dynamic assembly. A dynamic module created this way can be transient or persistable.

Restrictions on Type References

Assemblies can reference types defined in another assembly. A transient dynamic assembly can safely reference types defined in another dynamic transient assembly, a persistable dynamic assembly, or a static assembly. However, the runtime will not allow a persistable dynamic module to reference a type defined in a transient dynamic module. This is because when the persisted dynamic module is loaded subsequent to being saved to disk, the runtime will not be able to resolve the references to types defined in the transient dynamic module.

Restrictions on Emitting to Remote Application Domains

Some scenarios require a dynamic assembly to be created and executed in remote application domain. Reflection Emit does not allow a dynamic assembly to be emitted to a remote application domain. The workaround is to emit the dynamic assembly in the current application domain, save the emitted dynamic assembly to disk, and then load the dynamic assembly into the remote application domain.

Dynamic Assembly Access Modes

Dynamic assemblies can be created using one of the following access modes:

  • Run. The dynamic assembly represented by an AssemblyBuilder object is transient. The dynamic assembly can only be used to execute the emitted code.
  • Save. The dynamic assembly represented by an AssemblyBuilder object is persistable but cannot be executed until the saved PE has been loaded from disk.
  • RunAndSave. The dynamic assembly represented by an AssemblyBuilder object is persistable but can also be executed before and/or after the assembly is saved to disk.

The access mode needs to be specified when the dynamic assembly is defined and cannot be changed at a later point in time. The runtime uses the access mode of a dynamic assembly to optimize the assembly’s internal representation.


Reflection Emit Application Scenarios

This section describes some application scenarios for Reflection Emit.

Executing a Script in a Web Browser

In this scenario, a Web browser loads an HTML page containing a script program and executes the program. The steps are as follows:

  • The Web browser launches the script engine in the Web browser’s application domain.
  • The Web browser extracts the script from the HTML page and passes it the script engine.
  • The script engine creates a transient dynamic assembly.
  • The script engine uses the Reflection Emit APIs to emit the code into the dynamic assembly.
  • The script engine loads the dynamic assembly into a newly created application domain and executes the application domain. The code is executed in a separate application domain to isolate the code from the script engine.

Executing a Script in an ASP+ Page

In this scenario, an ASP+ page containing a script program is downloaded to a Web browser for the first time. The steps are as follows:

  • ASP+ starts its application domain and launches the script engine to compile and execute the program.
  • The script engine creates a persistable dynamic assembly and a persistable dynamic module in the dynamic assembly.
  • The script engine compiles the program using the Reflection Emit APIs and emits the code into the persistable dynamic module.
  • The compiled code is run and an eval expression is encountered.
  • The script engine creates a transient dynamic module in the dynamic assembly to contain the code for the eval expression.
  • The script engine evaluates the eval expression by executing the transient dynamic module.
  • ASP+ requests the script engine to save the dynamic assembly and the persistable dynamic module in the cache so that the script does not need to be recompiled when the same page is visited again.

Compiling a Regular Expression

In this scenario, a compiler uses the Reflection Emit APIs to compile regular expressions in source code. A user can declare which regular expressions should be compiled.

  • The compiler processes the first use of a regular expression in a user’s source code.
  • The compiler compiles the regular expression into a custom scanner class using the Reflection Emit APIs since the user has requested that the expression be compiled. The regular expression is first translated to the compiler’s regular expression bytecodes. For example, one instruction might say, “try to match zero or more a’s.” The regular expression bytecodes are then translated to IL. For example, the compiler might generate a loop that iterates over all the a’s that it found.
  • The compiler saves the compiled scanner class for future use.
  • The compiler instantiates the compiled scanner class and executes the code in the current application domain.
  • The compiler processes a subsequent use of the same regular expression in the user’s source code.
  • The compiler retrieves the compiled scanner class for the regular expression, instantiates it, and executes the code in the current application domain.

The user incurs a performance penalty when the first instance of a regular expression is used because the compiler needs to compile the expression into a scanner class. However, subsequent uses of the same regular expression are executed efficiently. For regular expressions that are used frequently, the one-time compilation penalty is insignificant since 30-500% speedup in execution time is possible.


Security Issues in Reflection Emit

This section describes the security issues to be aware of when creating dynamic assemblies using the Reflection Emit APIs. Some scenarios require specific permissions. Other scenarios require permissions to be established for generated code. In the description below, caller refers to the creator of the dynamic assemblies and/or dynamic modules.

Scenarios Requiring Specific Permissions

  • The runtime requires the authoring assembly to have the ReflectionPermissionFlag.ReflectionEmit permission for invoking certain operations on AssemblyBuilder objects and ModuleBuilder objects.

Establishing Permissions for Generated Code

  • A caller needs the right file permissions to persist a dynamic assembly.
  • Callers with the SecurityPermissionFlag.ControlEvidence permission will be able to supply evidence for the generated code. This evidence will get mapped through policy to determine the granted permissions.
  • Any caller may supply null evidence in which case the assembly will pick up the permission set of the caller. This ensures that the generated code does not have more permissions than its caller.
  • After a dynamic assembly has been persisted to disk, subsequent loads will get grants based on policies associated with the location where the assembly was persisted. That is, the assembly is treated like any other assembly that is loaded from disk.
  • Code generated by semi-trusted callers will always be verified. Specifically, the runtime always verifies code that does not get the SecurityPermissionFlag.SkipVerification permission. Fully trusted callers can opt to skip verification or require the generated code to be verified.


Reflection Emit APIs

This section provides a brief description of the Reflection Emit APIs. The section is organized by specific tasks. The following notes apply to the task descriptions.

Notes

  • The SpecialName attributes applies to types (specified using the TypeAttributes enum), methods (specified using the MethodAttributes enum), fields (specified using the FieldAttributes enum), properties (specified using the PropertyAttributes enum), and events (specified using the EventAttributes enum).
The SpecialName attribute is used to specify that the name is special to compilers, tools, etc. The runtime does not check this attribute.
The RTSpecialName attribute is used to specify that the name is special to the runtime Execution Engine. For example, the runtime sets this attribute on constructors. Applications can inspect the attribute but should not set it.


Defining a Dynamic Assembly

The Reflection Emit APIs provide many ways of creating dynamic assemblies. Dynamic assemblies are created using the various System.AppDomain.DefineDynamicAssembly methods. DefineDynamicAssembly returns an AssemblyBuilder object. DefineDynamicAssembly requires the caller to specify the AssemblyBuilderAccess enum. The enum defines whether the dynamic assembly will be run only, saved only, or run and/or saved. Some of the methods require the caller to supply evidence. evidence is the set of information that constitutes input to security policy decisions, such as what permissions can be granted to code. Other methods require the caller to supply permission requests. The permission requests come in three flavors: required, optional, and refused.

  • Define a named dynamic assembly.
  • Define a named dynamic assembly given the directory for saving the assembly.
  • Define a named dynamic assembly given the evidence.
  • Define a named dynamic assembly given the permission requests.
  • Define a named dynamic assembly given the evidence and the directory for saving the assembly.
  • Define a named dynamic assembly given the permission requests and the directory for saving the assembly.
  • Define a named dynamic assembly given the evidence and the permission requests.
  • Define a named dynamic assembly given the evidence, the permission requests, and the directory for saving the assembly.
  • Define a named dynamic assembly given the evidence, the permission requests, the directory for saving the assembly, and a Boolean parameter indicating if the creation of modules, types, and members in the dynamic assembly are to be synchronized.

A persistable dynamic assembly is saved using the method AssemblyBuilder.Save. Save specifies the name of the file to which the assembly should be written.

Strongly Named Assemblies

A dynamic assembly can be partially signed or fully signed.

For partial signing, the Originator must be specified in the AssemblyName argument passed to DefineDynamicAssembly. The runtime allocates the space within the PE for a strong name signature blob, but does not actually sign the assembly. The resulting assembly can be fully signed in a post-processing step using LM, ALINK, or SN.

For full signing, in addition to the Originator, a public/private key pair must be provided. These entities are usually stored in a file or disk or in a key container owned by a Crypto API CSP (Cryptographic Service Provider). Low security keys are often generated by software based CSP's and exported to a file so they can be checked into source code management systems during project development. High security keys are often generated by hardware that usually prevents export of the keys for security reasons. Such key pairs can only be accessed indirectly through a key container. The strong name key pair is specified using the System.Reflection.StrongNameKeyPair class.

For example,

FileStream fs = FileStream.Open(“SomeKeyPair.snk”);
StrongNameKeyPair kp = new StrongNameKeyPair(fs);
fs.Close();
AssemblyName an = new AssemblyName();
an.KeyPair = kp;
AppDomain appDomain = Thread.CurrentThread.GetDomain();
AssemblyBuilder ab = appDomain.DefineDynamicAssembly(an,…);


Defining a Dynamic Module

Defining a Dynamic Module in a Static Assembly

Dynamic modules in a static assembly are defined using the Assembly.DefineDynamicModule method. DefineDynamicModule returns a ModuleBuilder object. These modules are always transient. The following variations of Assembly.DefineDynamicModule are provided:

  • Define a dynamic module with the given name.
  • Define a dynamic module with the given name and conditionally emit symbolic information using the default implementation of the symbol writer interface.

Defining a Transient Dynamic Module in a Dynamic Assembly

Transient dynamic modules in a dynamic assembly are created using the AssemblyBuilder.DefineDynamicModule(String name) method. This method is inherited from System.Assembly.

Defining a Persistable Dynamic Module in a Dynamic Assembly

Persistable dynamic modules in a dynamic assembly are created using the AssemblyBuilder.DefineDynamicModule methods. The dynamic assembly must have been created with the AssemblyBuilderAccess argument specified as Save or RunAndSave. DefineDynamicModule returns an AssemblyBuilder object. The following variations of AssemblyBuilder.DefineDynamicModule are provided:

  • Define a persistable dynamic module with the given name and the given file name.
  • Define a persistable dynamic module with the given name and the given file name and conditionally emit symbolic information using the default implementation of the symbol writer interface.

The dynamic module is saved when the containing dynamic assembly is saved. To generate an executable, the ModuleBuilder.SetEntryPoint method must be called to identify the method that is the entry point to the module. If no entry point is specified, a DLL will be generated.


Defining a Type

Types are defined in the scope of a dynamic module using the ModuleBuilder.DefineType methods. DefineType returns a TypeBuilder object. In what follows, the type name is always a full path name. The Reflection Emit APIs provide the following options for defining types:

  • Define a class or interface with the given name.
  • Define a class or interface with the given name and attributes.
  • Define a class with the given name, attributes, and superclass.
  • Define a class with the given name, attributes, superclass, and the set of interfaces that the class implements.
  • Define a class with the given name, attributes, superclass, and packing size.
  • Define a class with the given name, attributes, superclass, and the class size as a whole.
  • Define a class with the given name, attributes, superclass, packing size, and the class size as a whole.

Before a type is used, the TypeBuilder.CreateType method must be called. CreateType completes the creation of the type. Following the call to CreateType, the caller can instantiate the type (using Type.CreateInstance) and invoke members of the type (using Type.InvokeMember). It is an error to invoke methods that change the implementation of a type after CreateType has been called. For example, the runtime will throw an exception if the caller tries to add new members to a type.

A class initializer is created using the method TypeBuilder.DefineTypeInitializer. DefineTypeInitializer returns a ConstructorBuilder object.

Nested types are defined using one of TypeBuilder.DefineNestedType methods.

The TypeBuilder.AddDeclarativeSecurity method adds declarative security to a type being built. AddDeclarativeSecurity may be called several times with each call specifying a security action (e.g., Demand, Assert, Deny) and a set of permissions that the action applies to.

Attributes

  • Interfaces are specified using the TypeAttributes.Interface and TypeAttributes.Abstract attributes.
  • Value types are specified using the TypeAttributes.ValueType attribute.
  • Concrete classes, i.e., classes that may not be extended are specified using the TypeAttributes.Sealed attribute.
  • There are several attributes that determine type visibility. See the description of the TypeAttributes enum.
  • If the TypeAttributes.LayoutSequential is specified, the class loader lays out fields in the order that it reads them from metadata. The class loader takes into account the packing size specified but ignores any specified field offsets. The metadata preserves the order in which the field definitions are emitted. Even across a merge, the metadata will not reorder the field definitions. The loader will honor the specified field offsets only if TypeAttributes.ExplicitLayout is specified.

Known Issues

  • Reflection.Emit does not verify if a non-abstract class that implements an interface has implemented all the methods declared in the interface. However, if the class does not implement all the methods declared in an interface, the execution engine will fail to load the class.
  • Although TypeBuilder is a subclass of Type, some of the abstract methods defined in the Type class are not fully implemented in TypeBuilder. These TypeBuilder methods throw the NotSupportedException. The desired functionality can be obtained by retrieving the created type using Type.GetType or Assembly.GetType and reflecting on the retrieved type. Two examples of methods that are not fully implemented are TypeBuilder.CreateInstance and TypeBuilder.InvokeMember.


Defining an Enum

Enums are defined in the scope of a dynamic module using the ModuleBuilder.DefineEnum method. DefineEnum returns an EnumBuilder object. Nested enums are defined using TypeBuilder.DefineNestedEnum.

An enum’s field is defined using the EnumBuilder.DefineLiteral method.

Before the enum is used, the EnumBuilder.CreateType method must be called. CreateType completes the creation of the enum.


Defining a Constructor

A constructor is defined using the TypeBuilder.DefineConstructor method. DefineConstructor returns a ConstructorBuilder object. DefineConstructor requires the caller to specify the constructor attributes using the enum MethodAttributes.

The default constructor for a class is defined using the TypeBuilder.DefineDefaultConstructor method. DefineDefaultConstructor returns a ConstructorBuilder object. The default constructor simply calls the constructor of the parent class. The runtime will automatically define a default constructor for a class if the caller does not define a constructor for the class.

Attributes

  • The runtime sets the attributes MethodAttributes.SpecialName and MethodAttributes.RTSpecialName for the constructor.
  • Private constructors are specified using the MethodAttributes.Private attribute. For other visibility attributes see the description of the MethodAttributes enum.

Known Issues

  • Although ConstructorBuilder is a subclass of ConstructorInfo, some of the abstract methods defined in the ConstructorInfo class are not fully implemented in ConstructorBuilder. These ConstructorBuilder methods throw the NotSupportedException. The desired functionality can be obtained by retrieving the type containing the constructor and reflecting on it. For example the ConstructorBuilder.Invoke method is not fully implemented.
  • Custom modifiers are not currently supported.


Defining a Method

This section describes how to define global methods and methods as type members using the Reflection Emit APIs. The APIs that define methods return MethodBuilder objects.

The MethodBuilder.AddDeclarativeSecurity method adds declarative security to a method being built. AddDeclarativeSecurity may be called several times with each call specifying a security action (e.g., Demand, Assert, Deny) and a set of permissions that the action applies to.

A global method is defined using the ModuleBuilder.DefineGlobalMethod method. DefineGlobalMethod returns a MethodBuilder object.

Global methods must be static. If a dynamic module contains global methods, the method ModuleBuilder.CreateGlobalFunctions must be called before persisting the dynamic module or the containing dynamic assembly. This is because the runtime postpones fixing up the dynamic module until all global functions have been defined.

A global native method is defined using the method ModuleBuilder.DefinePInvokeMethod. PInvoke methods must not be declared abstract or virtual. The runtime sets the MethodAttributes.PinvokeImpl attribute for a PInvoke method.

A method is defined as a type member using the TypeBuilder.DefineMethod method. DefineMethod returns a MethodBuilder object.

A PInvoke method is defined as a type member using the method TypeBuilder.DefinePInvokeMethod. DefinePInvokeMethod returns a MethodBuilder object. PInvoke methods must not be declared abstract or virtual. The runtime sets the MethodAttributes.PinvokeImpl attribute for a PInvoke method.

Attributes

  • Static methods are specified using the MethodAttributes.Static attribute.
  • Final methods, i.e, methods that cannot be overridden, are specified using the MethodAttributes.Final attribute.
  • Virtual methods are specified using the MethodAttributes.Virtual attribute.
  • Abstract methods are specified using the MethodAttributes.Abstract attribute.
  • There are several attributes that determine method visibility. See the description of the MethodAttributes enum.
  • Methods that implement overloaded operators must set the MethodAttributes.SpecialName attribute.
  • Finalizers must set the MethodAttributes.SpecialName attribute.

Known Issues

  • Although MethodBuilder is a subclass of MethodInfo, some of the abstract methods defined in the MethodInfo class are not fully implemented in MethodBuilder. These MethodBuilder methods throw the NotSupportedException. For example the MethodBuilder.Invoke method is not fully implemented. You can reflect on these methods by retrieving the enclosing type using Type.GetType or Assembly.GetType.
  • Custom modifiers are not currently supported.


Defining a Field

A field is defined as a type member using the TypeBuilder.DefineField method. The DefineField method requires the caller to specify the field name, the field type, and the attributes of the field (using the FieldAttributes enum). DefineField returns a FieldBuilder object.

FieldBuilder.SetOffset defines field layout. FieldBuilder.SetMarshal defines how a field is marshaled. FieldBuilder.SetConstant defines the default value of a field.

A field can also reference data in the .sdata section of the PE file. Fields representing initialized data are defined using FieldBuilder.DefineInitializedData. Fields representing uninitialized data are defined using FieldBuilder.DefineUninitializedData.

Attributes

  • Static fields are specified using the FieldAttributes.Static attribute.
  • Fields that can be initialized but not written to subsequently are specified using the FieldAttributes.InitOnly attribute.
  • Fields that are compile-time constants are specified using the FieldAttributes.Literal attribute.
  • There are several attributes that determine field visibility. See the description of the FieldAttributes enum.


Defining a Property

A property is defined using the TypeBuilder.DefineProperty method. DefineProperty returns a PropertyBuilder object. The DefineProperty method requires the caller to specify the property name, the signature of the property, the property attributes (using the enum PropertyAttributes), the backing field.

The PropertyBuilder class allows the caller to specify the Getter method, the Setter method and any other methods.

The default value of the property can be set using PropertyBuilder.SetConstant.

Known Issues

  • The PropertyBuilder class is not a subclass of PropertyInfo. This will be fixed in a future release.


Defining an Event

An event is defined using the TypeBuilder.DefineEvent method. DefineEvent returns an EventBuilder object.The DefineEvent method requires the caller to specify the event name, the event attributes (using the enum EventAttributes), and the event type.

The EventBuilder class allows the caller to specify the following:

  • The method for the AddOn event.
  • The method for the RemoveOn event.
  • The method for the Fire event.
  • Any other methods.


Defining a Parameter

Parameter names and parameter attributes are optional. Optional information for a method parameter, such as name and attributes (specified using the ParameterAttributes enum) can be defined using MethodBuilder.DefineParameter which returns a ParameterBuilder object. Optional information for a constructor parameter can be defined using ConstructorBuilder.DefineParameter which returns a ParameterBuilder object.

ParameterBuilder.SetMarshal is used to specify how the parameter is marshaled. ParameterBuilder.SetConstant is used to specify the default value for a parameter.


Defining a String Constant

A string constant is defined using the ModuleBuilder.GetStringConstant method. String constants are scoped by the module in which they are defined.


Emiting IL Instructions

The ILGenerator class provides methods that can be used to emit IL. ConstructorBuilder.GetILGenerator returns an ILGenerator object for a constructor. MethodBuilder.GetILGenerator returns an ILGenerator object for a method. The following services are provided by the ILGenerator class:

  • Emit various kinds of instructions using different forms of ILGenerator.Emit methods. These instructions take different kinds of operands depending on the type of instruction.
  • Declare a label. The position of the label in the instruction stream is specified using a different method.
  • Mark a position in the IL stream with a label.
  • Throw an exception.
  • Write a line to the console.
  • Define an exception block.
    • ILGenerator.BeginExceptionBlock begins an exception block.
    • ILGenerator.BeginExceptFilterBlock begins a filtered exception handler.
    • ILGenerator.BeginCatchBlock begins a typed exception handler.
    • ILGenerator.BeginFinallyBlock begins a finally handler.
    • ILGenerator.BeginFaultBlock begins a fault handler.
    • ILGenerator.EndExceptionBlock ends an exception block. Exception handling is described in Execution Engine Architecture.
For catch handlers, the sequence of calls should resemble the following template:
Emit some IL
BeginExceptionBlock
Emit the IL for the try block.
BeginCatchBlock
Emit the IL for the handler
BeginCatchBlock
Emit the IL for the handler
BeginFaultBlock
Emit the IL for the fault block
BeginFinallyBlock
Emit the IL for the finally block
EndExceptionBlock
For filtered handlers, the sequence of calls should resemble the following template:
Emit some IL
BeginExceptionBlock
Emit the IL for the try block.
BeginExceptFilterBlock
Emit the IL for the filtered exception handler
BeginCatchBlock
Emit the IL for the catch block. The catch handler should be supplied with a null type.
BeginFaultBlock
Emit the IL for the fault block
BeginFinallyBlock
Emit the IL for the finally block
EndExceptionBlock


Emitting Symbolic Information

The Reflection Emit APIs allow symbolic information to be defined for a dynamic module. Symbolic information can be defined using the default symbol writer interface or a caller specified symbol writer interface. The Reflection Emit APIs provide a convenient high-level abstraction of the low-level symbol writer interfaces in System.Diagnostics.SymbolStore.

The caller must express an intention that a dynamic module will contain symbolic information by calling the appropriate method to define the dynamic module. The various options for creating dynamic modules with symbolic information are described in the section Defining a Dynamic Module. It is an error to call a symbol writer method for a dynamic module that has not been created with the intention to contain symbolic information.

Reflection Emit provides the following methods for emitting symbolic information:

  • The symbol writer interface associated with a dynamic module can be obtained using ModuleBuilder.GetSymWriter.
  • The user entry point for a dynamic module can be defined using ModuleBuilder.SetUserEntryPoint. The user entry point is the method that the user has identified as the entry point rather than the caller generated stubs before the main method.
  • Document objects are defined using ModuleBuilder.DefineDocument.
  • Custom attributes stored in the symbol store are defined respectively for modules, methods, and constructors using ModuleBuilder.SetSymCustomAttribute, ConstructorBuilder.SetSymCustomAttribute, or ModuleBuilder.SetSymCustomAttribute. These custom attributes are different from the metadata custom attributes defined using the ICustomAttributeBuilder interface.
  • Sequence points are identified using ILGenerator.MarkSequencePoint. See the NGWS Runtime Debugging Services specification for a definition of sequence point.
  • Lexical scopes are defined using ILGenerator.BeginScope and ILGenerator.EndScope.
  • The name of a local variable in a method or a constructor can be set using LocalBuilder.SetLocalSymInfo. Optionally, the lexical scope of the local variable can be specified. A LocalBuilder object is obtained by calling ILGenerator.DeclareLocal.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

System.Resources Overview

© 2000 Microsoft Corporation. All rights reserved.


Overview of System.Resources Classes

Nearly every production quality application needs to make uses of resources. A resource is any non-executable data that is logically deployed with an application. This data can take any number of forms, for example: string literals, images and persisted objects. A large part of the job of making applications easily localizable involves making these resources changeable without changing the executable content. The runtime explicitly does not support localizing component metadata. Instead, resource data is persisted in separate resource files and/or in a designated section of a PE file.

The low-level resource model described in NGWS Runtime Resources Specification is most efficient at a coarse granularity. However, clients often want to retrieve resources at a fine granularity (a particular String for example). For this reason, the runtime provides a mechanism to encapsulate fine grain data such as Strings, persisted objects, etc. into larger chunks that can be more efficiently managed by the low-level resource infrastructure. The resources are serializable data consisting of name/value pairs, where the name is a string that identifies a resource and the value is an object such as a string literal. As an example of how localizable resource files are used, the base class library uses them to localize the message strings carried by exceptions, and they can be used to localize the text and images that appear in GUI objects. The Base Class Library resource classes use the low-level APIs to retrieve resources.

The primary goal of the Base Class Library classes is to provide easy access to culture correct data at runtime. The classes provides a means by which to create, manage, and read resource files that are persisted in the default resource file format. These managed classes impose some policy and consistency to how resources are stored and retrieved. The classes also provide extensibility points for advanced scenarios.

Resources should be ideally created for every culture, or at the minimum a meaningful subset of the cultures. The resource file names follow the naming convention basename.cultureName.resources. basename can be the name of the application, or depending on the granularity desired, the name of a class. cultureName is determined from the Name property of CultureInfo. A resource file for the language neutral culture (returned by CultureInfo.InvarientCulture) should be named basename.resources. If a subculture is specified, the naming convention for is basename.mainCultureName-subCultureName.resources.

The Base Class Library provides the following classes and interfaces in the System.Resources namespace:

ResourceManager. Encapsulates all the resources in a given application.

ResourceSet. A cache for the resources for a given culture. There is one ResourceSet per culture.

IResourceReader. An abstraction to read resources from a stream.

IResourceWriter. An abstraction to write resources to a stream.

ResourceReader. A default implementation of IResourceReader that reads resources in the runtime default binary resource file format.

ResourceWriter. A default implementation of IResourceWriter that writes resources in the runtime default binary resource file format.

How to Use the ResourceManager:

The ResourceManager class provides several constructors that are useful in different scenarios. This section describes how to use the ResourceManager class effectively.

To retrieve resources in a flat namespace using the assembly manifest, use:

 [C#] ResourceManager(String baseName, Assembly assembly, Type usingResourceSet, bool usingSatelliteAssemblies)

On large projects, this may lead to collisions in resource names. The individual resources can either all be contained in the specified main assembly, or they can be contained in satellite assemblies.

To retrieve resources in a type qualified namespace using the manifest of a specified type’s Assembly, use:

 [C#] ResourceManager(String baseName, Type locationInfo, Type usingResourceSet, bool usingSatelliteAssemblies)

On large projects, the use of type qualified namespace will reduce collisions in resource names. The individual resources can either all be contained in the specified main assembly, or they can be contained in satellite assemblies.

To retrieve resources in a type qualified namespace in the directory of the specified type’s module, use:

 [C#] ResourceManager(String baseName, Type locationInfo, Type usingResourceSet)

To retrieve resources in a specified directory, use:

 [C#] ResourceManager(String baseName, String resourceDir, Type usingResourceSet)

In each of the above constructors, the usingResourceSet parameter allows you to specify a implementation to be used. If you do not need a specific ResourceSet implementation but would like to use a custom resource file format, you should subclass ResourceSet, override ResourceSet.GetDefaultReader and ResourceSet.GetDefaultWriter, and pass that type to this constructor.

The last two constructors do not use the assembly cache. The resource files are assumed to reside in the specified directory and the constructors are used to access the resource files in the specified directory.

How to Create and use Resources:

To create a .resources file:

  1. Create a ResourceWriter with an appropriate file name (such as "myapp.resources")
  2. Call AddResource(String key, String value) N times on your ResourceWriter object.
  3. Call WriteFile.

To use a .resources file:

  1. Create a ResourceManager, passing in a base name (such as "myapp"), and the Assembly that contains the resources.
  2. Call GetString(String key), or GetString(String key, CultureInfo culture).

If you call GetString(String key), the resource that is returned is localized for the culture determined by the current regional settings of the current thread (this is accomplished via the Thread.CurrentUICulture property). If the resource has not been localized for that culture, the resource that is returned is localized for a "best match" (this is accomplished via the CultureInfo.Parent property). Otherwise, a null reference is returned.

If you call GetString(String key, CultureInfo culture), the resource that is returned is localized for the specified culture. If the resource has not been localized for that culture, the resource that is returned is localized for a "best match" (this is accomplished the CultureInfo.Parent property). Otherwise, a null reference is returned.

How to Localize Exception Strings:

  1. Define a static function that you can call when you want to format an exception string for a specific culture. For example, the runtime defines the following internal static function on the System.Environment class:
    internal static String GetResourceString(String key)
{
   if (_rm == null) {
      InitResourceManager();
   }
   String s = _rm.GetString(key);
   return s;
}
  2. To throw an exception, call GetResourceString to format the exception string for the culture of the current thread as in the following examples:
    throw new NotSupportedException(Environment.GetResourceString(“Unsupported”);
Where Unsupported is the name of the resource associated with the exception string.
throw new ArgumentException(String.Format(Environment. GetResourceString(“InvalidDirectory”), dir));
Where InvalidDirectory is the name of the resource associated with the exception string.

The Default Resource File Format:

There are three sections to the default file format. The first is the ResourceManager header, which consists of a magic number that identifies this as a Resource file, and a ResourceSet class name. The class name is written here to allow users to provide their own implementation of a ResourceSet (and a matching ResourceReader) to control policy. If objects greater than a certain size or matching a certain naming scheme shouldn't be stored in memory, users can tweak that with their own subclass of ResourceSet.

The second section in the system default file format is the RuntimeResourceSet specific header. This contains the version number of the resource file, the number of resources in the .resources file, the name and virtual offset of each resource, the number of different class types contained in the file, and a list of fully qualified class names. The class table allows the runtime to read multiple classes from the same file, including user defined types, in a more efficient way than would be possible by using Serialization provided the .resources file contains a reasonable proportion of base data types such as String and int. Serialization is used for all the non-intrinsic types.

The third section in the file is the data section, which consists of a type and a blob of bytes for each item in the file. The type is an integer index into the class table. The data is specific to that type, but may be a number written in binary format, a String, or a serialized Object.

The system default file format is as follows:

Item | Type of Data
ResourceManager header
Magic Number (0xBEEFCACE) | Int32
Class name of ResourceSet to parse this file | String
RuntimeResourceSet header
Version number | Int32
Number of resources in the file | Int32
Name & virtual offset of each resource | Set of (String, Int32) pairs
Number of classes in the class table | Int32
Name of each class | Set of Strings
ResourceReader Data Section
Type and Value of each resource | Set of (Int32, blob of bytes) pairs

The default implementation, when used with the default ResourceReader class, loads all the names from the resource file then loads the values on demand, storing them for later use. This implementation uses less memory than the ResourceSet class, assuming not all resources are loaded, but getting an individual resource will take longer. If a different IResourceReader is passed in, all the resource names and values are loaded on the first lookup.

In addition, the default implementation supports object serialization in a similar fashion. The runtime builds an array of class types contained in this file, and writes it to the ResourceReader header section of the file. Every resource will contain its type (as an index into the array of classes) with the data for that resource. The runtime will use the runtime's serialization support for this.

All strings in the file format are written with Stream.WriteUTF(), which writes out the length of the String as an Int32 then the contents as little endian Unicode chars.

The offsets of each resource string are relative to the beginning of the Data section of the file. This way, if a tool decided to add one resource to a file, it would only need to increment the number of resources, add it to the end of the name and offset list, possibly add it to the list of class types (and increase the number of items in the class table), and add the value at the end of the file. The other offsets would not need to be updated to reflect the longer header section.

Resource files are currently limited to 2 gigabytes due to these design parameters.

See Also

NGWS Runtime Resources Specification


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Threading Model

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Threading Model Concepts

© 2000 Microsoft Corporation. All rights reserved.


Summary of Concepts

The runtime provides a number of ways for users to synchronize access to instance and static methods and instance fields. This starts at the top of scale with the synchronized context property that has an easy to use construct; through synchronized code regions for more sophisticated usage; all the way down to primitives that allow users to roll their own synchronization mechanisms.

The runtime provides a thread model where classes are categorized into a number of categories that can be synchronized in a variety of different ways depending on the requirements.

The rows show the synchronization category. The columns show what synchronization support is provided for fields and methods.

Category | Global Fields | Static Fields | Static Methods | Instance Fields | Instance Methods | Specific Code blocks
Not Synchronized | No | No | No | No | No | No
Synchronized Context | No | No | No | Yes | Yes | No
Synchronized Context with Thread Affinity | No | No | No | Yes | Yes | No
Synchronized Code Region | No | No | Only Marked | No | Only Marked | On Marked
Synchronized Manually | Manual | Manual | Manual | Manual | Manual | Manual

No Synchronization

Default for objects. Instance and static methods and fields are not synchronized. Any thread can access any method or field at any time.

Key Points

  • Default for objects
  • No synchronization on methods or fields
  • Any thread can access any method or field at any time

Manual Synchronization

The synchronization primitives Interlocked, Monitor, ReaderWriterLock, Manual & Auto Reset Events can be used to manually take & release the lock for the particular primitive. They can be used to protect Global, Static and Instance fields. Likewise they can be used to protect global, static and instance methods.

Key Points

  • Atomic operations on a location
    • Automatic: certain reads and writes
    • Manual: interlocked
  • WaitHandle
    • WaitOne, WaitAny, WaitAll
    • Alertable Waits
  • ManualResetEvent
  • AutoResetEvent
  • Mutex
  • ReaderWriterLock (multiple readers, one writer)

Synchronized Code Regions

Code blocks, instance and static methods which are marked with a language keyword. Only a single thread at any one time is allowed to hold the sync block lock.

Key Points

  • Synchronization for:
    • Specified blocks of code
    • Marked Instance methods
    • Marked Static methods
  • No synchronization on fields
  • Code Regions
    • User specifies lock from an Object
    • Monitor: Enter, TryEnter, Exit, WaitOne, Pulse, PulseAll
  • Synchronized Methods
    • Instance methods use the lock from their instance (i.e this )
    • Static methods use the lock from the class
    • Access to certain regions of code can be synchronized to ensure that only one thread executes within the block at any given time. Some languages support a keyword or some kind of construct, which can be applied to a block of code, method or a class.

Synchronized code regions obtain the sync block lock when entering the region or waiting until the lock has been released and releasing the lock when exiting the synchronized code region.

Synchronized blocks of code

Some compilers support decorating blocks of code with a particular language keyword. The resulting generated code will try and take the lock when the code is executed. If the lock has already been taken then the executing code will wait until the lock becomes available. When the code exits the synchronized block of code the lock is released.

Synchronized Methods and MetaData

A runtime method can be marked with a metadata bit that will enforce synchronized access to the method. When the method is called the Runtime will synchronize on the SyncBlock.

Synchronized Instance Method

On instance methods, ‘this’ is used for synchronizing i.e. the SyncBlock for the object is used for synchronization.

Sychronized Static Methods

On static methods, the class object is used for synchronizing i.e. the SyncBlock for the class is used for synchronization.

Interrupt from a synchronized region

Thread.Interrupt can be used to break a thread out of blocking operations such as waiting for access to a synchronized region of code. Thread.Interrupt is also used to break threads out of operations like Object.Wait and Thread.Sleep.

Monitor

The Monitor class provides a mechanism for synchronizing on objects via the sync block. This could be used to provide synchronized access to state in an object. Multiple threads could synchronize on the Monitor Object.

C# and the lock keyword

The C# lock keyword uses Monitor.Enter and Monitor.Exit to lock the object.

class Foo
{
  void Method1()
  {
    lock(this)
    {
      //. . . Do some interesting work
    }
  }
}

If an exception is thrown in the “Do some interesting work” code block, the lock acquired by the lock(this) will be automatically released. The C# compiler emits a try / finally block with the Montior.Enter at the beginning of the try, and the Monitor.Exit in the finally block (therefore if an exception is thrown inside of the "lock" block, the finally handler will be run and therefore cleanup).

Synchronized Context

All instance methods and fields are synchronized. Multiple threads are allowed to access the methods and fields but only a single thread is allowed at any one time.

All objects in the same context / synchronization domain share the same lock. The type / class is attributed with the Synchronized Context Attribute Property.

Synchronized Domain with Thread Affinity

All instance methods and fields are synchronized. Only a single thread is ever used to access the methods and fields.

All objects in the same context / synchronization domain share the same thread. The type / class is attributed with the Thread Affinity Context Attribute Property


Thread Concepts

Threads are great and how they work

One or more threads run in an AppDomain. An AppDomain is a runtime representation of a logical process within a physical process. A thread is the basic unit to which the operating system allocates processor time. Each AppDomain is started with a single thread, but can create additional threads from any of its threads.

Each thread maintains exception handlers, a scheduling priority, and a set of structures the system will use to save the thread context until it is scheduled. The thread context includes the thread's set of machine registers and stack, in the address space of the thread's process.

Operating systems that support preemptive multitasking, create the effect of simultaneous execution of multiple threads from multiple processes. On a multiprocessor computer, some operating systems that are multi-processor aware can simultaneously execute as many threads as there are processors on the computer.

A multitasking operating system divides the available processor time among the processes or threads that need it. The system is designed for preemptive multitasking; it allocates a processor time slice to each thread it executes. The currently executing thread is suspended when its time slice elapses, allowing another thread to run. When the system switches from one thread to another, it saves the thread context of the preempted thread and restores the saved thread context of the next thread in the queue.

The length of the time slice depends on the operating system and the processor. Because each time slice is small, multiple threads appear to be executing at the same time. This is actually the case on multiprocessor systems, where the executable threads are distributed among the available processors. However, caution must be used when using multiple threads in an application, because system performance can decrease if there are too many threads.

Advantages of threads

To the user, the advantage of threading is the ability to have multiple activities occurring simultaneously i.e. multiple units of execution. For example, a user can edit a spreadsheet while another thread is recalculating other parts of the spreadsheet within the same application.

To the application developer, the advantage of threading is the ability to create applications that use more than one thread of execution. For example, a process can have a user interface thread that manages interactions with the user (keyboard and mouse input), and worker threads that perform other tasks while the user interface thread waits for user input. If you give the user interface thread a higher priority, the application will be more responsive to the user, while the worker threads use the processor efficiently during the times when there is no user input.

When to use threads

A multithreaded process can manage mutually exclusive tasks with threads, such as providing a user interface and performing background calculations. Creating a multithreaded process can also be a convenient way to structure a program that performs several similar or identical tasks concurrently. Your AppDomain could use multiple threads to accomplish the following tasks:

  • Communicating over a network, web server and to a database.
  • Performing operations that take a large amount of time.
  • Distinguish tasks of varying priority. For example, a high-priority thread manages time-critical tasks, and a low-priority thread performs other tasks.
  • Allow the user interface to remain responsive, while allocating time to background tasks.

Threads can be expensive and the consumption of OS resources

It is recommended is to use as few threads as possible, thereby minimizing the use of OS resources. This improves performance. Threading has resource requirements and potential conflicts to be considered when designing your application. The resource requirements are as follows:

  • The system consumes memory for the context information required by processes, AppDomain and threads. Therefore, the number of processes, AppDomains and threads that can be created is limited by available memory.
  • Keeping track of a large number of threads consumes significant processor time. If there are too many threads, most of them will not be able to make significant progress. If most of the current threads are in one process, threads in other processes are scheduled less frequently.

Threads can be difficult to get right – deadlocks and race conditions

Providing shared access to resources can create conflicts. To avoid them, you must synchronize access to shared resources. This is true for system resources (such as communications ports), resources shared by multiple processes (such as file handles), or the resources of a single AppDomain (such as global, static and instance fields) accessed by multiple threads. Failure to synchronize access properly (in the same or in different AppDomains) can lead to problems such as deadlock and race conditions. The runtime provides synchronization objects that can be used to coordinate resource sharing among multiple threads. Reducing the number of threads makes it easier and more effective to synchronize resources.

Thread the Unit of execution

The System.Threading.Thread class is an abstraction of the threads that execute within the Runtime. All management of threads is done through the System.Threading.Thread class, which represents a managed thread. This includes threads created by the Runtime and those created outside the Runtime that happen to be "wandering" into the Runtime environment to execute some managed code.

For reference details see: Thread.

Win32 | Runtime
CreateThread | Combination of Thread and ThreadStart. For example if the programmer was using CreateThread(…,&Method,…) they would now use new Thread(new ThreadStart(&oFoo::Method)).
TerminateThread | Thread.Stop
 | Thread.Abort
SuspendThread | Thread.Suspend
ResumeThread | Thread.Resume
Sleep | Thread.Sleep
WaitForSingleObject on the thread handle | Thread.Join
ExitThread |
GetCurrentThread | Thread.CurrentThread
SetThreadPriority | Thread.Priority
No equivalent | Thread.Name
No equivalent | Thread.IsBackground
No equivalent | Thread.ApartmentState

Creating Threads

When an AppDomain is created, the Runtime creates a thread to execute the code within that AppDomain. If the code being executed is managed code, then a System.Threading.Thread object for that thread can be obtained by retrieving the static property on the thread class Thread.CurrentThread.

Creating a new instance of a System.Threading.Thread object can create new managed threads. The constructor for System.Threading.Thread takes, as its only parameter, a Thread Delegate.

A delegate is an object that acts as reference to a method on another class. A thread delegate is a special type of delegate that references a method that will be executed by a new thread.

ThreadStart

Once you have an instance of a delegate, you can pass the delegate to the constructor of a ThreadStart object. The ThreadStart object is passed to the Thread constructor. The Thread function does not begin executing until Thread.Start is called. Once Thread.Start is called, execution begins at the first line of the method referred to by the thread delegate. Calling Thread.Start more than once will cause a ThreadStateException to be thrown.

Thread.Start – Submits a start request, this is an asynchronous / request call. The method may return even while the thread has not actually started. Use Thread.ThreadState and Thread.IsAlive to determine the state of the thread.

Thread.Stop – submit a stop request, this is an asynchronous / request call. The method may return even while the thread has not actually stopped. Use Thread.ThreadState and Thread.IsAlive to determine the state of the thread.

Working With Threads

Once a thread has been started, it’s often useful for that thread to pause for a fixed period of time. Calling Thread.Sleep causes the thread to immediately block for a fixed number of milliseconds. The Sleep method takes as a parameter a timeout, which is the number of milliseconds that the thread should remain blocked. The Sleep method is called when a thread wants to put itself to sleep. One thread cannot call Sleep on another thread. Calling Thread.Sleep(0) causes a thread to yield the remainder of its timeslice to another thread. Calling Thread.Sleep(Timeout.Infinite) causes a thread to sleep until it is interrupted by another thread that calls Thread.Interrupt.

A thread can also be paused by calling Thread.Suspend. When a thread calls Thread.Suspend on itself, the call blocks until the thread is resumed by another thread. When one thread calls Thread.Suspend on another thread, the call is a non-blocking call that causes the other thread to pause. Calling Thread.Resume breaks another thread out of the suspend state and causes the thread to resume execution. Calling Thread.Resume causes the thread to resume execution regardless of how many times Thread.Suspend was called. For example, if Thread.Suspend is called 5 consecutive times, then Thread.Resume is called, the thread will resume execution immediate following the call to Resume (it does not take 5 calls to Thread.Resume).

Unlike Thread.Sleep, Thread.Suspend does not cause a thread to immediately stop execution. The Runtime must wait until the thread has reached a safe point (defined below) before it can suspend the thread. A thread cannot be suspended if it has not been started or if it has already been stopped.

Threads can be blocked for any of several reasons. For example, a thread may be waiting for another thread to die i.e. Thread.Join, it may be waiting for access to a synchronized object i.e. Monitor.Wait, or it may just be sleeping i.e. Thread.Sleep. A thread that is waiting can be interrupted by another thread by calling Thread.Interrupt on the blocked thread. When a thread is interrupted, a ThreadInterruptedException is thrown which causes the thread to break out of the blocking call. The thread should catch the ThreadInterruptedException and do whatever is appropriate to continue working. If the thread ignores the exception, the Runtime will catch the exception and kill the thread but not before displaying a message informing the user that the thread was killed prematurely. Thread.Interrupt will also interrupt a thread that is waiting for access to a synchronized region of code.

  • Thread.Interrupt wakes the destination thread out of any wait it may be in and causes a particular exception to be thrown in the destination thread.
  • Thread.Stop is similar to Thread.Interrupt, but it does not wake the destination thread out of a wait. Instead, when the thread returns normally from the wait it will throw the exception.
  • Thread.Abort is identical to Thread.Stop, except for the exception that is thrown. In the case of Thread.Abort the ThreadAbortException is a special one that cannot be caught from managed code (although finally blocks are executed).

Safe point

When a Thread.Stop or Thread.Suspend is performed on a thread. The EE does not perform the action immediately. The Runtime records that a thread stop or suspend has been requested and waits until the thread has reached a safepoint before stopping or suspending the thread. A safepoint for a thread is a safe point for GC. If someone provokes a GC and the Runtime can successfully hijack a thread for the GC, the Runtime will Stop or Suspend it at that time. Hijacking is the act of bashing a return address on the stack so it returns to Runtime, rather than to its caller. This allows the Runtime to take control of threads at return points.

If a loop contains a control path with no calls somehow yield for a GC (if it had calls, hijacking would snag the thread for GC), the regular JIT detects this situation and makes the entire method fully interruptible. All instructions in the method are GC safe, because it creates GC tables for every instruction. The FJIT handles this situation by not even bothering to detect it. Instead, it makes all backward (loop-closing) branches check the EE to see if a GC is starting up.

Destroying Threads

The Thread.Stop method is used to stop a thread permanently. When stop is called, the Runtime waits for the thread to reach a safe point and then throws ThreadStoppedException. If the thread does not catch the exception, the Runtime quietly kills the thread without informing the user. The thread may choose to catch the exception and do some clean up, in which case the thread must re-throw the exception after the clean up is complete.

A ThreadStoppedException is the only exception that the Runtime will catch quietly. Any other exception caught by the Runtime will cause an Uncaught Exception message to display.

As Thread.Stop does not cause the thread to stop immediately (it waits for the thread to reach a safe point), the caller must wait on the thread if it needs to be sure the thread is indeed stopped. The caller can do this by calling Thread.Join. Join is a blocking call that does not return until the thread has actually stopped executing. Once a thread is stopped, it cannot be restarted. Calls to Thread.Suspend or Thread.Resume on a stopped thread are ignored.

The following example creates a thread, destroys it, and then calls Thread.Join to wait for it to stop executing. It tries to restart the stopped thread, however this expectantly throws the ThreadStateException

You can also call Thread.Join(int timeout) and specify a timeout period. If the thread dies before the timeout has elapsed, the call returns TRUE. Otherwise, if the time expires before the thread dies, the call returns FALSE. Threads that are waiting on a call to Thread.Join can be interrupted by other threads that call Thread.Interrupt.

Scheduling Threads

Every thread has a thread priority assigned to it. Threads created within the Runtime are initially assigned the priority of Normal. Threads created outside the Runtime retain the priority they had before they entered the Runtime. You can get the priority of any thread with the Thread.Priority property and you can change the priority of any thread with Thread.Priority property.

Threads are scheduled for execution based on their priority. Even though the threads are executing within the Runtime, the threads are assigned CPU time by the OS. The details of the scheduling algorithm used to determine the order in which threads are executed varies with each OS. Under some operating systems, the thread with the highest priority (of those threads that are runable can be executed) is always scheduled to run first. If multiple threads with the same priority are all available, the scheduler does a round robin among the threads at that priority giving each thread a fixed timeslice in which to execute. As long as a thread with a higher priority is available to run, lower priority threads do not get to execute. Once there are no more runable threads at a given priority that can be executed, the scheduler moves to the next lower priority and schedules the threads at that priority for execution. Once a higher priority thread becomes runable to be executed, the lower priority thread is preempted and the higher priority thread is allowed to execute once again. On top of all that, the operating system can also adjust thread priorities dynamically as an application’s UI is moved between foreground and background. Other operating systems may choose to use a different scheduling algorithm.

Thread States

A thread can be in multiple states at the same time. The property Thread.ThreadState provides a bit mask indicating the thread's current state. A thread is always in at least one of these possible states in the ThreadState enum.

Threads created within the Runtime are initially in the Unstarted state. The thread remains in the Unstarted state until it is transitioned into the started state by calling Thread.Start. External threads that wander in to the Runtime are already in the started state. Once in the started state, there are a number of actions that can cause the thread to change states. The following table lists the actions that cause a change of state along with the corresponding new state.

Action | State Transition
Another thread calls Thread.Start | Unchanged
The thread starts running | Running
The thread calls Thread.Sleep | WaitSleepJoin
The thread calls Monitor.Wait on another object | WaitSleepJoin
The thread calls Thread.Join on another thread | WaitSleepJoin
Another thread calls Thread.Suspend | SuspendRequested
The thread responds to a Thread.Suspend request | Suspended
Another thread calls Thread.Resume | Running
Another thread calls Thread.Stop | StopRequested
Another thread calls Thread.Interrupt | Running
The thread responds to a Thread.Stop request | Stopped
Another thread calls Thread.Abort | AbortRequested
The thread response to a Thread.Abort | Aborted

It is often the case that a thread is in more than one state at any given time. For example, if a thread is blocked on a call to Wait and another thread calls Stop on that same thread, the thread will be in both the Waiting and the Stopping state at the same time. In that case, as soon as the thread returns from the call to Wait or is interrupted, it will receive the ThreadStopException.

Once a thread leaves the Unstarted state as the result of a call to Thread.Start, it can never return to the Unstarted state. A thread can never leave the Stopped state, either.

The following diagram illustrates the thread states and the method calls that would cause the thread to leave each state.

Figure 1: Thread State Diagram

Foreground and Background Threads

In the Runtime, a thread is either a background thread or a foreground thread. Background threads are identical to foreground threads with one exception. Unlike a foreground thread, a background thread will not keep the Runtime alive. Once all foreground threads have been stopped, the Runtime kills all background threads and shuts down. A thread can be designated as a background or a foreground thread by setting the Thread.IsBackground property. For example a foreground thread can be designated a background thread by calling Thread.IsBackground = True. A background thread can be designated a foreground thread by calling Thread.IsBackground = False.

The Runtime kills the background threads by throwing a ThreadStoppedException. Background threads are given the opportunity to perform any cleanup by catching the exception and re-throwing it. In this situation, background threads need to handle the ThreadStoppedException in a timely fashion. If a background thread catches the exception and does not re-throw it within a reasonable period of time, the thread will be quietly killed as the Runtime is shut down.

The runtime kills the background threads by throwing a ThreadStoppedException. Background threads are given the opportunity to perform any clean up by catching the exception and re-throwing it. In this situation, background threads needs to handle the ThreadStoppedException in a timely fashion. If a background thread catches the exception and does not re-throw it within a reasonable period of time, the thread will be quietly killed as the Runtime is shut down.

Thread Local Store

Key Points

  • Data slots per thread.

The Thread Local Store (TLS) provides data slots for the thread. The data slots are unique per thread i.e. the state is not shared across threads. Slot types:

  • Allocated Data Store Slot. A slot is allocated and a slot object is returned. The slot object is used to access the data slot. The slot is freed automatically.
  • Named Slots. A name is provide and a slot is allocated and a slot object is returned. The slot object is used to access the data slot. The slot can be freed manually.

Apartments

A thread can be marked to indicate that it will host a single-threaded or multi-threaded apartment. The Thread.ApartmentState property returns and assigns the apartment state of a thread. If the property has not been set then the property returns Unknown. The property can be set when the thread is in the Unstarted or Running state, however it can be only be set once for a thread. The two valid property states are STA or MTA.

Threads and Garbage Collection

In order to perform a GC, all the threads must be suspended - except for the thread performing the GC, of course. Each thread must be brought to a “safe place” before it can be suspended. It has already been noted that threads might have to be brought to a safe place for other reasons like stopping a thread or the programmatic Thread.Suspend service. The Runtime supports bringing threads to a safe spot for the purposes of GC.

Thread Store

The Runtime tracks all the interesting threads in its process. Threads are interesting if they have ever executed code within the Runtime. In other words, the Runtime only tracks the threads it knows about.

Threads can enter the Runtime through COM Interop, because we expose the runtime objects out as COM objects. Other COM (not runtime) threads can then make calls on those exposed objects. Other ways a thread can enter the Runtime include DllGetClassObject() and PInvoke.

When one of our gateways, like a COM-Callable-Wrapper, is invoked on a thread, the gateway checks the TLS of that thread. It’s looking for one of our internal Thread objects. If one is found, we already know about this thread. Otherwise, it’s a thread we’ve never seen before. So we construct a new Thread object and install it in the TLS of that thread.

In addition to placing all our state about the thread in the Thread object in its TLS, we also keep a list of all the threads we’ve seen. This is the ThreadStore. It is used during thread suspension for GC and during product shutdown.


Thread Relative Statics Concepts

Thread Relative Statics provide for type member fields that are relative for the thread. Whenever the static field is accessed on a particular thread it will be unique or relative for that thread only. The static is not shared between threads. If the static field is access on another thread the field will contain a different value, since the field is unique per thread. Null initialization of thread local statics. The member field is marked with a custom attribute ThreadStaticAttribute.


ThreadPool Concepts

Queuing work items to the Thread Pool

Key Points

  • Queue work items for execution on the first available thread from the Thread Pool
  • One OS Thread Pool per process
  • Efficient use of thread resources
  • Switch to correct appdomain when executing threadpool callbacks
  • Supported on Windows2000, Windows NT4, Windows Millennium, Windows 98, Windows 95

Used to post workitems to the threadpool. There is only one managed thread pool per process.

There are many applications that create threads that spend a great deal of time in the sleeping state waiting for an event to occur. Other threads may enter a sleeping state only to be awakened periodically to poll for a change or update status information. Thread pooling enables you to use threads more efficiently by providing your application with a pool of worker threads that are managed by the system. One thread monitors the status of all wait operations queued to the thread pool. When a wait operation has completed, a worker thread from the thread pool executes the corresponding callback function.

You can also queue work items that are not related to a wait operation to the thread pool. To request that a work item be handled by a thread in the thread pool, call the System.Threading.ThreadPool.QueueUserWorkItem from managed code or CorQueueUserWorkItem from unmanaged code. This function takes a parameter to the delegate or function respectively, that will be called by the thread selected from the thread pool. There is no way to cancel a work item after it has been queued.

Timer-queue timers and registered wait operations also use the thread pool. Their callback functions are queued to the thread pool.

The thread pool is created the first time you call ThreadPool.QueueUserWorkItem, or when a timer or registered wait operation queues a callback function. The number of threads that can be created in the thread pool is limited only by available memory. Each thread uses the default stack size and runs at the default priority. Each thread can handle up to 63 wait operations.

Also see the article “New Windows 2000 Pooling Functions Greatly Simplify Thread Management” by Jeffrey Richter in the April 1999 issue of Microsoft Systems Journal.

For reference details see: ThreadPool System.Threading.WaitCallback

IO operations that complete asynchronously

This section describes NGWS runtime IO Completions and how shows how classlibrary writers could implement async IO operations that complete. The example of how the class libraries implemented async IO on file is only an illustration. Class library writers are free to implement async IO quite differently. The samples are in pseudo code.

IO operations on OS handles can complete asynchronously. This involves associating an OS IO handle with the thread pool and specifying a callback. When IO operations complete asynchronously, they are surfaced via a thread from the thread pool and the associated callback is called.

In order for this to occur correctly, the Win32 OVERLAPPED structure needs to be prepared so that the COR infrastructure is able to route the IO operation from the thread pool to the correct callback.

The APIs have been designed so that two statements are true:

  • The runtime can intercept every callback from the OS to the class libraries if it wants to.
  • The runtime can wire up the OS directly to the class libraries' callbacks, if it chooses not to intercept.

The rationale is that COR needs to be in the callback chain, because the thread pool thread needs to switch to co-operative GC mode when surfacing from the thread pool.

Additionally we may find a need to do bookkeeping in the callback. Or we may have to switch the callback to a non-IO thread. But we don't want to incur the cost of pumping all callbacks through the runtime if it turns out we have no work to do there.

In order to perform interceptions, each overlapped struct submitted to the OS in an async operation must be "packed" by the runtime. If the runtime chooses to intercept the callback, this involves recording the class libraries' callback in the overlapped struct so the runtime can finish dispatching the callback.

Another motivation for the design, which is that the runtime assumes responsibility for pinning and unpinning the overlapped structure. This would otherwise be a constant source of difficult-to-diagnose catastrophic bugs, because the lifetime of pinning doesn't match the lifetime of the PInvoke call that initiates the IO.

By calling the static method ThreadPool.BindHandle an OS handle can be associated with the ThreadPool.


Timer Concepts

Key Points

  • Call a specific delegate at a specified time
  • Timers serviced by Thread Pool

Timers are lightweight objects that enable you to specify a delegate to be called at a specified time. The wait operation is performed by a thread in the thread pool.

To add a timer, call the Timer.Timer constructor. You can specify a delegate to be executed by a worker thread from the thread pool when the timer expires. To update a timer, call the Timer.Change method.

A timer is set to the signaled state when its specified due time arrives. Any thread with a handle to the timer can use one of the wait functions to wait for the timer state to be set to signaled.

To cancel a pending timer, call the Timer.Dispose function.

For reference details see: Timers.


Monitor Concepts

Key Points

  • Monitor associated with an object on demand.
  • Monitors are unbound i.e. can be called directly from any Context.
  • Monitors cannot be instanciated, they are associated with an object.

Monitors expose the ability to take and release the sync block lock on an object on demand via Enter, TryEnter and Exit.

The Monitor Wait, Pulse and PulseAll methods are related to SyncBlocks. It is necessary to be in a synchronized region on an object before calling Wait or Pulse on that object. Wait releases the lock if it is held and waits to be notified. When Wait is notified, it returns and has obtained the lock again. Notify signals for next thread in wait queue to proceed.

Each object has an associated SyncBlock index. If an object has not been used in a synchronization operation then the associated SyncBlock index is empty. When the object is used in a synchronization operation then a SyncBlock is retrieved from the SyncBlock cache. The SyncBlock is associated with the object by updating the object’s SyncBlock index.

A SyncBlock is a sparsely allocated data structure. It currently serves several purposes:

  • It contains the lock used to implement synchronized methods and synchronized blocks of code.
  • It contains the list of waiting threads used to implement Monitor.Wait, Monitor.Pulse and Monitor.PulseAll.

Generally most objects will not have SyncBlocks, unless a synchronization operation has performed against the object. Furthermore, the SyncBlock is accessed using a SyncBlockIndex which is allocated at a negative offset on the managed object. All objects have a SyncBlockIndex, but most of these indices have a value of 0 - meaning they share a single dummy SyncBlock. Logically, such objects don’t have a SyncBlock.

However, there’s a set of objects that have non-zero SyncBlockIndexes, but they don’t actually have SyncBlocks. These are objects that don’t otherwise need a SyncBlock, but which have had their Object.GetHashCode method called. If a class uses the default (runtime provided) implementation of GetHashCode, this method will allocate a SyncBlockIndex but will not allocate the SyncBlock (to save memory). The result is a relatively unique and stable hash code.

It is only relatively unique because SyncBlockIndexes will be recycled when objects are collected by GC, and because other classes which override GetHashCode can certainly return the same numbers. Also, it’s relatively stable because it does not persist with the object. It is only valid so long as the object is activated within a particular EE instance.

For reference details see: Object.


WaitHandle Concepts

The WaitHandle class is used to represent all synchronization objects in the runtime that allow multiple wait. Encapsulates Win32 synchronization handles.

Examples of subclasses of WaitHandle are:

  • Thread handle
  • Mutex handle
  • Event handle
  • Process handle
  • IO handle


ManualResetEvent Concepts

The state of a manual-reset event object remains signaled until it is set explicitly to the nonsignaled state by the Reset method. Any number of waiting threads, or threads that subsequently begin wait operations for the specified event object by calling one of the wait functions, can be released while the object's state is signaled. See WaitHandle for WaitOne, WaitAny and WaitAll.

For reference details see: ManualResetEvent


AutoResetEvent Concepts

An AutoResetEvent is automatically reset to nonsignaled by the system after a single waiting thread has been released. The state of an auto-reset event object remains signaled until a single waiting thread is released, at which time the system automatically sets the state to nonsignaled. If no threads are waiting, the event object's state remains signaled. See WaitHandle for WaitOne, WaitAny and WaitAll.

For reference details see: AutoResetEvent.


Mutex Concepts

Synchronization primitive that can also be used for interprocess synchronization.

WaitOne, WaitAll, and WaitAny can be used to request ownership of the mutex. The state of the mutex is signaled if no thread owns it.

The thread that owns a mutex can specify the same mutex in repeated wait function calls without blocking its execution. It must release the mutex as many times to release ownership.

If a thread terminates normally while owning a mutex, the state of the mutex is set to signaled and the next waiting thread gets ownership.


Interlocked Concepts

Interlocked – synchronized access to a variable

The System.Threading.Interlocked methods CompareExchange, Decrement, Exchange, and Increment provide a simple mechanism for synchronizing access to a variable that is shared by multiple threads. The threads of different processes can use this mechanism if the variable is in shared memory.

The Increment and Decrement functions combine the operations of incrementing or decrementing the variable and checking the resulting value. This atomic operation is useful in a multitasking operating system, in which the system can interrupt one thread's execution to grant a slice of processor time to another thread. Without such synchronization, one thread could increment a variable but be interrupted by the system before it can check the resulting value of the variable. A second thread could then increment the same variable. When the first thread receives its next timeslice, it will check the value of the variable, which has now been incremented not once but twice. The interlocked variable access functions protect against this kind of error.

The Exchange function atomically exchanges the values of the specified variables. The CompareExchange function combines two operations: comparing two values and storing a third value in one of the variables, based on the outcome of the comparison.

These services are implemented differently on UP vs. MP machines, to avoid the lock prefix.

For reference details see: Interlocked.


ReaderWriterLock Concepts

Defines the lock that implements single-writer / multiple-reader semantics that supports the following features:

  • Cheap enough to be used in large numbers such as per object synchronization.
  • Supports timeout. This is a valuable feature to detect deadlocks.
  • Supports caching of events. This allows the events to be moved from least contentious regions to the most contentious regions. In other words, the number of events needed by Reader-Writer locks are bounded by the number of threads in the process.
  • Supports nested locks by readers and writers.
  • Supports spin counts for avoiding context switches on multi processor machines.
  • Supports functionality for upgrading to a writer lock with a return argument that indicates intermediate writes. Downgrading from a writer lock restores the state of the lock.
  • Supports functionality to Release Lock for calling app code. RestoreLock restores the lock state and indicates intermediate writes.
  • Recovers from most common failures such as creation of events. In other words, the lock mainitains consistent internal state and remains usable.

For reference details see: ReaderWriterLock.


AppDomain Unhandle Exception Concepts

Key Points

  • Events are delivered in the same order as registration (not reverse order).
  • Can register the same delegate more than once -- and it will be called more than once.
  • If you remove a delegate that is in the list multiple times, the last instance is removed.


Unmanaged Thread API Concepts

The unmanaged Thread APIs provide access to the Runtime thread pool from unmanaged code.


Synchronized Concepts

Key Points

  • Mark type with Synchronization Context Attribute
  • Synchronized Domain – many participating Contexts share the same Synchronization lock
  • Great for naïve Users
  • Automatic Synchronization
    • Instance methods
    • Instance fields
  • No Synchronization
    • Static fields and methods
    • Specific code blocks
  • Synchronous calls wait until domain is free
  • Asynchronous calls are queued via the Thread Pool and the calling thread returns immediately.

For a user who wants all access, be it instance methods and instance fields access to be synchronized, this is the easiest approach. A type (class) is marked as context-bound and is attributed with the Synchronization custom attribute. The runtime with then allow only a single thread at any one time access to instance methods and fields.

[Synchronization()]
public class Foo : ContextBoundObject
{
. . . // implementation for Foo
}

// Make an instance of Foo
void SomeOtherMethod(void)
{
. . . Foo foo = new Foo();
}

When the object’s is called from outside the context synchronization occurs. If the call is synchronous then the call proceeds on the same thread. If the call is asynchronous then the message is queued to the thread pool and will be picked up by thread in the thread pool.

The Synchronization context property allows only a single thread to be active within the context at any one time. Multiple threads are able to enter the context but only one at a time. While a thread is active in the context, messages posted to the objects within the context will be added to the queue. This may be employed by one or more contexts to enforce a synchronization domain in which only one thread can execute at a time.

Synchronous calls are dispatched on the calling thread, which wait if the domain is occupied. This is done by contributing sinks that intercept and serialize in-coming calls for the respective contexts.

Asynchronous calls are dispatched using threads from the thread pool. The calling thread is returned immediately. Async methods use the Asynchronous Programming model mechanism that is described in Asynchronous Programming model specification.

By adding an instance of Synchronization to one or more contexts and obtaining proxies to objects living in these contexts, several threads can make Synchronous and Asynchronous calls on these objects. All the calls on these objects will be serialized due to the Synchronized context property.

If the synchronization property is marked for reentrancy, then call-outs are intercepted too. The call-out interception allows other waiting threads to enter the synchronization domain for maximal throughput.

[If reentrancy is false then no reentrancy is allowed even on the same physical thread

If reentrancy is true then all reentrancy is allowed and the context will service other calls during a call out.]


Thread Affinity Concepts

Key Points

  • Refinement of Synchronized Context
  • Mark type with Thread Affinity Context Attribute
  • Only a single thread allowed to execute within the Synchronization Domain, hence only that thread can access instance fields and methods
  • Thread initialized as STA and pumps Win32 messages on Wait*
  • Useful
    • User Interface objects
    • Objects that hold onto OS resource that should only be accessed on the thread that created them

The Thread Affinity Context Property provides a similar capability as the Synchronization Context property with the further constraint that only a single thread is allow to execute within the context or synchronization domain at any time. This is useful for objects that could hold onto OS resources that should only be accessed from a particular thread and only that thread. Thread Affinity is useful when users want to always access the object on the same thread. Certain UI resources should only be accessed on the same thread, on which they were created.

An instance of this property in a context enforces a thread affinity domain for the context (and all contexts that share the same instance). This means that for all contexts sharing an instance of this property, all methods on objects resident in those context will execute on the same thread for the lifetime of the objects.

[ThreadAffinity()]
public class Foo : ContextBoundObject
{
. . . // implementation for Foo
}

// Make an instance of Foo
void SomeOtherMethod(void)
{
. . . Foo foo = new Foo();
}

This is done by contributing message sinks that intercept and serialize in-coming calls.

If the property is marked for reentrancy, then call-outs are intercepted too. The call-out interception allows other waiting calls to enter the thread affinity domain.

The finalizer for thread affinity marked components is run on the thread for the context i.e. the same one the object was created on when the constructor was called.

For reference details see: ThreadAffinity.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Threading Model How To

© 2000 Microsoft Corporation. All rights reserved.


Thread HowTo

Creating a Thread

A thread delegate is declared in the following way:

using System;
using System.Threading;

public class Foo
{
//
  // The method that will be called when the thread is started
  //
  public void Baz()
  {
    Console.WriteLine("Foo Baz is running on another thread");
  }
};

public class Simple
{
  public static int Main(String[] args)
  {
    Console.WriteLine("Thread Simple Sample");

    Foo oFoo = new Foo();

    //
    // Create the thread object, passing in the Foo.Baz method
    // via a ThreadStart delegate
    //
    Thread oThread = new Thread(new ThreadStart(oFoo.Baz));

    //
    // Start the thread
    //
    oThread.Start();

    return 0;
  }
}


Thread Relative Statics HowTo

Marking fields as Thread Relative

class Foo
{
   int a;
   static int b;
[ThreadStaticAttribute()]
   static int c;

   int d=5;
   static int e=6;
    [ThreadStatic()]
   static int f=7;

   Foo()
   {
      a = 1;
      b = 2;
      c = 3;
   }

   void Method1()
   {
      Console.WriteLine(“{0}”, a);
      Console.WriteLine(“{0}”, b);
      Console.WriteLine(“{0}”, c);
      Console.WriteLine(“{0}”, d);
      Console.WriteLine(“{0}”, e );
      Console.WriteLine(“{0}”, f );
   }
}

Instance member field

a and d are instance members.

a. Instance member field. Uninitialized.

d. Instance member field. Initialized.

class static member field

b. class static member field. Uninitialized.

e. class static member field. Initialized.

The static is unique for the AppDomain.

thread relative static member field

c. thread relative static member field. Uninitialized.

f. thread relative static member field. Initialized.

The static is unique for the thread. The member field is not shared across threads.


Thread Pool HowTo

Queue a request to the Thread Pool

Class Library code -Perform BeginEnd and do EndRead on AsyncCallback

User Code

//---------------------------------------------------------------
// User code
//
// This is the code the application user will write
//
// Untrusted
//

class Foo
{
  //. . .
  // This AsyncCallback is called when IO Operations
  // complete asynchronously and are queued via
  // a thread pool thread
  //
  public void UserCallback(IAsyncResult ar)
  {
    // Extract AsyncFileStream from AsyncResult
    AsyncFileStream f =
                (AsyncFileStream)ar.AsyncObject;

    // End the Read call
    int bytesRead = f.EndRead(ar);

    // do something with the bytes read
    // and data in the byte array ‘b’
    //. . .
  }

  // The user opens the file in async and
  // issues an async read operation
  //

  public void Method2()
  {
    //. . .
    byte [] b;
    // Open the file in async mode
    AsyncFileStream f = File.OpenAsync("foo");

    //
    // The user can store any object they choose
    //
    Object state  = SomeUserStateObject;

    // Create new delegate
    // This delegate could be to a static method
    // This delegate could be shared across multiple
    // async calls
    //
    AsyncCallback ac = new AsyncDelegate(
                            this.UserCallback
                            );
    //Issue async read - pass byte array & size
    IAsyncResult ar = F.BeginRead (
                        b,
                        10,
                        ac,
                        state);
  }
  //. . .
}

Class Library code

File

//---------------------------------------------------------------
// This is code the class library writers could write.
//
// e.g. Async File Stream Code
//
// PInvoke capability needed
//
class File
{
  //. . .
  public static AsyncFileStream OpenAsync(String name)
  {
    //. . .
    // Open the file in Async mode
    AsyncFileStream f = new AsyncFileStream(name);
    //. . .
    return f;
  }
  //. . .
}

Async File Stream Async Result

// This is a async file stream class library specific AsyncResult
// The class library writer could hold any state they choose
// in the AsyncResult e.g. userCallback
//
class AsyncFileStream_AsyncResult : IAsyncResult
{
  //. . .

  // User code callback
  public AsyncCallback userCallback;

  // Regular fields
  public Object asyncObject;
  public Object userStateObject;

  //. . .

  // Regular IAsyncResult methods
  public Object AsyncObject
  {
      virtual get
      {
          return asyncObject;
      }
  }

  public Object AsyncState
  {
      virtual get
      {
          return userStateObject;
      }
  }
  //. . .
}

Async File Stream

class AsyncFileStream
{
  // This callback is called by COR when the IO operation completes
  // asynchronously and is queued via the thread pool
  //
  public static bool AsyncClassLibraryCallback(
    UInt32 errorCode,
    UInt32 numBytes,
    NativeOverlapped *pOVERLAP)
  {
     // Unpack overlapped
     Overlapped overlapped = Overlapped.Unpack(pOVERLAP);

     // Extract async result from overlapped
     AsyncFileStream_AsyncResult ar =
             (AsyncFileStream_AsyncResult)Overlapped.AsyncResult

    // Extract user callback from AsyncResult
    // then invoke user callback and pass in AsyncResult
    AsyncCallback ac = ar.userCallback;
    ac.Invoke(ar);
  }

  // Construct a new Async File Stream
  //
  public AsyncFileStream(String name)
  {
    //. . .
    // Open the file in Overlapped mode
    //
    int handle = Win32.CreateFile(
                  //. . .
                  FILE_FLAG_OVERLAPPED
                  //. . .
                  );

    IOCompletionCallback iocb = new IOCompletionCallback(
                   AsyncFileStream.AsyncClassLibraryCallback
                             );

    // Bind OS Handle with classLibrary Callback
    bool b = ThreadPool.BindHandleWithCallback(
                  handle,
                  );
    //. . .
  }

  // Async BeginRead operation
  //
  public IAsyncResult BeginRead(
                  byte[] b,
                  int size,
                  AsyncCallback cb,
                  Object stateObject)
  {
    // Make a new delegate
    IOCompletionCallback iocb =
      new IOCompletionCallback(
       AsyncFileStream.AsyncClassLibraryCallback
      );

    // Create and store async stream class library specific data in the
    // async result
    AsyncFileStream_AsyncResult ar = new AsyncFileStream_AsyncResult();
    ar.asyncObject = this;
    ar.userCallback = cb;
    ar.userStateObject = stateObject;

    // Create a managed overlapped class
    Overlapped overlapped = new Overlapped(0,0,0,ar);

    //. . .
    // Pack the Overlapped class
    NativeOverlapped *pOVERLAP = overlapped.Pack(iocb);

    // queue an async ReadFile operation and pass in
    // packed overlapped
    Win32.ReadFile(
           //. . .
           pOVERLAP,
           //. . .
        )
  }

  // Async EndRead operation
  //
  // Returns the number of bytes read
  //
  public int EndRead (IAsyncResult ar)
  {
    //. . .

    //. . .
  }
  //. . .
}


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Timer HowTo

Queue a Timer

Dispose a Timer

© 2000 Microsoft Corporation. All rights reserved.


Timer Reference

System.Threading.Timer

Timers are lightweight objects that enable you to specify a delegate to be called at a specified time. The wait operation is performed by a thread in the thread pool.

To add a timer, call the Timer.Timer constructor. To update a timer, call the Timer.Change method. You can specify a delegate to be executed by a worker thread from the thread pool when the timer expires.

A timer is set to the signaled state when its specified due time arrives. Any thread with a handle to the timer can use one of the wait functions to wait for the timer state to be set to signaled.

To cancel a pending timer, call the Timer.Dispose function.

public class Timer
{
  public Timer(TimerCallback   callback,
       Object        state,
       int            dueTime,
       int            period);
  public Timer(TimerCallback   callback,
       Object        state,
       TimeSpan       dueTime,
       TimeSpan     period);

  public Timer(TimerCallback   callback,
       Object        state,
       UInt32       dueTime,
       UInt32        period);

  public Timer(TimerCallback   callback,
       Object        state,
       long         dueTime,
       long          period);

  public bool Change(int dueTime, int period);
  public bool Change(TimeSpan dueTime, TimeSpan period);
  public bool Change(UInt32 dueTime, UInt32 period);
  public bool Change(long dueTime, long period);

  public bool Dispose(WaitHandle notifyObject);
  public bool Dispose();
}

Timer. Constructor that creates a timer-queue timer on the default timer queue. This timer expires at the specified due time, then after every specified period. When the timer expires, the delegate is called. The StateObject is passed to the TimerCallback when then Timer fires.

Change. Updates a timer-queue timer that was created by the Timer.AddTimer method.

Dispose. Deletes the timer from the time queue. When complete it signals the WaitHandle specified by notifyObject. An Application Runtime Exception is thrown if the timer fails to dispose.

System.Threading.TimerCallback

TimerCallback.

public delegate void TimerCallback(Object StateObject);

TimerCallback. Called when the timer fires. The StateObject passed in the Timer constructor.


Monitor HowTo

Acquire and Wait with Monitor

Monitor.Enter(b);
Monitor.Wait(b, 1000);
. . . // do some work
Monitor.Exit(b);


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

WaitHandle HowTo

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

ManualResetEvent HowTo

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

AutoResetEvent HowTo

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Mutex HowTo

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Interlocked HowTo

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

ReaderWriterLock HowTo

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

AppDomain Unhandle Exception HowTo

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Unmanaged Thread API HowTo

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Synchronized HowTo

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Thread Affinity HowTo

© 2000 Microsoft Corporation. All rights reserved.


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Threading Model Reference

© 2000 Microsoft Corporation. All rights reserved.


Thread Reference

Class for creating and managing a thread.

public class Thread
{
  public Thread(ThreadStart start);
  public void Start();
  public void Stop();
  [SecurityPermissionAttribute(SecurityAction.LinkDemand, Flags=SecurityPermissionFlag.UnmanagedCode)]
  public void Abort(Object stateInfo)
  public extern void Abort();
  [SecurityPermissionAttribute(SecurityAction.Demand, Flags=SecurityPermissionFlag.UnmanagedCode)]
  public static  void ResetAbort()
  public void Suspend();
  public void Resume();
  public extern void Interrupt();
  public ThreadPriority Priority {get;set}
  public bool IsAlive {get;}
  public void Join();
  public bool Join(int millisecondsTimeout);
  public bool Join(TimeSpan timeout)
  public static void Sleep(int millisecondsTimeout);
  public static void Sleep(TimeSpan timeout)
  public static Thread CurrentThread {get;}
  public bool IsBackground {get;set}
  public ThreadState ThreadState {get;}
  public ApartmentState ApartmentState {get;set}
  public static LocalDataStoreSlot AllocateDataSlot();
  public static LocalDataStoreSlot AllocateNamedDataSlot(String name);
  public static LocalDataStoreSlot GetNamedDataSlot(String name);
  public static void FreeNamedDataSlot(String name);
  public static Object GetData(LocalDataStoreSlot slot);
  public static void SetData(LocalDataStoreSlot slot, Object data);
  public CultureInfo CurrentUICulture {get;set};
  public CultureInfo CurrentCulture {get;set}
  public static Context CurrentContext {get;}
  public static IPrincipal CurrentPrincipal
  {
    get {}
    [SecurityPermissionAttribute(SecurityAction.Demand, Flags=SecurityPermissionFlag.ControlPrincipal)]
    set {}
  }
  public  String Name {get;set}

  internal Object ExceptionState
  {
  get { return m_ExceptionStateInfo;}
  set { m_ExceptionStateInfo = value;}
  }
}

Thread Constructor

Creates a new Thread object which will begin execution at start.ThreadStart on a new thread when the Start method is called.

Exceptions: ArgumentNullException if start == null.

Start

Creates a new Thread object which will begin execution at start.ThreadStart on a new thread when the Start method is called.

Exceptions: ArgumentNullException if start == null.

Stop

Raises a ThreadStopException in the thread, which will usually results in the thread's death (unless it catches and ignores the exception). The thread is not stopped immediately--you must Join on the thread to guarantee it has stopped. If Stop is called on a thread that has not been started, the thread will stop when Start is called. If Stop is called on a dead thread, this method does nothing.

ThreadStopException will cause finally clauses of try statements to be executed before the thread's "state" is marked ThreadState.ThreadStopped, and it has totally stopped.

public class TX
{
   public TX( )
   {
   }
   public void run( )
   {
      try
      {
         while (true)
         {
            StringBuilder   oSB   = new StringBuilder("AB");
            oSB.Append("C");
         }
      }
      finally
      {
         Console.WriteLine("Cleanup");
      }
   }
   public static void Main(String[] args)
   {
      TX   tx   = new TX( );
      Thread   t1   = new Thread(new ThreadStart(tx.run));
      t1.Start();
      Thread.Sleep(2000);
      t1.Stop();
      t1.Join();
   }

}

Abort

Raises a ThreadAbortException in the thread, which usually results in the thread's death. The ThreadAbortException is a special exception that is not catchable. The finally clauses of all try statements will be executed before the thread dies. This includes the finally that a thread might be executing at the moment the Abort is raised. The thread is not stopped immediately--you must Join on the thread to guarantee it has stopped. It is possible for a thread to do an unbounded amount of computation in the finally's and thus indefinitely delay the threads death. If Abort is called on a thread that has not been started, the thread will abort when Start is called. If Abort is called twice on the same thread, a DuplicateThreadAbort exception is thrown.

It is possible for a thread to do an unbounded amount of computation in the finally's and thus indefinitely delay the threads death. It is also possible for a thread to swallow the ThreadAbort in unmanaged code. The runtime will detect this and will re-throw the ThreadAbort exception when the thread returns back to managed code.

If Abort is called on a thread that has not been started, the thread will abort when Start is called. If Abort is called on a thread that has been suspended, the thread is resumed and then aborted. If Abort is called on a thread that is blocked or is sleeping, the thread is interrupted and then aborted.

If two aborts come at the same time, it is possible that the state info gets set by one, and the actual abort gets delivered by another. But this is not distinguishable by an application.

To call Abort, code needs to have the UnmanagedCode permission.

ResetAbort

Resets a thread abort. Should be called by trusted code only.

Code needs to have the UnmanagedCode permission to call ResetAbort.

AllocateDataSlot

Allocates an un-named data slot. The slot is allocated on ALL the threads.

AllocateNamedDataSlot

Allocates a named data slot. The slot is allocated on ALL the threads. Named data slots are "public" and can be manipulated by anyone.

GetNamedDataSlot

Looks up a named data slot. If the name has not been used, a new slot is allocated. Named data slots are "public" and can be manipulated by anyone.

FreeNamedDataSlot

Frees a named data slot. The slot is allocated on ALL the threads. Named data slots are "public" and can be manipulated by anyone.

GetData

Retrieves the value from the specified slot on the current thread.

SetData

Sets the data in the specified slot on the currently running thread.

CurrentContext

Returns the current content the thread is executing in.

CurrentPrincipal

Get and set thread's current principal (for role based security).

Name

Retrieves the name of the thread. If the component does not have a name, null is returned.

ApartmentState

An unstarted thread can be marked to indicate that it will host a single-threaded or multi-threaded apartment.

Exceptions:

ArgumentException if state is not a valid apartment state (STA or MTA).

Enum to represent the different threading models.

public enum ApartmentState
{
        STA = 0,
        MTA = 1,
        Unknown = 2,
};

IsAlive

Returns true if the thread has been started and is not dead.

Priority

Returns the priority of the thread.

Exceptions: ThreadStateException if the thread is dead.

Constants for thread priorities.

public enum ThreadPriority
{
        Lowest = 0,
        BelowNormal = 1,
        Normal = 2,
        AboveNormal = 3,
        Highest = 4,
};

CurrentThread

Returns the current thread the code is executing on.

IsBackground

If the thread is marked as background the Execution Engine will not wait until the thread has stopped when shutting down.

ThreadState

Return the thread state as a consistent set of bits. This is more general then IsAlive or IsBackground.

Constants for thread states.

public enum ThreadState
{
  Running = 0,
  StopRequested = 1,
  SuspendRequested = 2,
  Background = 4,
  Unstarted = 8,
  Stopped = 16,
  WaitSleepJoin = 32,
  Suspended = 64,
  AbortRequested = 128,
  Aborted = 256
};

Suspend

Suspends the thread. If the thread is already suspended, this call has no effect.

Exceptions: ThreadStateException if the thread has not been started or it is dead.

Resume

Resumes a thread that has been suspended.

Exceptions: ThreadStateException if the thread has not been started or it is dead or it isn't in the suspended state.

Interrupt

Interrupts a thread that is inside a Wait, Sleep or Join. If that thread is not currently blocked in that manner, it will be interrupted when it next begins to block.

Sleep.

Suspends the current thread for timeout milliseconds / timespan. If timeout == 0, forces the thread to give up the remainer of its timeslice. If timeout == Timeout.Infinite, no timeout will occur.

Exceptions:

ArgumentException if timeout < 0.

ThreadInterruptedException if the thread is interrupted while sleeping.

Join

Waits for the thread to die or for timeout milliseconds to elapse. Returns true if the thread died, or false if the wait timed out. If Timeout.Infinite is given as the parameter, no timeout will occur.

Exceptions:

ArgumentException if timeout < 0.

ThreadInterruptedException if the thread is interrupted while waiting.

ThreadStateException if the thread has not been started yet.


Thread Relative Statics Reference

ThreadRelativeAttribute Reference

Custom attributed that is used to mark a static field as being relative for each thread i.e. each thread has its own static field.

public class  ThreadStaticAttribute : Attribute
{
  public ThreadStaticAttribute()
  {
  }
}


Thread Pool Reference

ThreadPool Reference

Class for posting workitems to the threadpool. There is only one managed thread pool per process.

Restriction

  • Disallow ability to queue callbacks as an APC on the wait thread.
  • Disallow specification of a long computation.
public class ThreadPool
{
  public static RegisteredWaitHandle RegisterWaitForSingleObject(
        WaitHandle          waitObject,
        WaitOrTimerCallback   callBack,
        Object               state,
        uint            millisecondsTimeOutInterval,
        bool            executeOnlyOnce);

  public static RegisteredWaitHandle RegisterWaitForSingleObject(
        WaitHandle          waitObject,
        WaitOrTimerCallback   callBack,
        Object               state,
        int               millisecondsTimeOutInterval,
        bool            executeOnlyOnce        );
  public static RegisteredWaitHandle RegisterWaitForSingleObject(
        WaitHandle          waitObject,
        WaitOrTimerCallback   callBack,
        Object               state,
        long                millisecondsTimeOutInterval,
        bool            executeOnlyOnce        );

  public static RegisteredWaitHandle RegisterWaitForSingleObject(
        WaitHandle            waitObject,
        WaitOrTimerCallback   callBack,
        Object                state,
        TimeSpan              timeout,
        bool                  executeOnlyOnce
        )

  public static extern bool QueueUserWorkItem(
           WaitCallback           callBack,
         Object                 state
        );

  public static bool QueueUserWorkItem(WaitCallback callBack);
  public static bool BindHandle(int osHandle);
}

RegisterWaitForSingleObject

RegisterWaitForSingleObject. Registers an object so that when the timeout occurs the delegate is called. The RegisterWaitForSingleObject method queues the specified delegate to the thread pool. A worker thread will execute the delegate when one of the following occurs:

  • The specified object is in the signaled state.
  • The time-out interval elapses.

Returns the RegisteredWaitHandle object that encapsulates the native handle.

The RegisterWaitForSingleObject method checks the current state of the specified object WaitHandle. If the object's state is nonsignaled, the method registers a wait operation. The wait operation is performed by a thread from the system thread pool. The delegate is executed by a worker thread when the object's state becomes signaled or the time-out interval elapses.

Timeout in milliseconds. A value of –1 indicates no timeout. A value of 0 indicates immediate timeout. If timeOutInterval is not zero and executeOnlyOnce is not false, the timer is reset every time the event is signaled or the time-out interval elapses.

To cancel the wait operation, call the RegisteredWaitHandle.Unregister method. Do not make a blocking RegisteredWaitHandle.Unregister call within the delegate.

The wait thread uses the WaitForMultipleObjects function to monitor registered wait operations. Therefore, if you must use the same native OS handle in multiple calls to RegisterWaitForSingleObject, you must duplicate the handle using the Win32 DuplicateHandle function. Note that you should not pulse an event object passed to RegisterWaitForSingleObject, because the wait thread might not detect that the event is signaled before it is reset.

Before returning, the function modifies the state of some types of synchronization objects. Modification occurs only for the object whose signaled state caused the wait condition to be satisfied. For example, the count of a semaphore object is decreased by one.

The RegisterWaitForSingleObject method can wait for the following objects:

  • Change notification
  • Console input
  • Event
  • Job
  • Mutex
  • Process
  • Semaphore
  • Thread
  • Waitable timer

QueueUserWorkItem

QueueUserWorkItem. Queues a user work item to the thread pool. State data can be passed during queuing. By default, the callback function is queued to a non-I/O worker thread.

BindHandle

BindHandle. Bind an OS handle to ThreadPool.

NativeOverlapped Reference

The NativeOverlapped value type has explicit layout that is visible from unmanaged code and will have the same layout as the Win32 OVERLAPPED structure with additional reserved field at the end. The NativeOverlapped NGWS runtime valuetype represents the (unmanaged) Win32 OVERLAPPED structure the layout of this structure must be identical to OVERLAPPED. Two additional dwords are reserved at the end

[structlayout(LayoutKind.Sequential)]
public struct NativeOverlapped
{
  public int   internalLow;
  public int   internalHigh;
  public int   offsetLow;
  public int   offsetHigh;
  public int   eventHandle;
  public int   reservedCOR;
  public int   reservedClasslib;
}

reservedCOR. Reserved for COR. COR manages this field.

reservedClassLib. Reserved for Class Library data. COR manages this field.

The Win32 OVERLAPPED structure

typedef struct _OVERLAPPED
{
    DWORD  Internal;
    DWORD  InternalHigh;
    DWORD  Offset;
    DWORD  OffsetHigh;
    HANDLE hEvent;
} OVERLAPPED;

Overlapped Reference

This class is used to pack and unpack the managed overlapped to and from the native overlapped.

public class Overlapped
{
  public Overlapped() { }

  public Overlapped(int offsetLo,
                    int offsetHi, int hEvent, IAsyncResult ar);

  public IAsyncResult AsyncResult { get; set }
  public int OffsetLow {get; set}
  public int OffsetHigh {get; set}
  public int EventHandle {get; set}

  public NativeOverlapped* Pack(
                 IOCompletionCallback iocb)

  public static Overlapped Unpack(
                 NativeOverlapped* nativeOverlappedPtr)
}

Overlapped. Construct an empty overlapped.

  Overlapped();

Overlapped. Construct an overlapped from the constructor arguments.

Overlapped(int offsetLo, int offsetHi, int hEvent, IAsyncResult ar);

AsyncResult. Property. IAsyncResult interface.

EventHandle. Property. Maps to OVERLAPPED.

OffsetLow. Property. Maps to OVERLAPPED.

OffsetHigh. Property. Maps to OVERLAPPED.

Pack. Packs a managed overlapped class into a NativeOverlapped value type. An unmanaged pointer is returned. This unmanaged pointer can be passed to the OS in overlapped IO operations. The NativeOverlapped value type is fixed in physical memory until Unpack is called.

public NativeOverlapped* Pack(IOCompletionCallback iocb)

Unpack. Unpacks an unmanaged NativeOverlapped value type into a managed overlapped class. The NativeOverlapped value type is freed from physical memory.

public static Overlapped Unpack(NativeOverlapped* nativeOverlappedPtr)

IOCompletionCallback Reference

Delegate that is called when an IO operation completes on the thread pool.

The callback receives the error code, number of bytes and overlapped value type.

public delegate void IOCompletionCallback(
          uint errorCode,
          uint numBytes,
          NativeOverlapped* pOVERLAP);

errorCode. Error code

numBytes. No. of bytes transferred

pOVERLAP. Unmanaged pointer to native oOverlapped value type

WaitCallback Reference

This class is a Delegate which defines the callback method for threadpool user work items. That method must match this delegate.

public delegate void WaitCallback(
                        Object StateObject
                     );

WaitCallback.

WaitOrTimerCallback Reference

Wait or Timer Delegate definition.

public delegate void WaitOrTimerCallback(
                        boolean wasSignalled,
                        Object StateObject
                  );  // signalled or timed out

WaitOrTimerCallback. Receives a boolean parameter which determines whether the handle was signaled or the WaitHandle timed out.


Monitor Reference

public sealed class Monitor
{
  public static void Enter(Object obj);
  public static void Exit(Object obj);
  public static bool TryEnter(Object obj);
  public static bool TryEnter(Object obj, int millisecondsTimeout);
  public static bool TryEnter(Object obj, TimeSpan timeout);
  public static bool Wait(Object obj, int millisecondsTimeout, bool exitContext);
  public static bool Wait(Object obj, TimeSpan timeout, bool exitContext);
  public static bool Wait(Object obj, int millisecondsTimeout);
  public static bool Wait(Object obj, TimeSpan timeout);
  public static bool Wait(Object obj, bool exitContext);
  public static bool Wait(Object obj);
  public static void Pulse(Object obj);
  public static void PulseAll(Object obj);
}

Enter

public static void Enter(Object obj);

Obtain the monitor lock of obj. Will block if another thread holds the lock. Will not block if the current thread holds the lock, however the caller must ensure that the same number of Exit calls are made as there were Enter calls.

Enter the Monitor within an object. An object is passed as a parameter. If object parameter is null a ArgumentNullException is thrown. This call will block if another thread has entered the Monitor of the same object. It will not block if the current thread has previously entered the Monitor, however the caller must ensure that the same number of Exit calls are made as there were Enter calls.

Thread.Interrupt can interrupt threads waiting to enter a Monitor on an object. A ThreadInterrupted exception will be thrown.

Exceptions: ArgumentNullException if object is null.

Exit

public static void Exit(Object obj);

Release the monitor lock. If one or more threads are waiting to acquire the lock, and the current thread has executed as many Exits as Enters, one of the threads will be unblocked and allowed to proceed.

Leave a Monitor. If one or more threads are waiting to enter the critical section, and the current thread has executed as many Exits as Enters, one of the threads will be unblocked and allowed to proceed.

Exceptions:

ArgumentNullException if object is null.

SynchronizationLockException if this method is not called inside a synchronized block of code.

Pulse

public static void Pulse(Object obj)

Sends a notification for a single waiting object

Exceptions:

ArgumentNullException if object is null.

SynchronizationLockException if this method is not called inside a synchronized block of code.

PulseAll

public static void PulseAll(Object obj)

Sends a notification to all waiting objects. SynchronizationLockException if this method is not called inside a synchronized block of code.

Exceptions:

ArgumentNullException if object is null.

SynchronizationLockException if this method is not called inside a synchronized block of code.

TryEnter

public static bool TryEnter(Object obj);

Similar to Enter, but will never block. That is, if the current thread can acquire the monitor lock without blocking, it will do so and TRUE will be returned. Otherwise FALSE will be returned.

Exceptions:

ArgumentNullException if object is null.

TryEnter

public static bool TryEnter(Object obj, int timeout)

Version of TryEnter that will block, but only up to a timeout period expressed in milliseconds. If timeout == Timeout.Infinite the method becomes equivalent to Enter.

Exceptions:

ArgumentNullException if object is null.

ArgumentException if timeout < 0.

Wait

public static bool Wait(Object obj);

Waits for notification from the object (via a Pulse/PulseAll).

Exceptions:

ArgumentNullException if object is null.

SynchronizationLockException if this method is not called inside a synchronized block of code.

Wait

public static bool Wait(Object obj, int millisecondsTimeout);

Waits for notification from the object (via a Pulse/PulseAll). millisecondsTimeout indicates how long to wait before the method returns.

Exceptions:

ArgumentNullException if object is null.

SynchronizationLockException if this method is not called inside a synchronized block of code.

Wait

public static bool Wait(Object obj, TimeSpan timeout);

Waits for notification from the object (via a Pulse/PulseAll). TimeSpan indicates how long to wait before the method returns.

Exceptions:

ArgumentNullException if object is null.

SynchronizationLockException if this method is not called inside a synchronized block of code.

Wait

public static bool Wait(Object obj, int millisecondsTimeout, bool exitContext);

Waits for notification from the object (via a Pulse/PulseAll). millisecondsTimeout indicates how long to wait before the method returns.

If exitContext is true then the synchronization domain for the context (if in a synchronized context) is exited before the wait and reacquired

Exceptions:

ArgumentNullException if object is null.

SynchronizationLockException if this method is not called inside a synchronized block of code.

Wait

public static bool Wait(Object obj, TimeSpan timeout, bool exitContext);

Waits for notification from the object (via a Pulse/PulseAll). timeout indicates how long to wait before the method returns.

If exitContext is true then the synchronization domain for the context (if in a synchronized context) is exited before the wait and reacquired

Exceptions:

ArgumentNullException if object is null.

SynchronizationLockException if this method is not called inside a synchronized block of code.


WaitHandle Reference

public  class WaitHandle
{
  public WaitHandle();

  public virtual int GetHandle();

  [SecurityPermissionAttribute(SecurityAction.LinkDemand, Flags=SecurityPermissionFlag.UnmanagedCode)]
  public virtual void SetHandle(int handle);

  public virtual bool WaitOne (int millisecondsTimeout, bool exitContext);
  public virtual bool WaitOne (TimeSpan timeout, bool exitContext);
  public virtual bool WaitOne ();

  public static bool WaitAll(
              WaitHandle[] waitHandles,
              int millisecondsTimeout,
              bool exitContext);
  public static bool WaitAll(
                   WaitHandle[] waitHandles,
                   TimeSpan timeout,
                  bool exitContext);
  public static bool WaitAll(WaitHandle[] waitHandles);

  public static int WaitAny(
              WaitHandle[] waitHandles,
              int millisecondsTimeout,
              bool exitContext);
  public static int WaitAny(
                   WaitHandle[] waitHandles,
                   TimeSpan timeout,
                  bool exitContext);
  public static int WaitAny(WaitHandle[] waitHandles);

  public void Close();
}

Notes:

  • An ArgumentNullException will be thrown if any of the objects passed to WaitHandle.Wait[Any/All] is null.
  • An ArgumentNullException will be thrown if duplicate objects are passed to WaitHandle.Wait[Any/All].

Close

Close the OS handle.

WaitHandle

Construct a new WaitHandle object.

GetHandle

Return the native OS handle.

SetHandle

Set the native OS handle.

WaitOne

Waits for the WaitHandle to be signaled. timeout indicates how long to wait before the method returns. If exitContext is true then the synchronization domain for the context (if in a synchronized context) is exited before the wait and reacquired

WaitAll

Takes an array of WaitHandles and a timeout in milliseconds. Returns when the wait terminates, either when all the objects are signaled or timeout occured. On some implementations if more that 64 Objects are passed a NotSupportedException is thrown. If there are duplicates in the array the call will fail.

If exitContext is true then the synchronization domain for the context (if in a synchronized context) is exited before the wait and reacquired

Shorthand for WaitAll with timeout = Timeout.Infinite and exitContext = true

Shorthand for WaitAll with exitContext = true

WaitAny

Takes an array of WaitHandles and a timeout. Returns when the wait terminates, either when any of the handles are signaled or timeout occured. The return value indicates the array index of the object that satisfied the wait. If more than one object became signaled during the call, this is the array index of the signaled object with the smallest index value of all the signalled objects. On some implementations if more that 64 Objects are passed a NotSupportedException is thrown. If there are duplicates in the array the call will fail.

If exitContext is true then the synchronization domain for the context (if in a synchronized context) is exited before the wait and reacquired

Shorthand for WaitAny with timeout = Timeout.Infinite and exitContext = true

Shorthand for WaitAny with exitContext = true


RegisteredWaitHandle Reference

Represents a handle that has been registered when calling ThreadPool.RegisterWaitForSingleObject.

public class RegisteredWaitHandle
{
   public boolean Unregister(
                     WaitHandle waitObject
                  );
}

Unregister. Cancels a registered wait operation issued by the ThreadPool. RegisterWaitForSingleObject function. Specifies the WaitHandle to be signaled when the function is successful and all callback functions have completed. If this parameter is NULL, the method returns immediately. If this method is –1, the function does not return until the event has been canceled and all associated callback functions have completed.


ManualResetEvent Reference

public class ManualResetEvent : WaitHandle
{
   public ManualResetEvent(boolean initialState);
   public boolean Reset();
   public boolean Set();
}

ManualResetEvent

Initializes a new ManualResetEvent. InitialState specifies the initial state of the event object. If TRUE, the initial state is signaled; otherwise, it is nonsignaled.

Reset

The Reset method sets the state of the specified event object to nonsignaled.

Set

The Set method sets the state of the specified event object to signaled.


AutoResetEvent Reference

public class AutoResetEvent : WaitHandle
{
   public AutoResetEvent(boolean initialState);
   public boolean Set();
}

AutoResetEvent

Initializes a new AutoResetEvent. InitialState specifies the initial state of the event object. If TRUE, the initial state is signaled; otherwise, it is nonsignaled.

Set

The Set method sets the state of the specified event object to signaled.


Mutex Reference

public sealed class Mutex : WaitHandle
{
  public Mutex(bool initiallyOwned, String name);
  public Mutex(bool initiallyOwned);
  public Mutex();
  public void ReleaseMutex();
}

Mutex

public Mutex()              ==  Mutex(false, null)
public Mutex(bool initiallyOwned)      ==  Mutex(initiallyOwned, null)
public Mutex (bool initiallyOwned,    String name)

If true, and the caller created the OS mutex, it owns the created mutex. Ignored if the named OS mutex already existed.

Name of the mutex, can be null

ReleaseMutex

Releases the mutex once.


Interlocked Reference

public class Interlocked
{
  public static int Increment(ref int location);
  public static int Decrement(ref int location);
  public static int Exchange(ref int location1, int value);
  public static int CompareExchange(ref int location1, int value, int comparand);
  public static float Exchange(ref float location1, float value);
  public static float CompareExchange(ref float location1, float value, float comparand);
  public static Object Exchange(ref Object location1, Object value);
  public static Object CompareExchange(ref Object location1, Object value, Object comparand);
}

CompareExchange – int

Compares two values for equality and replaces the destination value if they are equal.

CompareExchange – float

Compares two floating point values for equality and replaces the destination value if they are equal.

CompareExchange – object

Compares two Objects for equality and replaces the destination value if they are equal.

Increment

Increments a specified variable as an atomic operation

Decrement

Decrements a specified variable as an atomic operation.

Exchange – int

Sets a variable to a specified value as an atomic operation.

Exchange – float

Sets a variable to a specified value as an atomic operation.

Exchange - object

Sets an Object variable to a specified value as an atomic operation and returns the original value.


ReaderWriterLock Reference

public class ReaderWriterLock
{
  public ReaderWriterLock()
  public bool IsReaderLockHeld {get;}
  public bool IsWriterLockHeld {get;}
  public int WriterSeqNum {get;}
  public void AcquireReaderLock(int millisecondsTimeout);
  public void AcquireReaderLock(TimeSpan timeout)
  public void AcquireWriterLock(int millisecondsTimeout);
  public void AcquireWriterLock(TimeSpan timeout)
  public void ReleaseReaderLock();
  public void ReleaseWriterLock();
  public LockCookie UpgradeToWriterLock(int millisecondsTimeout);
  public LockCookie UpgradeToWriterLock(TimeSpan timeout)
  public void DowngradeFromWriterLock(ref LockCookie lockCookie);
  public LockCookie ReleaseLock();
  public void RestoreLock(ref LockCookie lockCookie);
  public bool AnyWritersSince(int seqNum);
}

ReaderWriterLock

Constructor

IsReaderLockHeld

Property that returns TRUE if the reader lock is held by the current thread

IsWriterLockHeld

Property that returns TRUE if the writer lock is held by the current thread

WriterSeqNum

Property that returns the current writer sequence number. The caller should be a reader or writer for getting meaningful results

AcquireReaderLock

Acquires reader lock. The thread will block if a different thread has writer lock.

AcquireWriterLock

Acquires writer lock. The thread will block if a different thread has reader lock. It will dead lock if this thread has reader lock. Use UpgardeToWriterLock when you are not sure if the thread has reader lock

ReleaseReaderLock

Releases reader lock.

ReleaseWriterLock

Releases writer lock.

UpgradeToWriterLock

Upgrades the thread to a writer. If the thread has is a reader, it is possible that the reader lock was released before writer lock was acquired.

DowngradeFromWriterLock

Restores the lock status of the thread to the one it was in when it called UpgradeToWriterLock.

ReleaseLock

Releases the lock irrespective of the number of times the thread

acquired the lock

RestoreLock

Restores the lock status of the thread to the one it was in when it called ReleaseLock.

AnyWritersSince

Returns true if there were intermediate writes since the sequence number was obtained. The caller should be a reader or writer for getting meaningful results


App Domain Reference

AppDomain Reference

class AppDomain
{
  //. . .
  public void AddOnUnhandledException(
               UnhandledExceptionEventHandler handler)

  public void RemoveOnUnhandledException(
               UnhandledExceptionEventHandler handler)
  //. . .
}

AddOnUnhandledException

A domain can be notified when an unhandled exception happens

RemoveOnUnhandledException.

Unsubscribe from the unhandled exception event

UnhandledExceptionEvent Reference

public class UnhandledExceptionEvent : EventArgs

{

public UnhandledExceptionEvent(Object exception, bool isTerminating);

public Object ExceptionObject { get; }

public bool IsTerminating { get; }

}

UnhandledExceptionEventHandler Reference

public delegate void UnhandledExceptionEventHandler(Object sender, UnhandledExceptionEvent e);


Unmanaged Thread Reference

ICorThreadpool interface

The ICorThreadpool interface methods in mscoree.dll can be used through the combination of cortpoolhdr.h, mscoree.h, mscoree.idl, and mscoree.lib

typedef VOID (__stdcall *WAITORTIMERCALLBACK)(PVOID, BOOL);

interface ICorThreadpool : IUnknown
{
  //. . .
  HRESULT CorRegisterWaitForSingleObject(
                     [in] HANDLE* phNewWaitObject,
                     [in] HANDLE hWaitObject,
                     [in] WAITORTIMERCALLBACK Callback,
                     [in] PVOID Context,
                     [in] ULONG timeout,
                     [in] BOOL  executeOnlyOnce,
                     [out] BOOL* result  );

  HRESULT CorUnregisterWait(
                     [in] HANDLE hWaitObject,
                     [in] HANDLE CompletionEvent,
                     [out] BOOL* result);

  HRESULT CorQueueUserWorkItem(
                     [in] LPTHREAD_START_ROUTINE Function,
                     [in] PVOID Context,
                     [in] BOOL executeOnlyOnce,
                     [out] BOOL* result );

  HRESULT CorCreateTimer(
                      [in] HANDLE* phNewTimer,
                      [in] WAITORTIMERCALLBACK Callback,
                      [in] PVOID Parameter,
                      [in] DWORD DueTime,
                      [in] DWORD Period,
                      [out] BOOL* result);

  HRESULT CorChangeTimer(
                      [in] HANDLE Timer,
                      [in] ULONG DueTime,
                      [in] ULONG Period,
                      [out] BOOL* result);

  HRESULT CorDeleteTimer(
                      [in] HANDLE Timer,
                      [in] HANDLE CompletionEvent,
                      [out] BOOL* result);

  HRESULT CorBindIoCompletionCallback(
                      [in] HANDLE fileHandle,
                      [in] LPOVERLAPPED_COMPLETION_ROUTINE callback);
  // . . .
}

CorQueueUserWorkItem

The CorQueueUserWorkItem function queues a work item to a worker thread in the thread pool.

CorRegisterWaitForSingleObject

The CorRegisterWaitForSingleObject function queues the specified callback function to the thread pool. A worker thread will execute the function when one of the following occurs:

  • The specified object is in the signaled state.
  • The time-out interval elapses.

The CorRegisterWaitForSingleObject function checks the current state of the specified object. If the object's state is nonsignaled, the function registers a wait operation. The wait operation is performed by a thread from the thread pool. The callback routine is executed by a worker thread when the object's state becomes signaled or the time-out interval elapses.

To cancel the wait operation, call the CorUnregisterWait function. Do not make a blocking CorUnregisterWait call within the callback function.

Internally the wait thread use WaitForMultipleObjects to monitor registered wait operations. Therefore, if you must use the same handle in multiple calls to CorRegisterWaitForSingleObject, you must duplicate the handle using the DuplicateHandle function. Note that you should not pulse an event object passed to CorRegisterWaitForSingleObject, because the wait thread might not detect that the event is signaled before it is reset.

Before returning, the function modifies the state of some types of synchronization objects. Modification occurs only for the object whose signaled state caused the wait condition to be satisfied. For example, the count of a semaphore object is decreased by one.

The CorRegisterWaitForSingleObject function can wait for the following objects:

  • Change notification
  • Console input
  • Event
  • Job
  • Mutex
  • Process
  • Semaphore
  • Thread
  • Waitable timer

CorUnregisterWait

The CorUnregisterWait function cancels a registered wait operation issued by the CorRegisterWaitForSingleObject function.

If the wait thread is executing the callback function, you can call CorUnregisterWait inside the callback function. If a worker thread is executing the callback function, do not call CorUnregisterWait inside the callback function.

If CompletionEvent is not NULL or –1, no new callback functions can be executed. However, there may be callback functions already queued to worker threads whose execution has not started.

CorCreateTimer

The CorCreateTimer function creates a timer. This timer expires at the specified due time, then after every specified period. When the timer expires, the callback function is called.

If the DueTime and Period parameters are both nonzero, the timer will be signaled first at the due time, then periodically.

To cancel a timer, call the CorDeleteTimer function.

CorDeleteTimer

The CorDeleteTimer function cancels a timer.

You can set CompletionEvent to –1 when calling this function from within the timer callback of another timer as long as the callback function is not executed in the timer thread. However, a deadlock may occur if two callback functions attempt a blocking CorDeleteTimer call on each others' timers.

CorChangeTimer

The CorChangeTimer function updates a timer that was created by the CorCreateTimer function.

You can call CorChangeTimer in a timer callback.

If you call CorChangeTimer on a one-shot timer (it's period is zero) that has already expired, the timer is not updated.

Do not call CorChangeTimer after calling CorDeleteTimer on a handle.

CorBindIOCompletionCallback

The CorBindIOCompletionCallback allows users associate an OS handle with the thread pool and specify a completion callback. When async IO operations complete on the OS handle the callback is called.

osHandle. The OS handle to bind to the IO completion port.

callback. The callback to be called when an async operation completes on the IO completion port.

LPOVERLAPPED_COMPLETETION_ROUTINE has the following signature:

VOID OVERLAPPED_COMPLETETION_ROUTINE(
     DWORD dwErrorCode,                // completion code
     DWORD dwNumberOfBytesTransfered,  // number of bytes transferred
     LPOVERLAPPED lpOverlapped         // I/O information buffer
   );


Synchronization Reference

package System.Remoting;

public class Synchronization
    : ContextAttribute, IContributeServerContextSink,
                IContributeClientContextSink
{
    . . .
    public static final int NOT_SUPPORTED  = 0x00000001;
    public static final int SUPPORTED      = 0x00000002;
    public static final int REQUIRED       = 0x00000004;
    public static final int REQUIRES_NEW   = 0x00000008;

    public Synchronization();
    public Synchronization(boolean reEntrant);
    public Synchronization(int flag);
    public Synchronization(int flag, boolean reEntrant);
    . . .
}

Synchronization Flags

NOT_SUPPORTED. The class should not be instantiated in a context that has Synchronization.

SUPPORTED. The class does not care if the context has Synchronization or not.

REQUIRED. The class should be instantiated in a context that has Synchronization.

REQUIRES_NEW. The class should be instantiated in a context with a new instance of Synchronization property each time

Synchronization

Synchronization(). Default constructor. flag=REQUIRED and reEntrant=false.

Synchronization(boolean reEntrant). If reEntrant is true, other calls are allowed to come in if the currently running call leaves the domain for a call-out. Flag=REQUIRED.

Synchronization(int flag). flag=one of the Synchronization flags. reEntrant = false

Synchronization(int flag, boolean reentrant). Flag=one the of Synchronization flags and reentrant=true othersise false.Synchronization

For reference details see: Synchronization.


ThreadAffinity Reference

public class ThreadAffinity
{
  public const int NOT_SUPPORTED     = 0x00000001;
  public const int SUPPORTED         = 0x00000002;
  public const int REQUIRED       = 0x00000004;
  public const int REQUIRES_NEW    = 0x00000008;

  public ThreadAffinity();
  public ThreadAffinity(bool bReEntrant);
  public ThreadAffinity(int flag);
  public ThreadAffinity(int flag, bool bReEntrant);
}

ThreadAffinity Flags

NOT_SUPPORTED. The class should not be instantiated in a context that has ThreadAffinity.

SUPPORTED. The class does not care if the context has ThreadAffinity or not.

REQUIRED. The class should be instantiated in a context that has ThreadAffinity.

REQUIRES_NEW. The class should be instantiated in a context with a new instance of ThreadAffinity property each time

ThreadAffinity

ThreadAffinity(). Default constructor. flag=REQUIRED and reEntrant=false.

ThreadAffinity (boolean reEntrant). If reEntrant is true, other calls are allowed to come in if the currently running call leaves the domain for a call-out. Flag=REQUIRED.

ThreadAffinity(int flag). flag=one of the Synchronization flags. reEntrant = false

ThreadAffinity (int flag, boolean reentrant). Flag=one the of Synchronization flags and reentrant=true otherwise false.Synchronization

For reference details see: ThreadAffinity


Versioning in NGWS

The versioning problems encountered with Windows applications today exist in part because there is no way for an application developer to express versioning rules between pieces of an application and have those rules honored and enforced by the system. Instead, our current approach to versioning is based upon developers following "common-sense" coding practices. These practices state that interface definitions are static once published, and that a single piece of code must maintain backward compatibility with previous versions. Furthermore, code is typically designed so that only a single version of it can be present and executing on a machine at any given time.

The versioning problem is further complicated by the fact that there is no way to maintain consistency between the set of components that were built together and the set that will be present at run time.

The manifestation of these versioning problems is one aspect of what we refer to as DLL Hell. Specifically, installing one application can inadvertently break an existing application because a certain software component was installed that was not fully backward compatible with a previous version. Once this situation occurs, there is no support in the system itself for diagnosing and fixing the problem.

To solve the versioning problem, the runtime must:

  • Allow developers to specify version rules between different software components,
  • Provide the infrastructure to allow multiple versions of a software component to be running simultaneously (we refer to this as side by side).

The remainder of this document describes how assemblies are used as the means for specifying version rules and how the NGWS runtime supports the concept of side by side execution.


Assemblies, Versioning and Identity

One of the primary goals of assemblies in NGWS is to solve the versioning problem. Specifically, assemblies provide a means for developers to specify version rules between different software components and have those rules enforced at runtime.

An assembly is the fundamental unit of sharing and reuse in NGWS. As such, it follows that assemblies are the unit at which version information is specified and enforced.

Each assembly has two version “numbers”. The first is called a compatibility version. This version number is part of the assembly’s identity and is a key part in the type resolution process at Runtime. We call it a compatibility number because the semantics of this number make statements about a particular version’s compatibility with previous versions. The second version “number” is called an informational version. The informational version is actually a string and is used to represent additional version information that is included for documentation purposes only. This version information corresponds to the product’s marketing literature, packaging, etc.. and is not used at runtime at all. For example, an informational version may be something like “NGWS SDK Beta1”, or “Acme Control SP 2”.


Compatibility Version

Each assembly has a specific compatibility version number as part of its identity. As such, two assemblies that differ by compatibility version are completely different assemblies as far as the NGWS runtime class loader is concerned.

This compatibility version number is physically represented as a four part number with the following format:

<major version>.<minor version>.<build number>.<revision>

Each portion of this number has a specific meaning to the Assembly Resolver and the class loader as they decide which version of an assembly to load. Logically, the compatibility version number has three parts, with the following meanings:

incompatible: A change has been made to the assembly that is known to be incompatible with previous versions. Example: Major new release of the product.

maybe compatible: A change has been made to the assembly that is thought to be compatible and carries less risk than an incompatible change. However, backwards compatibility is not guaranteed. Example is a Service Pack or the release of a new daily build.

QFE: An emergency fix that customers must upgrade to. Example: emergency security fix.

These three logical parts map to the physical 4-part version number as follows:

For example, an assembly with compatibility version number 5.4.0.0 is considered incompatibly with an assembly whose compatibility number is 4.1.11.00. Also, compatibility number 5.4.0.23 is considered a QFE to compatibility number 5.4.0.0.


Informational Version

As describe above, the informational version is a string used to attach additional version information to an assembly for documentation purposes only – this data is never used at runtime. In Beta2 this information will be displayed by admin tools and the property pages visible in the shell.

The information version is represented using the custom attribute System.Reflection.AssemblyInformationalVersionAttribute. The definition of that attribute is:


Assembly References and Version Policy

At build time, development tools record dependency information for each assembly that is referenced. This information is kept in the assembly manifest as AssemblyRefs. A key element of an AssemblyRef is the compatibility version number of the dependent assembly. The NGWS runtime uses these version numbers to load the proper version of an assembly at runtime.

The Assembly Resolver distinguishes between private and shared assemblies for the purposes of versioning. Specifically, no version checking is done when resolving references to private assemblies – the description of version checking described in this document applies only to references to shared assemblies.

The philosophy behind not checking versions on private assemblies is that such assemblies are authored by the same person or organization that produced the application that uses the assembly. Therefore, the private assembly is deployed along with the app and is placed in the same directory as the app, or a subdirectory thereof. In other words, the app author has control over the contents and distribution of the assembly.

The distinction between a reference to a private assembly verses a shared assembly is the presence of an Originator in the reference. The Originator is the public key used to give the assembly a shared name. All references to shared assemblies, i.e the reference contains an Originator, follow the version checking rules described in this document regardless of where they are deployed.

Note that it is possible to reference a shared assembly, but not provide the Originator in the reference. In this case, the Assembly Resolver will follow the binding rules for private assemblies.

The manifest records the compatibility version number of each dependency at the time your assembly was compiled. However, in many cases, it may be acceptable to run with different versions of the assemblies than those present at build time. For example, even if version 5 of a dependency was present at build time, a more recent version may be acceptable at runtime. The rules that specify the acceptable versions of an assembly are called version policies. Again, all version policies are based on the compatibility version number, not the informational version. Version polices are expressed using the <BindingPolicy> tag in the configuration files. Configurations can also be used to change version policy if a particular version of a dependency causes an application to stop working. In this case, a configuration can be used to "roll back" to the specific version of a dependency that was present at build time.

The NGWS runtime supports the following version policies:

  • Default Version Policy
  • Bind to a specific version
  • Don’t Accept Implicit QFE's
  • Safe Mode

Default Version Policy

The default version policy is to take the version of the dependent assembly that is recorded in the manifest and match major and minor numbers exactly then take the latest build and revision numbers.

Mapping back to the semantic parts of the version number described above, the default policy binds to the exact “incompatible” version, then looks in the global assembly cache for the latest “maybe_compatible” and “QFE” versions. For example, if the Assembly Resolver is given a reference to version 5.0.11.0, and a version 5.0.11.12 is present in the assembly cache, version 5.0.11.12 will be returned.

A few points about this default policy are worth stressing. First the “use latest” behavior described above only applies to the build and revision portions of the compatibility version number. The policy of taking the latest build and revision is often referred to as Automatic QFE Policy. This QFE policy allows software vendors to release critical bug fixes without requiring dependent applications to be rebuilt or reconfigured. It is worth noting that this policy is implicit in that there is nothing that the NGWS runtime does to enforce that a "QFE" is in fact backward compatible with the version it is replacing. We never implicitly take the latest major and minor numbers because they are, by definition, incompatible with what is being requested.

Second, the Assembly Resolver only looks in the global assembly cache for QFE’s. If a search path is specified through configuration, the Assembly Resolver DOES NOT look at all assemblies on the path searching for the one with the highest build and revision number. The performance cost of doing so is prohibitive.

Bind to Specific Version

There are times where it may be desirable to bind to a different version of an assembly than the version recorded in the manifest. Support for this scenario is provided through the <BindingPolicy> tag in the configuration files. This policy can be used to map either references to a specific version of a dependency or references to all versions of a dependency.

The following example maps references to all versions of an assembly called calcr to version 6.0.0.0:

   <BindingPolicy>
      <BindingRedir Name="Calcr"
                    Originator="32ab4ba45e0a69a1"
                    Version="*" VersionNew="6.0.0.0"
                    UseLatestBuildRevision="yes"/>
   </BindingPolicy>

Don’t Accept Implicit QFE's

The two policies described so far both employ automatic QFE policy. There are clearly times where this is not desirable. For example, consider the scenario where an administrator installs a bug fix release (QFE) to a shared assembly that ends up working for most applications, but breaks others. In this scenario, the administrator would like to disable the automatic QFE policy for those applications that have broken because of the update. The support for this scenario is also in the <BindingPolicy> tag. BindingPolicy has an attribute called UseLatestBuildRevision that can be set to “no” to disable automatic QFE policy for references to a particular assembly. For example, the following XML snippet binds all references to calcr to exactly version 6.1.1212.14 – no QFE’s to that version will be picked up:

   <BindingPolicy>
      <BindingRedir Name="Calcr"
                    Originator="32ab4ba45e0a69a1"
                    Version="*" VersionNew="6.1.1212.14"
                    UseLatestBuildRevision="no"/>

</BindingPolicy>

Safe Mode

Safe mode is the version policy used to ensure that the versions of the assemblies that were built and tested together are the same versions that are run together after they are deployed.  If an application passed tests and worked correctly prior to deployment, the safe mode version policy can be used to revert to this known state in the presence of failures caused by running with incompatible  (or just buggy) versions of dependent assemblies.  Safe mode always applies to an entire application, not to individual assembly references.  To get similar behavior for a particular reference use <BindingPolicy> to bind to a particular version and turn off automatic QFE policy.

When running with the safe mode policy, the Assembly Resolver ignores any other version policies and binds to exactly the version of the dependent assembly that is recorded in the manifest.  All four parts of the version number are checked to ensure an exact match – no QFE’s are applied.

Safe Mode is turned on using the <BindingMode> tag in the configuration files.  The following XML snippet turns on Safe Mode for an application:

   <BindingMode>
      <AppBindingMode Mode="safe"/>
   </BindingMode>


Side by Side

The ability to run multiple versions of the same code simultaneously is referred to as side-by-side execution. Supporting side-by-side is essential to providing robust versioning in the NGWS runtime. Code that is side-by-side capable has more flexibility it terms of providing compatibility with previous versions. For example, code that cannot run side-by-side must remain completely backward compatible with all previous versions because only one version (typically the latest) can be running on the system at the same time. Given a robust versioning system, side-by-side components do not need to maintain strict backwards compatibility. Consider the situation where ClassX supports side-by-side and makes an incompatible change between version 1 and version 2. Callers of ClassX that expressed a dependency on version 1 will always get version 1 regardless of how many later versions of ClassX are installed on the machine. The caller would only get version 2 if it specifically "upgraded" its version policy.

Support for side by side storage and execution of different versions of the same assembly is clearly an integral part of the versioning story and is built into the infrastructure of the NGWS runtime. As mentioned, the compatibility version number is part of an assembly’s identity. As a result, the storage of multiple versions of the same assembly in the global assembly cache and the loading of those assemblies at runtime just works.

Discussions of side-by-side execution often distinguish between running side-by-side on the same machine and running side-by-side in the same process.

Side-by-side on the same machine simply means that multiple versions of the same application can run on the same machine at the same time without interfering with each other. For example, if versions 4 and 5 of IE supported side-by-side on the same machine, a user could run IE 4 and IE 5 on the same machine simultaneously. An application that supports running side-by-side on the same machine must not have specific dependencies on machine-wide state. For example, if an application uses a file at a specific location for a cache it must either handle the case where multiple versions of the application are accessing the file at the same time, or must remove the dependency on the specific location and allow each version to have its own cache file.

Running side-by-side in the same process places more stringent requirements on the developer of the code. To run side-by-side in the same process multiple versions cannot have any strict dependencies on process-wide resources (i.e like using a single name for a memory mapped file, for example). The ability to run side-by-side in the same process provides a great deal of flexibility for hosting environments. As an example, consider the case where a web server is hosting applications from two different vendors that both use a common component like a data access library. If the data access library did not support side-by-side in the same process, the web server would need to ensure that applications with conflicting dependencies such as this were run in separate processes. It is also possible for a single application with a large dependency graph to use multiple versions of the same component. As a further refinement to side by side in the same process, the NGWS runtime also allows multiple versions of the same assembly to run in the same AppDomain. Whether this behavior is useful is still up for debate.


Reference

Please see the Configuration File Syntax specification for detailed info about each configuration tag and possible attributes.

http://comrtime/specs/admin/config_syntax.doc


NGWS SDK Documentation

This is preliminary documentation and subject to change.
To comment on this topic, please send us email at ngwssdk@microsoft.com. Thanks!

Technical Overview of the NGWS Runtime

© 2000 Microsoft Corporation. All rights reserved.


Overview of the NGWS Runtime

This document serves as a high-level technical introduction to the architecture of the NGWS runtime. Where appropriate, this document provides pointers to more detailed information contained in other documents. At the center of the runtime is a single type system, the Virtual Object System (VOS), which is shared by compilers, tools, and the runtime itself. It is the model that defines the rules the runtime follows when declaring, using, and managing types. The VOS establishes a framework that enables cross-language integration, type safety, and high performance code execution. This document describes the architecture of the NGWS runtime by describing the VOS and its implementation by the runtime.

The following four areas are covered in this document:

  • The VOS Type System. The Virtual Object System (VOS) provides a rich type system that supports the types and operations found in many programming languages. The VOS Type System is intended to support the complete implementation of a wide range of programming languages.
  • Metadata. The NGWS runtime uses metadata to describe and reference the types defined by the VOS Type System. Metadata can be stored (“persisted”) in a way that is independent of any particular programming language. Thus, metadata provides a common interchange mechanism for use between tools that manipulate programs (compilers, debuggers, etc.) as well as between these tools and the Virtual Execution System.
  • The Common Language Specification. While not strictly part of the VOS, the Common Language Specification is an agreement between language designers and framework (class library) designers. It specifies a subset of the VOS Type System and a set of usage conventions. Languages provide their users the greatest ability to access frameworks by implementing at least those parts of the VOS Type System that are part of the CLS. Similarly, frameworks can be most widely used if their publicly exposed aspects (classes, interfaces, methods, fields, etc.) use only types that are part of the CLS and adhere to the CLS conventions.
  • The Virtual Execution System. The Virtual Execution System (VES) implements and enforces the VOS model. The VES is responsible for loading and running programs written for the NGWS runtime. It provides the services needed to execute managed code and data, using the metadata to connect separately generated modules together at runtime (late binding).

Together, these aspects of the NGWS runtime form a unifying framework for designing, developing, deploying, and executing distributed components and applications. The appropriate subset of the VOS Type System is available from each programming language that targets the NGWS runtime. Language-based tools communicate with each other and with the Virtual Execution System using metadata to define and reference the types used to construct the application. The Virtual Execution System uses the metadata to create instances of the types as needed and to provide data type information to other parts of the infrastructure (such as remoting services, assembly downloading, security, etc.).


Problems Addressed

The Virtual Object System addresses a number of issues that have complicated the creation and deployment of distributed applications:

  • Similar and but subtly incompatible types - Dates, Times, Integers, SQL nullable type, etc.
  • Limited code reuse - Cannot import a type from a different language and treat it the same as types defined directly in the language.
  • Non-uniform object models – Differing ways of dealing with events, dynamic behaviors, persistence, properties, exceptions, etc.

The VOS abstracts and simplifies the details of the language/tool that must be known before a service can be built. By providing a common framework, it allows the runtime and associated services to automate much of the work that is performed manually today. The common framework repairs the following weaknesses in today’s infrastructure:

  • No common execution model - No uniform way to inspect the state of an executing program. Crucial for code access security, enables declarative runtime services, and simplifies tools like profilers.
  • Brittle binding mechanisms – All the things that lead to “DLL hell” and the general versioning problems that arise from software evolution.

In short, too little is known about a program after it is compiled. NGWS addresses this problem by providing:

  • a common type system (the VOS Type System),
  • a means of persisting information about types along with the components that use them (metadata),
  • a specification of the subset of types that have broad reach across programming languages, and
  • a means of building instances given type descriptions (the Virtual Execution System).


Relationship to Type Safety

Type safety is usually discussed in terms of what it does, e.g. guaranteeing encapsulation between different objects, or in terms of what it prevents, e.g. memory corruption by writing where one shouldn’t. However, from the point of view of the Virtual Object System, type safety is about guaranteeing:

  • References are what they say they are - Every reference is typed and the thing referenced, the definition, also has a type, and they are compatible in a strict sense.
  • Identities are who they say they are - There is no way to corrupt or spoof an object, and by implication a user or security domain. The access to an object is through accessible functions and fields. An object can still be poorly designed. The key is that a local analysis of the object and the things it uses, as opposed to a global analysis of all uses of an object, is sufficient to understand the vulnerabilities
  • Only appropriate operations can be invoked – The reference type defines the accessible functions and fields. This includes limiting visibility based on where the reference is, e.g. protected fields only visible in subclasses

The Virtual Object System promotes type safety e.g. everything is typed. Type safety can be optionally enforced. The hard problem is determining if an implementation conforms to a typesafe declaration. Since the declarations are carried along as metadata with the compiled form of the program, a compiler from IL to native code (see Type Safety and Verification) can type-check the implementations. When coupled with code signing, the issue is then when to type-check and when to trust. For more information, see the security specifications.


Relationship to Managed Metadata-driven Execution

Metadata describes code by describing the types that the code defines and the types that it references externally. The compiler produces the metadata when the code is produced. Enough information is stored in the metadata to:

  • Manage code execution – not just load and execute, but also memory management and execution state inspection.
  • Administer the code – Installation, resolution, and other services
  • Reference types in the code – Importing into other languages and tools as well as scripting and automation support.

The Virtual Object System assumes that the execution environment is metadata-driven. Using metadata allows the NGWS runtime to support:

  • Multiple execution models - The metadata also allows the execution environment to deal with a mixture of interpreted, JITted, native and legacy code and still present uniform services to things like debuggers or profilers, consistent exception handling and unwinding, reliable code access security, and efficient memory management.
  • Auto support for services - Since the metadata is available at execution time, the execution environment and the base runtime libraries can automatically supply support for reflection, automation, inter-op with existing Unmanaged COM applications, and inter-op with existing unmanaged native code with little or no effort on the part of the programmer.
  • Better optimization – Using metadata references instead of physical offsets, layouts, and sizes allows the execution engine to optimize the physical layouts of members and dispatch tables. In addition, this allows the generated code to be optimized to match the particular CPU or environment.
  • Reduced binding brittleness – Using metadata references also reduces the constraints on what constitutes binary compatibility between implementations, thereby reducing version to version brittleness and/or build order/phase breakage.
  • Flexible deployment resolution - Since we can have metadata for both the reference and the definition of a type, more robust and flexible deployment and resolution mechanisms are possible. Resolution boils down to answering the question: by looking in the appropriate set of places, find the implementation that best satisfies these requirements for use in this context. There are five pieces of information in the foregoing. Two items are made available via metadata (requirements and context). The others come from the application packaging and deployment story (where to look, how to find an implementation, and how to decide the best match).


Managed Code

Managed code is simply code that provides enough information to allow the NGWS runtime to provide a set of core services, including

  • Given an address inside the code for a method, locate the metadata describing the method
  • Walk the stack
  • Handle exceptions
  • Store and retrieve security information


Managed Data

Managed data is data that is allocated and released automatically by the NGWS runtime, through a process called garbage collection. Only managed code can access managed data, but programs that are written in managed code can access both managed and unmanaged data.


Relationship to Unmanaged COM

The VOS type system is centered on structured data, classes, and interfaces, which provide methods, properties and events, much like Unmanaged COM. The VOS type system adds the further notions of single inheritance, fields, exceptions, constructors, static fields and static methods.

The types managed by the NGWS runtime for all languages are represented as types within the Virtual Object System. The components of a type (instance and static fields; virtual, instance, and static methods; events; and properties) are defined logically in terms of other VOS types.

All types are represented in metadata and are stored with the code that implements the type. Once a language reads the metadata that defines a VOS type, it can create instances, create referencing variables, and invoke methods - just as if the type had been defined in the importing language. If the language supports defining new types via inheritance, the language can make a subclass of the imported class. As such, the VOS makes language integration, a significant step beyond traditional interoperation, possible.


Introduction to the Common Language Specification (CLS)

The Common Language Specification (CLS) is discussed in greater detail below (see Common Language Specification). It is a set of conventions intended to promote language interoperability. Throughout this document, and collected together in a single section (see Collected CLS Rules), there are specific rules that must be followed in order to conform to the CLS. These rules apply only to items that are exposed for use by other programming languages. In particular, they apply to types that are visible in assemblies other those in which they are defined, and to the members (fields, methods, properties, events, and nested types) that are accessible outside the assembly (i.e. those that have an accessibility of public, family, or family or assembly).

The rules are described in a common format where they are first introduced. For example, the first rule is introduced as follows:

CLS Rule 1: Boxed value types are not part of the CLS. Instead, use System.Object, System.ValueType or System.Enum, as appropriate.

CLS (consumers): need not import boxed value types.

CLS (extenders): need not provide syntax for defining or using boxed value types.

CLS (frameworks): must not use boxed value types in their publicly exposed aspects.

The first paragraph specifies the rule itself. This is then followed by a brief description of how the rule applies in three distinct cases:

  • CLS (consumers): describes how the rule applies to CLS consumer languages. These are languages that are designed to allow programmers to access all of the features supplied by CLS-compliant frameworks (libraries). Programmers in these languages may not be able to extend these frameworks by creating new types, but they can make use of any predefined types.
  • CLS (extenders): describes how the rule applies to CLS extender languages. These are languages that are designed to allow programmers to both use and extend CLS-compliant frameworks. Programmers can use existing types and define new types.
  • CLS (frameworks): describes how the rule applies to the design of CLS-compliant frameworks. These frameworks (libraries) are designed for use by a wide range of programming languages, including both CLS consumer and extender languages.


Summary

Unmanaged COM is about interoperation among languages. The Virtual Object System is about integration between languages. The former is about invoking another language; the latter is about using another language’s objects as if they were one’s own.

NGWS is all about making it easier to write components and applications from any language. It does this by defining a standard set of types, making all components fully self-describing, and providing a high performance common execution environment. This ensures that all NGWS runtime compliant system services and components will be accessible to all NGWS runtime aware languages and tools. In addition, this simplifies deployment of components and applications that use them; all in a way that allows compilers and other tools to leverage the high performance execution environment. The NGWS runtime Virtual Object System covers, at a high level, the concepts and interactions that make all of this possible.

The discussion is broken down into three areas:

  • Type System – What types are and how to define them.
  • Metadata - How types are externalized and persisted.
  • Virtual Execution System - How code is executed and types are instantiated, interact, and die.


Type System

Types describe values and specify a contract (see Contracts) that all values of that type must support. Because the VOS supports both Object-Oriented (OOP) as well as functional and procedural programming languages, it deals with two kinds of entities: Objects and Values. Values are simple bit patterns for things like integers and floats; each value has a type that describes both the storage that it occupies and the meanings of the bits in its representation, and also the operations that can be performed on that representation. Values are intended for representing the corresponding simple types in programming languages like C, and also for representing non-objects in languages like C++ and Java.

Objects have rather more to them than do values. Each object is self-typing, that is, its type is explicitly stored in its representation. It has an identity that distinguishes it from all other objects, and it has slots that store other entities (which may be either objects or values). While the contents of its slots may be changed, the identity of an object never changes.

There are several kinds of Objects and Values, as shown in the following diagram.


Relationship to Object-Oriented Programming

The term type is often used in the world of value-oriented programming to mean data representation. In the object-oriented world it usually refers to behavior rather than to representation. In the VOS, type is used to mean both of these things: two entities have the same type if and only if they have both compatible representations and behaviors. Thus, in the VOS, if one type is derived from a base type, then instances of the derived type may be substituted for instances of the base type because both the representation and the behavior are compatible.

In the VOS, unlike some OOP languages, two objects that have fundamentally different representations have different types. Some OOP languages use a different notion of type. They consider two objects to have the same type if they respond in the same way to the same set of messages. This notion is captured in the VOS by saying that the objects implement the same interface.

Similarly, some OOP languages consider message passing to be the fundamental model of computation. In the VOS, this corresponds closely (but not exactly) to calling virtual methods (see Virtual Methods).

The VOS itself does not directly capture the notion of “typeless programming.” That is, there is no way to call a method (other than a static method) without knowing the type of the object. Nevertheless, typeless programming can be implemented in the VES by writing a custom message dispatch mechanism and using it directly, generally based on the facilities provided by the types in System.Reflection. When interoperating with other languages this dispatch mechanism needs to be packaged into exposed methods so that other languages can access the underlying mechanism. Similarly, the message dispatch mechanism must understand and interoperate with the VOS method mechanism to allow objects implemented in other languages to be accessed.


Values and Types

Types describe values. All places where values are stored, passed, or operated upon have a type, e.g. all variables, parameters, evaluation stack locations, and method results. The type defines the allowable values and the allowable operations supported by the values of the type. All operators and functions have expected types for each of the values accessed or used. Let’s use the notation v:T to indicate a value v is of type T.

A value can be of more than one type. A value that supports many interfaces is an example of a value that is of more than one type, as is a value that inherits from another.


Value Types and Reference Types

There are two kinds of types: Value Types and Reference Types.

  • Value Types - Value Types describe values that are represented as sequences of bits.
  • Reference Types – Reference Types describe values that are represented as the location of a sequence of bits. There are three kinds of Reference Types:
    • An object type is a reference type of a self-describing value (see Classes, Interfaces and Objects). Some object types (e.g. abstract classes) are only a partial description of a value.
    • An interface type is always a partial description of a value, potentially supported by many object types.
    • A pointer type is a compile time description of a value whose representation is a machine address of a location.


Built-In Types

The following data types are an integral part of the VOS and are supported directly by the Virtual Execution System (VES). They have special encoding in the persisted metadata:

Name in IL assembler (VOS) | CLS Type? | Name in class library | Description
bool | Yes | System.Boolean | True/false value
char | Yes | System.Char | Unicode 16-bit char.
class System.Object | Yes | System.Object | Object or boxed value type
class System.String | Yes | System.String | Unicode string
float32 | Yes | System.Single | IEEE 32-bit float
float64 | Yes | System.Double | IEEE 64-bit float
int8 | No | System.SByte | Signed 8-bit integer
int16 | Yes | System.Int16 | Signed 16-bit integer
int32 | Yes | System.Int32 | Signed 32-bit integer
int64 | Yes | System.Int64 | Signed 64-bit integer
native int | No |  | Signed integer, native size
native unsigned int | No |  | Unsigned integer, native size
typedref | No | System.TypedReference | Pointer plus runtime type
unsigned int8 | Yes | System.Byte | Unsigned 8-bit integer
unsigned int16 | No | System.UInt16 | Unsigned 16-bit integer
unsigned int32 | No | System.UInt32 | Unsigned 32-bit integer
unsigned int64 | No | System.UInt64 | Unsigned 64-bit integer


Classes, Interfaces, and Objects

Every value has an exact type that fully describes the value. A type fully describes a value if it completely defines the value’s representation and the operations defined on the value.

For a Value Type, defining the representation entails describing the sequence of bits that make up the value’s representation. For a Reference Type, defining the representation entails describing the location and the sequence of bits that make up the value’s representation.

A named method describes an operation that can be performed on values of an exact type. Defining the set of operations allowed on values of an exact type entails specifying named methods for each operation.

Some types are only a partial description, e.g. interface types. Interface types describe a subset of the operations and none of the representation, and hence, cannot be an exact type of any value. Hence, while a value has only one exact type, it may also be a value of many other types as well. Furthermore, since the exact type fully describes the value, it also fully specifies all of the other types that a value of the exact type can have.

While it is true that every value has an exact type, it is not always possible to determine the exact type by inspecting the representation of the value. In particular, it is never possible to determine the exact type of a value of a Value Type. Consider two of the built-in Value Types, 32-bit signed and unsigned integers. While each type is a full specification of their respective values, i.e. an exact type, there is no way to derive that exact type from a value’s particular 32-bit sequence.

For some values, called objects, it is always possible to determine the exact type from the value. Exact types of objects are also called object types. Objects are values of Reference Types, but not all Reference Types describe objects. Consider a value that is a pointer to a 32-bit integer, a kind of Reference Type. There is no way to discover the type of the value by examining the pointer bits, hence it is not an object. Now consider the built-in VOS Reference Type System.String. The exact type of a value of this type is always determinable by examining the value, hence values of type System.String are objects and System.String is an object type.


Boxing and Unboxing of Values

For every Value Type, the VOS defines a corresponding Reference Type called the boxed type. The reverse is not true: Reference Types do not in general have a corresponding Value Type. The representation of a value of a boxed type (a boxed value) is a location where a value of the Value Type can be stored. A boxed type is an object type and a boxed value is an object.

All Value Types have an operation called box. Boxing a value of any Value Type produces its boxed value, i.e. a value of the corresponding boxed type containing a bit copy of the original value. All boxed types have an operation called unbox. Unboxing is the opposite operation to boxing: it copies the bit sequence from the boxed value to create a value of the underlying Value Type.

Notice that interfaces and inheritance are defined only on Reference types. Thus, while a Value Type definition (see Value Type Definition) can specify both interfaces that must be implemented by the Value Type and a class from which it inherits, these apply only to boxed values.

Note: In PDC, the implementation of boxed value types is incomplete. While any value type can be boxed, the result should be considered to have type System.Object, System.ValueType, or System.Enum rather than a specific type that refers only to the particular boxed type.

CLS Rule 1: Boxed value types are not part of the CLS. Instead, use System.Object, System.ValueType or System.Enum, as appropriate.

CLS (consumers): need not import boxed value types.

CLS (extenders): need not provide syntax for defining or using boxed value types.

CLS (frameworks): must not use boxed value types in their publicly exposed aspects.


Identity and Equality of Values

There are two binary operators defined on all pairs of values, identity and equality, which return a Boolean result. Both of these operators are mathematical equivalence operators, i.e. they are:

  • Reflexive - a op a is true.
  • Symmetric - a op b is true if and only if b op a is true.
  • Transitive - if a op b is true and b op c is true, then a op c is true

In addition, identity always implies equality, but not the reverse, i.e., the equality operator need not be the same as the identity operator as long as two identical values are also equal values.

To understand the difference between these operations, consider three variables whose type is System.String, where the arrow is intended to mean “is a reference to”:

The values of the variables are identical if the locations of the sequences of characters are the same, i.e., there is in fact only one string in memory. The values stored in the variables are equal if the sequences of characters are the same. Thus, the values of variables A and B are identical, the values of variables A and C as well as B and C are not identical, and the values of all three of A, B, and C are equal.

Identity

The identity operator is defined by the VOS as follows.

  • If the values have different exact types, then they are not identical.
  • If their exact type is a Value Type, then they are identical if and only if the bit sequences of the values are the same, bit by bit.
  • If their exact type is a Reference Type, then they are identical if and only if the locations of the values are the same.

Equality

The exact types of the values supply the equality operator. Definitions of equality must obey the following two rules:

  • Equality is an equivalence operator, as defined above.
  • Identity must imply equality, as stated earlier.
  • If either (or both) operand is a boxed value, equality is computed by
    • first unboxing any boxed operand(s), and then
    • applying the usual rules for equality on the resulting values.


Locations

Values are stored in locations. A location can hold a single value at a time. All locations are typed. The type of the location embodies the requirements that must be met by values that are stored in the location. Examples of locations are local variables and parameters.

More importantly, the type of the location specifies the restrictions on usage of any value that is loaded from the location. For example, a location can hold values of potentially many exact types as long as all of the values are assignment compatible with the type of the location (see below). All values loaded from a location are treated as if they are of the type of the location. Only operations valid for the type of the location can be invoked even if the exact type of the value stored in the location is capable of additional operations.


Assignment Compatible Locations

A value can be stored in a location only if one of the types of the value is assignment compatible with the type of the location. A type is always assignment compatible with itself. Assignment compatibility can often be determined at compile time, in which case there is no need for testing at run time. Assignment compatibility is described in detail in Assignment Compatibility.


Coercion

Sometimes it is desirable to take a value of a type that is not assignment compatible with a location and convert the value to a type that is assignment compatible. This is accomplished through coercion of the value. Coercion takes a value of a particular type and a desired type and attempts to create a value of the desired type that has equivalent meaning to the original value. Coercion can result in representation changes as well as type changes, hence coercion does not necessarily preserve the identity of two objects.

There are two kinds of coercion: widening, which never loses information, and narrowing, in which information may be lost. An example of a widening coercion would be coercing a value that is a 32-bit signed integer to a value that is a 64-bit signed integer. An example of a narrowing coercion is the reverse: coercing a 64-bit signed integer to a 32-bit signed integer. Programming languages often implement widening coercions as implicit conversions and narrowing coercions as explicit conversions.

Some widening coercion is built directly into the VES operations on the built-in types (see the EE Architecture Specification. All other coercion must be explicitly requested. For the built-in types, the VOS provides operations to perform widening coercions with no runtime checks and narrowing coercions with runtime checks.


Casting

Since a value can be of more than one type, a use of the value needs to clearly identify which of its types is being used. Since values are read from locations, which are typed, the type of the value that is used is the type of the location from which the value was read. If a different type is to be used, the value is cast to one of its other types. Casting is usually a compile time operation, but if the compiler cannot statically know that the value is of the target type, a runtime cast check is done. Unlike coercion, a cast never changes the actual type of an object nor does it change the representation. Casting preserves the identity of objects.

For example, a runtime check may be needed when casting a value read from a location that is typed as holding values of a particular interface. Since an interface is an incomplete description of the value, casting that value to be of a different interface type will usually result in a runtime cast check.


Type Members

As stated above, the type defines the allowable values and the allowable operations supported by the values of the type. If the allowable values of the type have a substructure, that substructure is described via fields or array elements of the type. If there are operations that are part of the type, those operations are described via methods on the type. Fields, array elements, and methods are called members of the type. Properties and events are also members of the type, but they are described later.


Fields, Array Elements, and Values

The representation of a value can be subdivided into sub-values. These sub-values are either named, in which case they are called fields, or they are accessed by an indexing expression, in which case they are called array elements. Types that describe values composed of array elements are array types. Types that describe values composed of fields are compound types. A value contains either fields or array elements, although a field of a compound type can be an array type and an array element can be a compound type.

Array elements and fields are typed. All of the elements of an array type must have the same type. Each field of a compound type may have a different type. The element type of an array type and the field types of a compound type never change.


Methods

A type can associate operations with the type or with each value of the type. Such operations are called methods. A method is named, and has a signature (see Signatures) that specifies the allowable types for all of its arguments and for its return value, if any.

A method that is associated with the type itself is called a static method.

A method that is associated with a value of the type is either an instance method or a virtual method. When they are invoked, instance and virtual methods are passed the value on which this invocation is to operate (known as this or a this pointer).

The fundamental difference between an instance method and a virtual method is in how the implementation is located. An instance method is invoked by specifying a class and the instance method within that class. The object passed as this may be null (a special value indicating that no particular value is being specified) or an instance of any type that inherits (see Type Inheritance) from the class which defines the method. A virtual method can also be called in this manner. This occurs, for example, when an implementation of a virtual method wishes to call the implementation supplied by its parent class. The VOS allows this to be null inside the body of a virtual method.

A virtual method (but not an instance method) may also be called by a different mechanism, a virtual call. Any type that inherits from a type that defines a virtual method can provide its own implementation of that method (this is known as overriding, see Hiding, Overriding, and Layout). It is the exact type of the object (determined at runtime) that is used to decide which of the implementations to invoke


Static Fields and Static Methods

Types may declare locations that are associated with the type rather than any particular value of the type. Such locations are static fields of the type. As such, static fields declare a location that is shared by all values of the type. Just like non-static (instance) fields, a static field is typed and that type never changes.

Similarly, types can also declare methods that are associated with the type rather than with values of the type. Such methods are static methods of the type. Since an invocation of a static method does not have an associated value on which the static method operates, there is no this pointer available within a static method.


Virtual Methods

An object type can declare any of its methods as virtual. Unlike other methods, each exact type that implements the type may provide its own implementation of a virtual method. A virtual method may be invoked through the ordinary method call mechanism that uses the static type, method name, and types of parameters to choose an implementation, in which case the this pointer may be null. In addition, however, a virtual method can be invoked by a special mechanism (a virtual call) that chooses the implementation based on the dynamically detected type of the instance used to make the virtual call rather than the type statically known at compile time. Virtual methods may be marked final (see Method Inheritance).


Naming

Names are given to entities of the type system so that they can be referred to by other parts of the type system or by the implementations of the types. Types, fields, methods, properties and events have names. With respect to the type system values, locals, and parameters do not have names. An entity of the type system is given a single name, e.g. there is only one name for a type.


Valid Names

The VOS itself imposes two restrictions on names:

  1. All names are encoded as 16-bit Unicode strings.
  2. Names are not permitted to have an embedded (16-bit) value of 0x0000.

All comparisons are done on a byte-by-byte (i.e. case sensitive, locale-independent, also known as code-point comparison) basis. Where names are used to access built-in VES-supplied functionality (for example, the class initialization method) there is always an accompanying indication so as not to build in any set of reserved names.

CLS Rule 2: Languages must follow Annex 7 of Technical Report 15 of the Unicode Standard 3.0 (ISBN 0-201-61633-5) governing the canonicalization of identifiers, available on-line at http://www.unicode.org/unicode/reports/tr15/tr15-18.html. While the CLS allows the use of case-insensitive comparison (canonicalization KC), identifiers must be persisted in their original case (canonicalization C) so that case sensitive comparisons work as expected.

CLS Rule 3: Languages must provide an escape mechanism to reference identifiers that happen to be keywords in their own language. In addition, extender languages must provide a mechanism for defining and overriding virtual methods with names that are keywords in their language.

CLS (consumer): need not consume types that violate these rules, but must have an escape mechanism to allow access to named data that uses one of its own keywords as the name.

CLS (extender): need not create types that violate these rules. Must provide a mechanism for defining names that obey these rules but are the same as a keyword in the language. In order to override an inherited definition the VOS requires the precise encoding of the original declaration be used.

CLS (framework): must not export types that violate these rules. Should avoid the use of names that are commonly used as keywords in programming languages.


Assemblies and Scoping

Generally, names are not unique. Names are collected into groupings called scopes. Within a scope, a name may refer to multiple entities as long as they are of different kinds or have different signatures. In the VOS, the following kinds exist: methods, fields, nested types, properties, and events.

CLS Rule 4: All names introduced in a scope shall be distinct independent of kind. That is, while the VOS allows a single type to use the same name for a method and a field, the CLS does not.

CLS Rule 5: Fields and nested types must be distinct by identifier comparison alone, even though the VOS allows distinct signatures to be distinguished. Methods, properties, and events that have the same name (by identifier comparison) must differ by more than just the return type (although there is a special exception discussed in Conversion Operators).

CLS (consumers): need not consume types that violate these rules after ignoring any members that are marked as not CLS-compliant.

CLS (extenders): need not provide syntax for defining types that violate these rules.

CLS (frameworks): must not mark types as CLS-compliant if they violate these rules unless they mark sufficient offending items within the type as not CLS-compliant so that the remaining members do not conflict with one another.

A named entity has its name in exactly one scope. Hence, to identify a named entity, both a scope and a name need to be supplied. The scope is said to qualify the name. Types provide a scope for the names in the type; hence types qualify the names in the type. For example, consider a compound type Point which has a field named x. The name “field x” by itself does not uniquely identify the named field, but the qualified name “field x in type Point” does.

Since types are named, the names of types are also grouped into scopes. To fully identify a type, the type name must be qualified by the scope that includes the type name. Type names are scoped by the assembly that contains the implementation of the type. An assembly is a configured set of loadable code modules and other resources that together implement a unit of functionality. The type name is said to be in the assembly scope of the assembly that implements the type. Assemblies themselves have names that form the basis of the VOS naming hierarchy; see the Assemblies specification.

The type definition:

  • Defines a name for the type being defined, i.e. the type name, and specifies a scope in which that name will be found
  • Defines a member scope in which the names of the different kinds of members (fields, methods, events, and properties) are bound. The tuple of (member name, member kind, and member signature) is unique within a member scope of a type.
  • Implicitly assigns the type to the assembly scope of the assembly that contains the type definition.

The VOS supports two kinds of type alias:

  • A type distinct alias, also called an enum or enumeration type, provides an alternate name for an existing type. For purposes of matching signatures a type distinct alias is not the same as the underlying type. Instances of a type distinct alias, however, are assignment compatible with the underlying type and vice versa. That is: no cast or coercion is required to convert from the alias to the underlying type, nor are they required from the underlying type to the alias. A type distinct alias is considerably more restricted than a true type:
    • It must have exactly one instance field, and the type of that field defines the underlying type of the enumeration.
    • It may not have any methods of its own.
    • It may not specify a base type nor any inherit from any interfaces.
    • It may not have any properties or events.
    • It may not have any static fields unless they are literal.

Note: In V1, the underlying type must be a built-in integer type. They must be derived from System.Enum , hence they are value types. Like all value types, they must be sealed.

CLS Rule 6: The underlying type of an enum must be a built-in CLS integer type.

CLS Rule 7: There are two distinct kinds of enums, indicated by the presence or absence of the System.FlagsAttribute custom attribute. One represents named integer values, the other named bit flags that can be combined together to generate an unnamed value. While languages are encouraged to distinguish these, they need not do so.

CLS Rule 8: Literal static fields of an enum must have the type of the enum itself.

CLS (consumer): Must accept definition of enums that follow these rules, but need not distinguish flags from named values.

CLS (extender): Must provide syntax for defining enums that follow these rules, including allowing users to indicate for each enum type whether it is a flag or a named value, even though the language need not enforce the distinction.

CLS (frameworks): must not expose enums that violate these rules, and must not assume that enums have only the specified values (even for enums that are named values).

  • A type indistinct alias also called a typedef,, creates an additional name for an existing type. The two may be used completely interchangeably. A type name is particularly useful for supplying a name for an implicit type. While a type name may be defined in terms of another type name, it always names an actual type.

Note: type indistinct aliases are not supported in V1.


Visibility, Accessibility, and Security

To refer to a named entity in a scope, both the scope and the name in the scope must be visible. Visibility is determined by the entity that contains the reference (the referent) and the entity that contains the name being referenced. Consider the following pseudo-code:

class A
{ int32 IntInsideA;
}
class B inherits from A
{ method X(int32, int32) returning Boolean
  { IntInsideA := 15;
  }
}

If we consider the reference to the field IntInsideA in class A:

  • We call class B the referent because it has a method that refers to that field,
  • We call IntInsideA in class A the referenced entity.

There are two fundamental questions that need to be answered in order to decide whether the referent is allowed to access the referenced entity. The first is whether the name of the referenced entity is visible to the referent. If it is visible, then there is a separate question of whether the referent is accessible.

Access to a member of a type is permitted only if all three of the following conditions are met:

  1. The type is visible.
  2. The member is accessible.
  3. All relevant security demands have been granted.

Visibility of Types

Only type names, not member names, have controlled visibility. Type names fall into one of the following three categories

  • Exportable from the assembly in which they are defined. While a type can be marked to allow it to be exported from the assembly, it is the configuration of the assembly that decides whether the type name is made available. See the Assembly specification for more details.
  • Not exported outside the assembly in which they are defined.
  • Nested within another type. In this case, the type itself has the visibility of the type inside of which it is nested (its enclosing class). See Nested Types.

Note: In V1, all exportable types are in fact exported from the assembly in which they are defined.

Accessibility of Members

A type scopes all of its members, and it also specifies the accessibility rules for its members. Except where noted, accessibility is decided based only on the statically visible type of the member being referenced and the type and assembly that is making the reference. The VOS supports six different rules for accessibility:

  • Private – accessible only to referents in the implementation of the exact type that defines the member.
  • Family – accessible to referents that support the same type, i.e. an exact type and all of the types that inherit from it. For verifiable code (see Type Safety and Verification), there is an additional requirement that may require a runtime check: the reference must be made through an item whose exact type supports the exact type of the referent. That is, the item whose member is being accessed must inherit from the type performing the access.
  • Assembly – accessible only to referents in the same assembly that contains the implementation of the type.
  • Family and Assembly – accessible only to referents that qualify for both Family and Assembly access.
  • Family or Assembly – accessible only to referents that qualify for either Family or Assembly access.
  • Public – visible to all referents.

In general, a member of a type can have any one of these accessibility rules assigned to it. There are two exceptions, however:

  1. Members defined by an interface must be public.
  2. When a type defines a virtual method that overrides an inherited definition, the accessibility must either identical in the two definitions or the overriding definition must permit more access than the original definition. For example, it is possible to override an assembly virtual method with a new implementation that is public virtual, but not with one that is family virtual. In the case of overriding a definition derived from another assembly, it is not considered restricting access if the base definition has family or assembly access and the override has only family access.
Rationale: Languages including C++ allow this “widening” of access. Restricting access would provide an incorrect illusion of security since simply casting an object to the base class (which occurs implicitly on method call) would allow the method to be called despite the restricted accessibility.

CLS Rule 9: Accessibility must not be changed when overriding inherited methods, except when overriding a method inherited from a different assembly with accessibility family or assembly. In this case the override must have accessibility family.

CLS (consumer): need not accept types that widen access to inherited virtual methods.

CLS (extender): need not provide syntax to widen access to inherited virtual methods.

CLS (frameworks): must not rely on the ability to widen access to a virtual method, either in the exposed portion of the framework or by users of the framework.

Security Permissions

Access to members is also controlled by security demands that can be attached to an assembly, type, method, property, or event. Security demands are not part of a type contract (see Contracts), and hence are not inherited. There are two kinds of demands:

  • An inheritance demand. When attached to a type it requires that any type that wishes to inherit from this type must have the specified security permission. When attached to a non-final virtual method it requires that any type that wishes to override this method must have the specified permission. It may not be attached to any other member.
  • A reference demand. Any attempt to resolve a reference to the marked item must have specified security permission.

Only one demand of each kind can be attached to any item. Attaching a security demand to an assembly implies that it is attached to all types in the assembly unless another demand of the same kind is attached to the type. Similarly, a demand attached to a type implies the same demand for all members of the type unless another demand of the same kind is attached to the member.

Nested Types

A type (called a nested type) can be a member of an enclosing type. A nested type has the same visibility as the enclosing type and has an accessibility as would any other member of the enclosing type. This accessibility determines which other types can make references to the nested type. That is, for a class to define a field or array element of a nested type, have a method that takes a nested type as a parameter or returns one as value, etc., the nested type must be both visible and accessible to the referencing type. A nested type is part of the enclosing type so its methods have access to all members of its enclosing type, as well as family access to members of the type from which it inherits (see Type Inheritance). The names of nested types are scoped by their enclosing type, not their assembly (only top-level types are scoped by their assembly). There is no requirement that the names of nested types be unique within an assembly.


Contracts

Contracts are named. They are the shared assumptions on a set of signatures (see the Signatures section) between all implementers and all users of the contract. The signatures are the part of the contract that can be checked and enforced.

Contracts are not types; rather they specify requirements on the implementation of types. Types state which contracts they abide by, i.e. which contracts all implementations of the type must support. An implementation of a type can be verified to check that the enforceable parts of a contract, the named signatures, have been implemented. The kinds of contracts are:

  • Class contract – A class contract is specified with a class definition. Hence, a class definition defines both the class contract and the class type. The name of the class contract and the name of the class type are the same. A class contract specifies the representation of the values of the class type. Additionally, a class contract specifies the other contracts that the class type supports, e.g., which interfaces, methods, properties and events must be implemented. A class contract, and hence the class type, can be supported by other class types as well. A class type that supports the class contract of another class type is said to inherit from that class type.
  • Interface contract – An interface contract is specified with an interface definition. Hence, an interface definition defines both the interface contract and the interface type. The name of the interface contract and the name of the interface type are the same. Many types can support an interface contract. Like a class contract, interface contracts specify which other contracts the interface supports, e.g. which interfaces, methods, properties and events must be implemented. Note that an interface type can never fully describe the representation of a value. Therefore an interface type can never support a class contract, and hence can never be a class type or an exact type.
  • Method contract – A method contract is specified with a method definition. A method contract is a named operation that specifies the contract between the implementation(s) of the method and the callers of the method. A method contract is always part of a type contract (class, value type, or interface), and describes how a particular named operation is implemented. The method contract specifies the contracts that each parameter to the method must support and the contracts that the return value must support, if there is a return value.
  • Property contract – A property contract is specified with a property definition. There is an extensible set of operations for handling a named value, which includes a standard pair for reading the value and changing the value. A property contract specifies method contracts for the subset of these operations that must be implemented by any type that supports the property contract. A type can support many property contracts, but any given property contract can be supported by exactly one type. Hence, property definitions are a part of the type definition of the type that supports the property.
  • Event contract – An event contract is specified with an event definition. There is an extensible set of operations for managing a named event, which includes three standard methods (register interest in an event, revoke interest in an event, fire the event). An event contract specifies method contracts for all of the operations that must be implemented by any type that supports the event contract. A type can support many event contracts, but any given event contract can be supported by exactly one type. Hence, event definitions are a part of the type definition of the type that supports the event.


Signatures

Signatures are the part of a contract that can be checked and automatically enforced. Signatures are formed by adding constraints to types and other signatures. A constraint is a limitation on the use of or allowed operations on a value or location. Example constraints would be whether a location can be overwritten with a different value or whether a value can ever be changed.

All locations have signatures, as do all values. Assignment compatibility requires that the signature of the value, including constraints, is compatible with the signature of the location, including constraints. There are four fundamental kinds of signatures: type signatures, location signatures, parameter signatures, and method signatures.

CLS Rule 10: All types appearing in a signature must be CLS-compliant.

CLS Rule 11: The visibility and accessibility of types and members must be such that types in the signature of any member must be visible and accessible whenever the member itself is visible and accessible. For example, a public method that is visible outside its assembly must not have an argument whose type is visible only within the assembly.

CLS (consumer): need not accept types whose members violate these rules.

CLS (extender): need not provide syntax to violate these rules.

CLS (framework): must not violate this rule in its exposed types and their members.

Type Signatures

Type signatures define the constraints on a value and its usage. A type, by itself, is a valid type signature. Only one constraint can be added to a type to produce a type signature:

  • The constant constraint requires that the value not be changed. Either directly or by storing the value in a location that would allow it to be changed, e.g. passing the value as a parameter to an operation that changes the value.

The type signature of a value cannot be determined by examining the value or even by knowing the class type of the value. The type signature of a value is derived from the location signature (see below) of the location from which the value is loaded. Normally the type signature of a value is the type in the location signature from which the value is loaded. If the location signature includes the constant constraint, then the type signature of the value loaded from that location also includes the constant constraint.

Note: The constant constraint cannot be expressed in V1 of the NGWS runtime.

Location Signatures

All locations are typed. This means that all locations have a location signature, which defines constraints on the location, its usage, and on the usage of the values stored in the location. Any valid type signature is a valid location signature. Hence, a location signature contains a type and may additionally contain the constant constraint. The location signature may also contain location constraints that give further restrictions on the uses of the location. The location constraints are:

  • The init-only constraint promises (hence, requires) that once the location has been initialized, its contents never change. Namely, the contents are initialized before any access, and after initialization, no value can be stored in the location. The contents are always identical to the initialized value (see Classes, Interfaces and Objects).
  • The literal constraint promises that the value of the location is actually a fixed value of a built-in type. The value is specified as part of the constraint. Compilers are required to replace all references to the location with its value, and the VES therefore need not allocate space for the location.

Note: In V1 the init-only constraint is allowed only on fields of compound types, and the literal constraints is allowed only on static fields of compound types.

  • The volatile constraint on a location requires that the value stored in the location not be cached between accesses.

Note: In V1 the volatile constraint is not supported. Instead, the IL instruction set encodes specific accesses that must be handled specially.

Init-only describes a location into which a new value cannot be stored. Volatile describes a location that is changing outside the control of this portion of the program. Constant declares that the values loaded from the location are not to be used to make modifications to the value. Any or all of these constraints can be specified as part of a location signature.

Consider a location whose type is “Array of 32-bit signed integers.” If the location signature contains the init-only constraint, then the location always contains a reference to the same array in memory, but this reference can be used to store different integers within that array. If the location signature contains the constant constraint (actually a type signature constraint, but stored in the location signature) then over time, the location may contain references to different arrays. However, at any given time the current array reference should not be used to change any of the integers stored within the array. If the location signature contains the volatile constraint, then any array reference read from this location should not be cached, i.e. every reference in the source code to the location should re-load the array reference from the location.

Local Signatures

A local signature specifies the contract on a local variable allocated during the running of a method. A local signature contains a full location signature, plus it may specify one additional constraint:

The byref constraint states that the content of the corresponding location is a managed pointer. A managed pointer may point either to a local variable, parameter, field of a compound type, or element of an array. Managed pointers must be reported to the garbage collector, since the location to which they refer may be moved.

Note: While strictly speaking the byref constraint means that a managed pointer is to be passed, when the call crosses a remoting boundary (see Proxies, Contexts, and Remoting) this may be implemented by a copy-in/copy-out mechanism. Thus programs must not rely on the aliasing behavior of true pointers.

In addition, there is one special local signature. The typed reference local signature states that the local will contain both a managed pointer to a location and the a runtime representation of the type that may be stored at that location. A typed reference signature is similar to a byref constraint, but while the byref specifies the type as part of the byref constraint (and hence as part of the type description), a typed reference provides the type information dynamically. Hence a typed reference is a full signature in itself and cannot be combined with other constraints. In particular, it is not possible to specify a byref whose type is typed reference.

Note: The typed reference signature is provided as a built-in value type in V1. In the Base Class Library, the type is known as System.TypedReference and in the assembler it is designated by the keyword typedref. This type can only be used for parameters and local variables. It is illegal to box it, to use it as the type of a field of a type, element of an array, return value, etc.

CLS Rule 12: Typed references are not CLS-compliant.

CLS (consumer): there is no need to accept this type.

CLS (extender): there is no need to provide syntax to define this type or to extend interfaces or classes that use this type.

CLS (framework): this type may not appear in exposed members.

Parameter Signatures

Parameter signatures define constraints on how an individual value is passed as part of a method invocation. Parameter signatures are declared by method definitions. Any valid local signature is a valid parameter signature.

Method Signatures

Method signatures are composed of

  • a calling convention,
  • a list of zero or more parameter signatures, one for each parameter of the method,
  • and a type signature for the result value if one is produced.

Method signatures are declared by method definitions. Only one constraint can be added to a method signature in addition to those of parameter signatures:

  • The varargs constraint can be included to indicate that all parameters past this point are optional. When it appears, the calling convention must be one that supports variable argument lists.

Method signatures are used in two different ways. They are used as part of a method definition and as a description of a calling site when calling through a function pointer. In this latter case, the method signature indicates

  • the calling convention (which may include platform-specific calling conventions)
  • the type of all the argument values that are being passed,
  • if needed, a varargs marker indicating where the fixed parameter list ends and the variable parameter list begins

When used as part of a method definition, the varargs constraint is represented by the choice of calling convention.

CLS Rule 13: The varargs constraint is not part of the CLS, and the only calling convention supported by the CLS is the standard managed calling convention.

CLS (consumer): there is no need to accept methods with variable argument lists or unmanaged calling convention.

CLS (extender): there is no need to provide syntax to declare varargs methods or unmanaged calling conventions.

CLS (framework): neither varargs methods nor methods with unmanaged calling conventions may be exposed externally.


Assignment Compatibility

The constraints in the type signature and the location signature affect assignment compatibility of a value to a location. Assignment compatibility of a value (described by a type signature) to a location (described by a location signature) is defined as follows:

  • One of the types supported by the exact type of the value is the same as the type in the location signature. This allows, for example, an instance of a class that inherits from a base class (hence supports the base class’s type contract) to be stored into a location whose type is that of the base class.
  • A value that has the constant constraint in its type signature can only be stored in a location that also has the constant constraint in its location signature.
  • The reverse is not true. A location signature that includes the constant constraint is compatible with a type signature that does not include the constant constraint.

Note: V1 does not support the constant constraint


Type Safety and Verification

Since types specify contracts, it is important to know whether a given implementation lives up to these contracts. An implementation that lives up to the enforceable part of the contract (the named signatures) is said to be typesafe. An important part of the contract deals with restrictions on the visibility and accessibility of named items as well as the mapping of names to implementations and locations in memory.

Typesafe implementations only store values described by a type signature in a location that is assignment compatible with the location signature of the location (see Signatures). Typesafe implementations never apply an operation to a value that is not defined by the exact type of the value. Typesafe implementations only access locations that are both visible and accessible to them. In a typesafe implementation, the exact type of a value cannot change.

Verification is a mechanical process of examining an implementation and asserting that it is typesafe. Verification is said to succeed if the process can prove that an implementation is typesafe. Verification is said to fail if that process cannot prove the type safety of an implementation. Verification is necessarily conservative: it may report failure for a typesafe implementation, but it never reports success for an implementation that is not typesafe. For example, most verification processes report implementations that do pointer-based arithmetic as failing verification, even if the implementation is in fact typesafe.

There are many different processes that can be the basis of verification. The simplest possible process simply says that all implementations are not typesafe. While correct and efficient this is clearly not particularly useful. By spending more resources (time and space) a process can correctly identify more typesafe implementations. It can be proven, however, that no mechanical process can in finite time and with no errors correctly identify all implementations as either typesafe or not typesafe. The choice of a particular verification process is thus a matter of engineering, based on the resources available to make the decision and the importance of detecting the typesafety of different programming constructs.


Type Definers

Type definers construct a new type from existing types. Implicit types (e.g., built-in types, arrays, pointers including function pointers, and delegates) are defined when they are used. The mention of an implicit type in a signature is in and of itself a complete definition of the type. Implicit types allow the VES to manufacture instances with a standard set of members, interfaces, etc. Implicit types need not have user-supplied names.

Note: In V1, function pointer types are not fully implemented and delegates are not implicit types.

All other types must be explicitly defined using an explicit type definition. The explicit type definers are:

  • interface definitions – used to define interface types
  • class definitions – used to define:
    • object types
    • value types and their associated boxed types

Note that while class definitions always define class types, not all class types require a class definition. Array types and pointer types, which are implicitly defined, are also class types. See Classes, Interfaces and Objects.

Similarly, not all types defined by a class definition are object types. Array types, explicitly defined object types, and boxed types are object types. Pointer types, function pointer types, and value types are not object types. See Classes, Interfaces and Objects.


Array Types

An array type is defined by specifying the element type of the array, the rank (number of dimensions) of the array, and the upper and lower bounds of each dimension of the array. All of these are included in any signature of an array type, although they may be marked as dynamically (rather than statically) supplied. Hence, no separate definition of the array type is needed.

Values of an array type are objects; hence an array type is a kind of object type (see Classes, Interfaces and Objects). Array objects are defined by the VOS to be a repetition of locations where values of the array element type are stored. The number of repeated values is determined by the rank and bounds of the array.

Note: In V1 only type signatures, not location signatures, are allowed as array element types.

Exact array types are created automatically by the VES when they are required. Hence, the operations on an array type are defined by the VOS. These generally are: indexing the array to read and write a value, computing the address of an element of the array (a managed pointer), and querying for the rank, bounds, and the total number of values stored in the array.

CLS Rule 14: Arrays elements must have a CLS-compliant type, a fixed number of dimensions, and all dimensions of the array must have zero lower bounds. While the abstract type System.Array is considered CLS-compliant it does have types that inherit from it but are not CLS-compliant.

CLS (consumer): there is no need to support arrays of other types, even when dealing with instances of System.Array.

CLS (extender): there is no need to provide syntax to define other types of arrays or to extend interfaces or classes that use other array types. Must provide access to the type System.Array, but may assume that all instances will have a CLS-compliant type.

CLS (framework): other array types may not appear in exposed members.

Array types form a hierarchy, with all array types inheriting from the type System.Array. This is an abstract class that represents all arrays regardless of the type of their elements or their rank. Arrays of one dimension with zero lower bound for their elements (sometimes called vectors) have a type based on the type of the elements in the array, regardless of the upper bound. Arrays with more than one dimension or one dimension but with non-zero lower bound have the same type if they have the same element type and rank, regardless of lower bound on the array. Zero-dimensional arrays are not supported.

Consider the following examples, using the syntax of the IL Assembler (ilasm):

Static specification of type | Actual type constructed | Allowed in CLS?
int32[] | vector of int32 | Yes
int32[0..5] | vector of int32 | Yes
int32[1..5] | array, rank 1, of int32 | No
int32[,] | array, rank 2, of int32 | Yes
int32[0..3, 0..5] | array, rank 2, of int32 | Yes
int32[1.., 0..] | array, rank 2, of int32 | No


Pointer Types

A pointer type is defined by specifying a location signature for the location the pointer references. Any signature of a pointer type includes this location signature. Hence, no separate definition of the pointer type is needed.

While pointer types are Reference Types, values of a pointer type are not objects (see Classes, Interfaces and Objects), and hence it is not possible, given a value of a pointer type, to determine its exact type. The VOS provides two typesafe operations on pointer types: one to load the value from the location referenced by the pointer and the other to store an assignment compatible value into that location. The VOS also provides three operations on pointer types (byte-based address arithmetic): adding and subtracting integers from pointers, and subtracting one pointer from another. The results of the first two operations are pointers to the same type signature as the original pointer. See the IL Instruction Set specification for details.

CLS Rule 15: Pointer types are not CLS-compliant.

CLS (consumer): there is no need to support pointer types.

CLS (extender): there is no need to provide syntax to define or access pointer types.

CLS (framework): pointer types must not be externally exposed.


Interface Type Definition

An interface definition defines an interface type. An interface type is a named group of methods, locations and other contracts that must be implemented by any object type that supports the interface contract of the same name. An interface definition is always an incomplete description of a value, and as such can never define a class type or an exact type, nor can it be an object type.

Zero or more object types can support an interface type, and only object types can support an interface type. An interface type can require that objects that support it must also support other (specified) interface types. An object type that supports the named interface contract must provide a complete implementation of the methods, locations, and other contracts specified (but not implemented by) the interface type. Hence, a value of an object type is also a value of all of the interface types the object type supports. Support for an interface contract is declared, never inferred, i.e. the existence of implementations of the methods, locations, and other contracts required by the interface type does not imply support of the interface contract.

CLS Rule 16: CLS-compliant tools must deal with the fact that a single type may implement two interfaces and those interfaces may each require the definition of a method of the same name and signature. These methods are considered distinct and need not have the same implementation.

CLS (consumer): must provide some means for accessing all methods of all interfaces supported by a type

CLS (extender): must provide a mechanism for providing independent implementations for all methods of all interfaces supported by a type.

CLS (framework): must assume that implementations of methods of the same name and signature on different interfaces are independent.

Interfaces types are necessarily incomplete since they say nothing about the representation of the values of the interface type. For this reason, an interface type definition cannot provide field definitions for values of the interface type (i.e. instance fields), although it can declare static fields (see Static Fields and Static Methods).

Similarly, an interface type definition cannot provide implementations for any methods on the values of the types. However, an interface type definition can and usually does define method contracts (method name and method signature) that must be implemented by supporting types. An interface type definition can define and implement static methods (see Static Fields and Static Methods) since static methods are associated with the interface type itself rather than with any value of the type.

Note: In V1, interfaces can have static or virtual methods, but not instance methods.

CLS Rule 17: CLS-compliant interfaces do not define static or instance methods, nor do they define fields.

CLS (consumer): need not accept interfaces that violate these rules.

CLS (extender): need not provide syntax to author interfaces that violate these rules.

CLS (framework): must not externally expose interfaces that violate these rules. Where static methods, instance methods, or fields are required a separate class may be defined that provides them.

Interface types can also define event and property contracts that must be implemented by object types that support the interface. Since event and property contracts reduce to sets of method contracts (Contracts), the above rules for method definitions apply. For more information, see Event Definitions and Property Definitions.

Interface type definitions can specify other interface contracts that implementations of the interface type are required to support. See Interface Type Inheritance for specifics.

An interface type is given a visibility attribute, as described in section Visibility, Accessibility, and Security, that controls from where the interface type can be referenced. An interface type definition is separate from any object type definition that supports the interface type. Hence, it is possible, and often desirable, to have a different visibility for the interface type and the implementing object type. However, since accessibility attributes are relative to the implementing type rather than the interface itself, all members of an interface must have public accessibility, and no security permissions may be attached to members or to the interface itself.


Class Type Definition

All types other than interfaces, and those types for which a definition is automatically supplied by the VOS, are defined by class definitions. A class type is a complete specification of the representation of the values of the class type and all of the contracts (class, interface, method, property, and event) that are supported by the class type. Hence, a class type is an exact type. A class definition, unless it specifies that the class is an abstract object type, not only defines the class type: it also provides implementations for all of the contracts supported by the class type.

A class definition, and hence the implementation of the class type, always resides in some assembly. An assembly is a configured set of loadable code modules and other resources that together implement a unit of functionality.

Note that while class definitions always define class types, not all class types require a class definition. Array types and pointer types, which are implicitly defined, are also class types. See Classes, Interfaces and Objects.

An explicit class definition is used to define:

  • An object type (see Classes, Interfaces and Objects).
  • A value type and its associated boxed type (see Boxing and Unboxing of Values).

An explicit class definition:

  • Names the class type.
  • Implicitly assigns the class type name to a scope, i.e. the assembly that contains the class definition, (see Assemblies and Scoping).
  • Defines the class contract of the same name (see Contracts).
  • Defines the representations and valid operations of all values of the class type using member definitions for the fields, methods, properties, and events (Member Definitions).
  • Defines the static members of the class type (see Member Definitions).
  • Specifies any other interface and class contracts also supported by the class type.
  • Supplies implementations for member and interface contracts supported by the class type.
  • Explicitly declares a visibility for the type, either public or assembly (see Visibility).
  • May optionally specify a method to be called to initialize the type before any access to it is permitted. In general, this method is guaranteed to be called before any instances are created, any static variables are referenced (or their address computed), and any methods on it are called.


Object Type Definitions

All objects are instances of an object type. The object type of an object is set when the object is created and it is immutable. The object type describes the physical structure of the instance and the operations that are allowed on it. All instances of the same object type have the same structure and the same allowable operations. Object types are explicitly declared by a class type definition, with the exception of Array types, which are intrinsically provided by the VES.

Scope and Visibility

Since object type definitions are class type definitions, object type definitions implicitly specify the scope of the name of object type to be the assembly that contains the object type definition, see the Scoping of Names section. Similarly, object type definitions must also explicitly state the visibility attribute of the object type (either public or assembly); see Visibility.

Concreteness

An object type may be marked as abstract by the object type definition. An object type that is not marked abstract is by definition concrete. Only object types can be declared as abstract. Only an abstract object type is allowed to define method contracts for which the type or the VES does not also provide the implementation. Such method contracts are called abstract methods (see Member Definitions). All methods on an abstract class need not be abstract.

It is an error to attempt to create an instance of an abstract object type, whether or not the type has abstract methods. An object type that derives from an abstract object type can be concrete if it provides implementations for any abstract methods in the base object type and is not itself marked as abstract. Instances can be made of such a concrete derived class. Locations may have an abstract type, and instances of a concrete type that derives from the abstract type may be stored in them.

Type Members

Object type definitions include member definitions for all of the members of the type. Briefly, members of a type include fields into which values are stored, methods that can be invoked, properties that are available, and events that can be raised. Each member of a type can have attributes as described in Type Members.

  • Fields of an object type specify the representation of values of the object type by specifying the component pieces from which it is composed (see Fields, Array Elements, and Values). Static fields specify fields associated with the object type itself (see Static Fields and Static Methods). The fields of an object type are named and they are typed via location signatures. The names of the members of the type are scoped to the type (see Assemblies and Scoping). Fields are declared using a field definition (Field Definitions).
  • Methods of an object type specify operations on values of the type (see Methods). Static methods specify operations on the type itself (see Static Fields and Static Methods). Methods are named and they have a method signature. The names of methods are scoped to the type (see Assemblies and Scoping). Methods are declared using a method definition (see Method Definitions).
  • Properties of an object type specify named values that are accessible via methods that read and write the value. The name of the property is the grouping of the methods; the methods themselves are also named and typed via method signatures. The names of properties are scoped to the type (see Assemblies and Scoping). Properties are declared using a property definition (see Property Definitions).
  • Events of an object type specify named state transitions in which subscribers can register/unregister interest via accessor methods. When the state changes, the subscribers are notified of the state transition. The name of the event is the grouping of the accessor methods; the methods themselves are also named and typed via method signatures. The names of events are scoped to the type (see Assemblies and Scoping). Events are declared using an event definition (see Event Definitions).

Supporting Interface Contracts

Object type definitions can declare that they support zero or more interface contracts. Declaring support for an interface contract places a requirement on the implementation of the object type to fully implement that interface contract. Implementing an interface contract always reduces to implementing the required set of methods, i.e. the methods required by the interface type.

The different types that the object type implements, i.e. the object type and any implemented interface types, are each a separate logical grouping of named members. If a class Foo implements an interface IFoo and IFoo declares a member method int a() and the class also declares a member method int a(), there are two members, one in the IFoo interface type and one in the Foo class type. An implementation of Foo must provide an implementation for both, potentially shared.

Similarly, if a class implements two interfaces IFoo and IBar each of which defines a method int a() the class must supply two method implementations, one for each interface, although they may share the actual code of the implementation.

CLS Rule 18: CLS-compliant classes, boxed value types, and interfaces must not require the implementation of non-CLS-compliant interfaces.

Supporting Class Contracts

Object type definitions can declare support for one other class contract. Declaring support for another class contract is synonymous with object type inheritance (see Object Type Inheritance).

Constructors

New values of an object type are created via constructors. Constructors are defined via a special form of method contract, which defines the method contract as a constructor for a particular object type. The constructors for an object type are part of the object type definition. While the VOS and VES ensure that only a properly defined constructor is used to make new values of an object type, the ultimate correctness of a newly constructed object is dependent on the implementation of the constructor itself.

Object types must define at least one constructor method, but that method need not be public. Creating a new value of an object type by invoking a constructor involves the following steps in order:

  1. Space for the new value is allocated in managed memory.
  2. VES data structures of the new value are initialized and user-visible memory is zeroed.
  3. The specified constructor for the object type is invoked.

Inside the constructor, the object type can do any initialization it chooses (possibly none).

CLS Rule 19: An object constructor must call a base class constructor before any access occurs to inherited instance data.

CLS Rule 20: An object constructor shall not be called except as part of the creation of an object, and no object should be initialized twice.

CLS (consumer): Must provide syntax for choosing the constructor to be called when an object is created.

CLS (extender): Must provide syntax for defining constructor methods with different signatures. May issue a compiler error if the constructor does not obey these rules.

CLS (framework): May assume that object creation includes a call to one of the constructors, and that no object is initialized twice. System.MemberwiseClone and deserialization (including object remoting) may not run constructors.

Finalizers

A class definition that creates an object type can supply an instance method to be called when an instance of the class is no longer accessible. The garbage collector calls finalizers, at most once per instance, automatically.


Value Type Definition

Not all types defined by a class definition are object types (see Classes, Interfaces and Objects); in particular, value types are not object types but they are defined using a class definition. A class definition for a value type defines both the (unboxed) value type and the associated boxed type (see Boxing and Unboxing of Values). The members of the class definition define the representation of both:

  • When a non-static method (i.e. an instance or virtual method) is called on the value type its this pointer is a managed reference to the instance, whereas when the method is called on the associated boxed type the this pointer is an object reference.

Note: In PDC boxed value types don’t exist as independent types. Therefore, instance methods on value types receive a this pointer that is a managed pointer to the unboxed type whereas virtual methods (including those on interfaces implemented by the value type) receive an instance of the boxed type.

  • Value types do not support interface contracts, but their associated boxed types do.
  • A value type does not inherit; rather the base type specified in the class definition defines the base type of the boxed type.
  • The base type of a boxed type cannot have any fields.
  • Unlike object types, instances of value types do not require a constructor to be called when an instance is created. Instead, the contents of a value type are guaranteed to be initially zero (null for object fields).


Type Inheritance

Inheritance of types is another way of saying that the derived type guarantees support for all of the type contracts of the base type. In addition, the derived type usually provides additional functionality or specialized behavior. A type inherits from a base type by implementing the type contract of the base type. An object type inherits from one other object type and zero or more interface types. An interface type inherits from zero or more other interfaces. Value types do not inherit, although the associated boxed type is an object type and hence inherits from other types

The derived class type must support all of the supported interfaces contracts, class contracts, event contracts, method contracts, and property contracts of its base type. In addition, all of the locations defined by the base type are also defined in the derived type. The inheritance rules guarantee that code that was compiled to work with a value of a base type will still work when passed a value of the derived type. Because of this, a derived type also inherits the implementations of the base type. The derived type may extend, override, and/or hide these implementations.


Object Type Inheritance

With the sole exception of System.Object, which does not inherit from any other object type, all object types must either explicitly or implicitly declare support for (inherit from) exactly one other object type. The graph of the inherits-relation must form a singly rooted tree with System.Object at the base, i.e. all object types must eventually inherit from the type System.Object.

An object type declares it cannot be used as a base type (be inherited from) by declaring that it is a sealed type. It is an error if an object type is marked both abstract and sealed at the same time.

CLS Rule 21: A CLS-compliant class must inherit from a CLS-compliant class. System.Object is CLS-compliant.

Arrays are object types and as such inherit from other object types. Since arrays object types are manufactured by the VES, the inheritance of arrays is fixed. See Array Types.


Value Type Inheritance

Value Types do not inherit from any other type. The boxed type associated with a value type is an object type, and as such must specify either implicitly or explicitly what object type is its base type, i.e. the object type from which it inherits. The base object type that the boxed type derives from must have no fields defined. A boxed type is sealed so no other type can inherit from it.

CLS Rule 22: The CLS restricts boxed value types to inherit directly from System.ValueType unless they are enumerations, in which case they must inherit directly from System.Enum.

Note: In V1, the CLS restriction listed above is directly enforced by the VES.


Interface Type Inheritance

Interface types can inherit from multiple interface types, i.e. an interface contract can list other interface contracts that must also be supported. Any type that implements support for an interface type must also implement support for all of the inherited interface types. This is different from object type inheritance in two ways.

  • Object types form a single inheritance tree; interface types do not.
  • Object type inheritance specifies how implementations are inherited; interface type inheritance does not, since interfaces do not define implementation. Interface type inheritance specifies additional contracts that an implementing object type must support.

To highlight the last difference, consider an interface, IFoo, that has a single method. An interface, IBar, which inherits from it is requiring that any object type that supports IBar also support IFoo. It does not say anything about what methods IBar itself must have.


Member Inheritance

Only object types can inherit implementations, hence only object types can inherit members (see Type Inheritance). Interface types, while they do inherit from other interface types, only inherit the requirement to implement method contracts, never fields or method implementations.


Field Inheritance

A derived object type inherits all of the non-static fields of its base object type. This allows instances of the derived type to be used wherever instances of the base type are expected (the shapes, or layouts, of the instances will be the same). Static fields are not inherited. Just because a field exists does not mean that it can be read or written. The type visibility, field accessibility, and security attributes of the field definition (see Visibility, Accessibility, and Security) determine if a field is accessible to the derived object type.


Method Inheritance

A derived object type inherits all of the instance and virtual methods of its base object type. It does not inherit constructors or static methods. Just because a method exists does not mean that it can be invoked. It must be accessible via the typed reference that is being used by the referencing code. The type visibility, method accessibility, and security attributes of the method definition (see Visibility, Accessibility, and Security) determine if a method is accessible to the derived object type.

A derived object type can hide a non-virtual (i.e. static or instance) method of its base type by providing a new method definition with the same name or same name and signature. Either method can still be invoked, subject to method accessibility rules, since the type that contains the method always qualifies a method reference.

Virtual methods can be marked as final, in which case they cannot be overridden in a derived object type. This ensures that the implementation of the method is available, by a virtual call, on any object that supports the contract of the base class that supplied the final implementation. If a virtual method is not final it is possible to demand a security permission in order to override the virtual method, so that the ability to provide an implementation can be limited to classes that have particular permissions. When a derived type overrides a virtual method, it may specify a new accessibility for the virtual method, but the accessibility in the derived class must permit at least as much access as the access granted to the method it is overriding. See Visibility, Accessibility, and Security.


Property and Event Inheritance

Properties and events are not inherited; they should be handled in the same way static methods are handled.


Hiding, Overriding, and Layout

There are two separate issues involved in inheritance. The first is what contracts an object must implement and hence what member names and signatures it must provide. The second is the layout of the object so that an instance of a derived type can be substituted for an instance of any of its base types. Only the non-static fields and the virtual methods that are part of the derived type affect the layout of an object.

The VOS provides independent control over both the names that are visible from a base type (hiding) and the sharing of layout slots in the derived class (overriding). Hiding is controlled by marking a member in the derived class as either hide by name or hide by name-and-signature. Hiding is always performed based on the kind of member, that is, derived field names may hide base field names, but not method names, property names, or event names. If a derived member is marked hide by name, then members of the same kind in the base class with the same name are not visible in the derived class; if the member is marked hide by name-and-signature then only a member of the same kind with exactly the same name and type (for fields) or method signature (for methods) is hidden in the derived class.

For example:

class Base
{ field System.Int32 A;
  field System.String A;
  method System.Int32 A();
  method System.Int32 A(System.Int32);
}
class Derived inherits from Base
{ hide-by-name field System.Int32 A;
  hide-by-name+signature method System.Int32 A();
}

The member names available in type Derived are:

Kind of member Type / Signature of member Name of member
Field

System.Int32

A
Method

() -> System.Int32

A

Method

(System.Int32) -> System.Int32

A

While hiding applies to all members of a type, overriding deals with object layout and is applicable only to instance fields and virtual methods. The VOS provides two forms of member overriding, new slot and expect existing slot. A member of a derived type that is marked as a new slot will always get a new slot in the object’s layout, guaranteeing that the base field or method is available in the object by using a qualified reference that combines the name of the base type with the name of the member and its type or signature. A member of a derived type that is marked as expect existing slot will re-use (i.e. share or override) a slot that corresponds to a member of the same kind (field or method), name, and type if one already exists from the base type; if no such slot exists, a new slot is allocated and used.

The general algorithm that is used for determining the names in a type and the layout of objects of the type is roughly as follows:

  • Flatten the inherited names (using the “hide by name” or “hide by name-and-signature” rule) ignoring accessibility rules.
  • For each new member that is marked “expect existing slot”, look to see if an exact match on kind (i.e. field or method), name, and signature exists and use that slot if it is found, otherwise allocate a new slot.
  • After doing this for all new members, add these new member-kind/name/signatures to the list of members of this type
  • Finally, remove any inherited names that match the new members based on the “hide by name” or “hide by name-and-signature” rules.


Member Definitions

Object type definitions, interface type definitions, and value type definitions can include member definitions. Field definitions define the representation of values of the type by specifying the substructure of the value. Method definitions define operations on values of the type and operations on the type itself (static methods). Property and event definitions can only be defined on object types. Property and events define named groups of accessor method definitions that implement the named event or property behavior. Nested type declarations define types whose names are scoped by the enclosing type and whose instances have full access to all members of the enclosing class.

Depending on the kind of type definition, there are restrictions on the member definitions allowed.


Method Definitions

Method definitions are composed of a name, a method signature, and optionally an implementation of the method. The method signature defines the calling convention, type of the parameters to the method, and the return type of the method (see Method Signatures). The implementation is the code to execute when the method is invoked. A value type or object type can define only one method of a given name and signature. However, a derived object type may have methods that are of the same name and signature as its base object type. See Method Inheritance and Hiding, Overriding, and Layout.

The name of the method is scoped to the type (see Assemblies and Scoping section). Methods can be given accessibility attributes (see Accessibility of Members). Methods can only be invoked with arguments that are assignment compatible with the parameters types of the method signature. The return value of the method must also be assignment compatible with the location in which it is stored.

Methods can be marked as static, indicating that the method is not an operation on values of the type but rather an operation associated with the type as a whole. Methods not marked as static define the valid operations on a value of a type. When a non-static method is invoked, a particular value of the type, referred to as this or the this pointer, is passed as an implicit parameter.

A method definition that does not include a method implementation must be marked as abstract, unless the method definition is part of an interface definition. All non-static methods of an interface definition are abstract. Abstract method definitions are only allowed in object types that are marked as abstract.

A non-static method definition in an object type may be marked as virtual, indicating that an alternate implementation may be provided in derived types. All method definitions in interface definitions must be virtual methods. Virtual method can be marked as final, indicating that derived object types are not allowed to override the method implementation.


Field Definitions

Fields definitions are composed of a name and a location signature. The location signature defines the type of the field and the accessing constraints, see Location Signatures. A value type or object type can define only one field of a given name. However, a derived object type may have fields that are of the same name as its base object type. See the Field Inheritance and Hiding, Overriding, and Layout sections.

The name of the field is scoped to the type (see Assemblies and Scoping). Fields can be given accessibility attributes, see the Visibility, Accessibility, and Security section. Fields can only store values that are assignment compatible with the type of the field (see the Assignment Compatible Locations section).

Fields can be marked as static, indicating that the field is not part of values of the type but rather a location associated with the type as a whole. Locations for the static fields are created when the type is loaded and initialized when the type is initialized.

Fields not marked as static define the representation of a value of a type by defining the substructure of the value (see the Fields, Array Elements, and Values section). Locations for such fields are created within every value of the type whenever a new value is constructed. They are initialized during construction of the new value. A non-static field of a given name is always located at the same place within every value of the type.

A field that is marked serializable is to be serialized as part of the persistent state of a value of the type. See the Serialization Specification for details of serialization and related security issues.


Property Definitions

A property definition defines a named value and the methods that access the value. A property definition defines the accessing contracts on that value. Hence, the property definition specifies which accessing methods exist and their respective method contracts. An implementation of a type that declares support for a property contract must implement the accessing methods required by the property contract. The implementation of the accessing methods defines how the value is retrieved and stored.

A property definition is always part of either an interface definition or a class definition. The name and value of a property definition is scoped to the object type or the interface type that includes the property definition. While all of the attributes of a member may be applied to a property (accessibility, static, etc.) these are not enforced by the VOS. Instead, the VOS requires that the method contracts that comprise the property must match the method implementations, as with any other method contract. There are no IL instructions associated with properties, just metadata.

By convention, properties define a getter method (for accessing the current value of the property) and optionally a setter method (for modifying the current value of the property). The VOS places no restrictions on the set of methods associated with a property, their names, or their usage.

CLS Rule 23: The methods that implement the getter and setter methods of a property must be marked mdSpecialName in the metadata.

CLS Rule 24: The accessibility of the property and of its accessors must be identical.

CLS Rule 25: The property and its accessors must all be static, all be virtual, or all be instance.

CLS Rule 26: The type of the property must match the return type of the getter and the last argument of the setter. The types of the parameters of the property must match the types of the parameters to the getter and the types of all but the final parameter of the setter. All of these types must be CLS-compliant, and must not be managed pointers (i.e. cannot be passed by reference).

CLS Rule 27: Properties must adhere to a specific naming pattern. See Naming Patterns.

CLS (consumer): Must ignore the mdSpecialName bit in appropriate name comparisons and must adhere to identifier rules. Otherwise, no direct support other than the usual access to the methods which define the property.

CLS (extender): Must ignore the mdSpecialName bit in appropriate name comparisons and must adhere to identifier rules. Otherwise, no direct support other than the usual access to the methods which define the property. In particular, an extender need not be able to define properties.

CLS (framework): Must design understanding that not all CLS languages will access the property using special syntax.


Event Definitions

The VOS supports events in precisely the same way that it supports properties (see the Property Definitions section). The conventional methods, however, are different and include means for subscribing and unsubscribing to events as well as for firing the event.

CLS Rule 28: The methods that implement the event must be marked mdSpecialName in the metadata.

CLS Rule 29: The add and remove methods for an event must both either be present or absent.

CLS Rule 39: The add and remove methods for an event must each take one parameter whose type defines the type of the event and that must be derived from System.Delegate.

CLS Rule 40: Events must adhere to a specific naming pattern. See Naming Patterns

CLS (consumer): Must ignore the mdSpecialName bit in appropriate name comparisons and must adhere to identifier rules. Otherwise, no direct support other than the usual access to the methods which define the event.

CLS (extender): Must ignore the mdSpecialName bit in appropriate name comparisons and must adhere to identifier rules. Otherwise, no direct support other than the usual access to the methods which define the event. In particular, an extender need not be able to define events.

CLS (framework): Must design understanding that not all CLS languages will access the event using special syntax.


Nested Type Definitions

A nested type definition is identical to a top-level type definition, with one exception: a top-level type has a visibility attribute, while the visibility of a nested type is the same as the visibility of the enclosing type. See Nested Types.


NGWS Runtime Metadata

New types – value types and reference types – are introduced into the NGWS runtime type system via type declarations expressed in metadata. Not limited to type declarations, metadata is a structured way to represent all information that the NGWS runtime uses to locate and load classes, lay out instances in memory, resolve method invocations, translate IL to native code, enforce security, and set up runtime context boundaries. Every NGWS PE/COFF image (see the File Format specification for details) carries a compact metadata binary that is emitted into the image by the NGWS runtime enabled development tool or compiler.

Each NGWS runtime enabled language will expose a language-appropriate syntax for declaring types and members and for decorating them with attributes that express what services they require of the runtime. Type imports are also handled in a language-appropriate way, and it is the development tool or compiler that consumes the metadata to expose the types that the developer sees.

In fact, the typical component or application developer will not need to be aware of the rules for emitting and consuming NGWS runtime metadata. While it may help a developer to understand the structure of metadata, the rules outlined in this section are primarily of interest to tool builders and compiler writers.


Components and Assemblies

Because each NGWS runtime image carries the metadata for declarations, implementations, and references specific to that image, the image-specific metadata is referred to as component metadata, and the resulting component is said to be self-describing. To draw an analogy, this information integrates and extends what a combination of typelibs, IDL files, DLLRegisterServer, and a myriad of custom .xxx files express in Unmanaged COM, in disparate formats and as side notes to the actual executable file. In fact, the presence of metadata in the image is part of what it means to be an NGWS component.

Collections of NGWS components and other files are packaged together for deployment into assemblies, discussed in more detail in a later section. More than just a deployment-time concept, an assembly is a logical unit of functionality that serves as the primary unit of reuse in NGWS. Effectively, assemblies establish a name scope for types.

This means that types declared and implemented in individual components are exported for use by other implementations via the assembly in which the component participates. All references to a type are scoped by the identity of the assembly in whose context the type is being used. NGWS provides services to locate a referenced assembly and “ask” that assembly to resolve the type reference. It is this mechanism that provides an isolation scope for applications: the assembly alone controls what implementation bits participate in the assembly, and the runtime has infrastructure to honor these rules.

Every assembly has a manifest that declares what components make up (or are allowed to make up) the assembly, what types are exported, how type references are resolved, how assembly references are resolved, configuration rules, etc. An assembly may carry an explicit manifest or the manifest may be implicit. When explicit, the manifest includes information about exactly what implementations (specific dependent component and assembly versions) were used to successfully build the manifest, and, optionally, version binding rules to be enforced at runtime. When implicit, the manifest is derived from the components that are declared to make up the assembly, where the binding rules are, effectively, “these bits only.”

Just as NGWS components are self-describing via component metadata persisted into the NGWS runtime image, so are assemblies self-describing via their manifests: All of the resource declarations and resolution rules are expressed declaratively and persisted into a binary image that can be consumed by the NGWS runtime metadata engine. When a single file makes up a NGWS runtime assembly, there is no need to carry two separate NGWS runtime files, one to contain the component metadata and one to contain the assembly metadata. Rather, the component and assembly metadata may be combined into the metadata binary of the NGWS runtime loadable image.


Accessing Metadata

Metadata is emitted into and read from a NGWS runtime image using either a set of unmanaged APIs documented in the NGWS runtime Metadata API Specification or through the managed Reflection and Reflection Emit Services documented in the NGWS runtime Base Class Library Specification. The physical data structures persisted in a metadata binary are not manipulated directly by tools or compilers, and so they are not specified here or elsewhere in NGWS runtime documentation. Instead the APIs provide a layer of isolation between the underlying data structures and clients of those data structures. This abstraction enables the runtime to continue to optimize the in-memory and on-disk structures for compile-time and runtime scenarios without requiring updating all compilers as these structures are improved. It also enables a pluggable persistence format architecture that allows NGWS runtime binary representations, Unmanaged COM type libraries, and other formats to be imported into or from memory transparently. The primary design goals for these APIs include maximizing performance and minimizing overhead – the metadata engine stops just short of providing direct access to the in-memory data structures.

While the metadata data structures allow the full VOS Type Model to be expressed, the unmanaged metadata APIs do not check or enforce constraints expressed in the model. This is to reduce compile-time overhead. Instead, the emitting tools and compilers are expected to check the constraints and emit valid NGWS runtime metadata. The NGWS SDK contains a tool, PEVerify, that can be used to verify a NGWS runtime image, including the metadata, during development.

When a class is loaded at runtime, the NGWS runtime loader imports the metadata into its own in-memory data structures, which can be browsed via the NGWS frameworks Reflection services. The Reflection services do much more work for a browser or tool than the unmanaged metadata APIs do, such as automatically walking the inheritance hierarchy to obtain information about inherited methods and fields; the unmanaged metadata APIs return only the direct member declarations for a given class and expect the API client to make additional calls to walk the hierarchy and enumerate inherited methods. The former approach exposes a higher-level view of metadata, where the latter approach puts the API client in complete control of walking the data structures.


Metadata Tokens

Although purely an implementation artifact, the phrase metadata token is found throughout the NGWS frameworks and runtime specifications. The File Format Specification talks about metadata tokens embedded in various sections of a NGWS PE/COFF image. Metadata tokens are embedded in IL and native code to encode method invocations and field accesses at call sites; the token is used by various runtime services to retrieve information from metadata about the reference and the type on which it was scoped in order to resolve the reference. While metadata tokens are primarily of used with the unmanaged Metadata APIs, some of them can also be used with the managed APIs, as shown in the following table.

Logically, a metadata token is a typed identifier of a metadata object (type declaration, member declaration, etc.): With each call to emit a declaration into metadata, a token is returned. Given a token, its type can be determined, and using the appropriate IMetaDataImport method the metadata engine can retrieve the specific metadata attributes that had been persisted for that metadata object. However, it’s important to understand that a metadata token is not a persistent identifier. Rather it is scoped to a specific metadata binary. Physically, a metadata token is an index into a metadata data structure, so access is fast and direct.

Compilers and tools emit metadata by calling the unmanaged emit APIs or managed Reflection Emit classes during compilation and link or, with RAD tools, as a part of building components or applications. The APIs work over in-memory data structures that, at save time, are compressed and persisted as a binary format into the target of the compilation unit (“.obj OBJ file”), executable file (“.EXE file”), shared library (“.DLL file”), or stand-alone metadata binary file.

When multiple compilation units are linked to form an .EXE or .DLL, the unmanaged APIs provide a method used to merge the metadata sections from each compilation unit into a single integrated metadata binary. When two scopes are merged, tokens from the import scope are remapped into tokens in the target emit scope. When a metadata scope is saved, there are various format optimizations that can result in token remaps, which must be handled by the application.

A list of token types and a more detailed explanation of how to manage metadata tokens are provided in the NGWS runtime Metadata API Specification.


Member Signatures in Metadata

Every location — including fields, parameters, method return values, and properties — has a type, and a specification for its type is carried in metadata.

A value type describes values that are represented as a sequence of bits. A reference type describes values that are represented as the location of a sequence of bits. The NGWS runtime provides an explicit set of built-in types, each of which has a default runtime form as either a value type or a reference type. The metadata APIs may be used to declare additional types, and part of the type specification of a variable encodes the identity of the type as well as what form (value or reference) the type is to take at runtime.

Metadata tokens representing encoded types are passed to IL instructions that accept a type (newobj, newarray, ldtoken). See the IL Instruction Set specification.

Encoded types are also embedded in member signatures. To optimize runtime binding of field accesses and method invocations, the type and location signatures associated with fields and methods are encoded into member signatures in metadata. A member signature embodies all of the contract information that is used to decide whether a reference to a member succeeds or fails.

The unmanaged metadata APIs do not provide any support for building up signature encodings. Instead, a tool or compiler that uses the unmanaged APIs is expected to supply a properly encoded binary signature with each member declaration or member reference. Refer to the NGWS Runtime Type and Signature Encoding Specification for rules on the type and signature encoding. The managed Reflection Emit classes provide support for constructing signatures. See the Base Class Library Specification.


Unmanaged COM and Unmanaged Code

To enable seamless interoperability with Unmanaged COM, any type declaration (class or interface) is also required to carry a GUID. In addition, a class declaration is expected to carry a version number, and may identify a collection of interfaces that it raises as events (as a collection of type reference or definition metadata tokens). The metadata also provides a mechanism to preserve the order of slots in a vtable, including the ability to declare “placeholder methods” to match externally imposed layout requirements. For details, refer to the COM Interop specification.

It is possible to pass data from NGWS runtime managed code to unmanaged code. This always involves a transition from managed to unmanaged code, which has some runtime cost, but data can often be transferred without copying. When data must be reformatted the VES provides a reasonable set of default behavior, but it is possible to use metadata to explicitly require other forms of marshalling (i.e. reformatted copying). The metadata also allows access to unmanaged methods through pre-existing mechanisms (DLL imports by name and by EAT offset).


Method Implementation Metadata

For each method for which an implementation is supplied in the current NGWS runtime image, the tool or compiler will emit information used by the IL-to-native code compilers, the NGWS runtime loader, and other runtime services. This information includes:

  • Whether the code is managed or unmanaged.
  • Whether the implementation is in native code or IL.
  • The location of the method body in the current image, as an address relative to the start of the image file in which it is located (a Relative Virtual Address, or RVA). Or, alternatively, the RVA is encoded as 0 and other metadata is used to tell the runtime where the method implementation can be found, including:
    • An implementation to be located via the NGWS runtime Interoperability Services. See related specifications for details.
    • Forwarding calls through an imported global static method.

The NGWS Runtime Metadata API Specification describes the various method calls for emitting implementation metadata, depending on whether the emitting tool is a RAD-style (rapid application development) tool, such as Visual Basic (VB) or a scripting language, or a high-level language compiler such as Visual C++ (VC).


Class Layout

In the general case, the NGWS runtime loader is free to lay out the instances of a class in any way it chooses, consistent with the rules of the VOS Type System. However, there are times when a tool or compiler needs more control over the layout. In the metadata, a class is marked with an attribute indicating whether its layout rule is:

  • autolayout: A class marked “autolayout” indicates that the loader is free to lay out the class in any way it sees fit; any layout information that may have been specified is ignored. This is the default.
  • layoutsequential: A class marked “layoutsequential” guides the loader to preserve field order as emitted, but otherwise the specific offsets are calculated based on the NGWS runtime type of the field; these may be shifted by explicit offset, padding, and/or alignment information.
  • explicitlayout: A class marked “explicitlayout” causes the loader to ignore field sequence and to use the explicit layout rules provided, in the form of field offsets and/or overall class size or alignment. There are restrictions on legal layouts, specified in the NGWS Runtime Metadata API Specification.

CLS Rule 31: CLS-compliant types must not use explicitlayout.

CLS (consumer): Need not provide support for types with explicit layout.

CLS (extender): Need not provide syntax for defining types with explicit layout.

CLS (framework): Must not expose types with explicit layout.

It is also possible to specify an overall size for a class. This enables a tool or compiler to emit a value type specification where only the size of the type is supplied. This is useful in declaring NGWS runtime built-in types (such as 32 bit integer). It is also useful in situations where the data type of a member of a structured value type does not have a representation in NGWS runtime metadata (e.g., C++ bit fields). In the latter case, as long as the tool or compiler owns the layout, and the runtime doesn’t need to know the details or play a role in the layout, this is sufficient. Note that this means that the VES can move bits around but can’t marshal across machines – the emitting tool or compiler will need to handle the marshaling.

Optionally, a developer may specify a packing size for a class. This is layout information that is not often used but it allows a developer to control the alignment of the fields. It is not an alignment specification, per se, but rather serves as a modifier that places a ceiling on all alignments. Typical values are 1, 2, 4, 8, or 16.


Assemblies: Name Scopes for Types

An assembly is a collection of resources that are built to work together to deliver a cohesive set of functionality. An assembly carries all of the rules necessary to ensure that cohesion. It is the unit of access to resources in the NGWS runtime.

From the outside looking in, an assembly is a collection of exported resources, including types. Resources are exported by name. From the inside, an assembly is a collection of public (exported) and private (internal to the assembly) resources. It is the assembly that determines what resources are to be exposed outside of the assembly and what resources are accessible only within the current assembly scope. It is the assembly that controls how a reference to a resource, public or private, is mapped onto the bits that implement the resource. For types in particular, the assembly may also supply runtime configuration information. An NGWS runtime image or file can be thought of as a packaging of type declarations and implementations, where the packaging decisions could change under the covers without affecting clients of the assembly.

The identity of a type in the runtime is its assembly scope + its declared name. The same declared type deployed in two different assemblies is considered two different runtime types.

Assembly Dependencies: An assembly may depend on other assemblies. This happens when implementations in the scope of one assembly reference resources that are scoped in / owned by another assembly.

  • All references to other assemblies are resolved under the control of the current assembly scope. This gives an assembly an opportunity to control how a reference to another assembly is mapped onto a particular version (or other characteristic) of that referenced assembly (although that target assembly has sole control over how the referenced resource is resolved to an implementation).
  • It is always possible to determine which assembly scope a particular implementation is running in. All requests originating from that assembly scope are resolved relative to that scope.

From a deployment perspective, an assembly may be deployed by itself, with the assumption that any other referenced assemblies will be available in the deployed environment. Or, it may be deployed with its dependent assemblies.

Manifests: Every assembly has a manifest that declares what modules make up the assembly, what types are exported, how type references are resolved, how assembly references are resolved, configuration rules, etc. An assembly may carry an explicit manifest or the manifest may be implicit. When explicit, the manifest includes information about exactly what implementations (specific dependent module and assembly versions) were used to successfully build the manifest, regardless of whether the assembly chooses to require binding to these specific versions at runtime. When implicit, the manifest is derived from the modules that are declared to make up the assembly, effectively binding to specific implementation versions.

Applications: Assemblies introduce isolation semantics for applications. An application is simply an assembly that has an external entry point that triggers (or causes a hosting environment such as IE to trigger) the creation of a new runtime Application Domain. This entry point is effectively the root of a tree of request invocations and resolutions. Some applications are a single, self-contained assembly. Others require the availability of other assemblies to provide needed resources. In either case, when a request is resolved to a module to load, the module is loaded into the same Application Domain from which the request originated. It is possible to monitor or stop an application via the Application Domain.

References: A reference to a type always qualifies a type name with the assembly scope within which the reference is to be resolved – that is, an assembly establishes the name scope of available resources. However, rather than establishing relationships between individual modules and referenced assemblies, every reference is resolved through the current assembly. This allows each assembly to have absolute control over how references are resolved. This is depicted by the following diagram:


Metadata Extensibility

NGWS runtime metadata is extensible. There are three reasons this is important:

  • The NGWS frameworks and runtime Common Language Specification (CLS) is a specification for conventions that languages and tools agree to support in a uniform way for better language integration. The CLS constrains parts of the VOS model, and the CLS introduces higher-level abstractions that are layered over the VOS. It is important that the metadata be able to capture these sorts of development-time abstractions that are used by tools even though they are not recognized or supported explicitly by the runtime.
  • It should be possible to represent language-specific abstractions in metadata that are neither NGWS nor CLS language abstractions. For example, it should be possible, over time, to enable languages like C++ to not require separate header files or IDL files in order to use types, methods, and data members exported by compiled modules.
  • It should be possible, in member signatures, to encode types and type modifiers that are used in language-specific overloading. For example, to allow C++ to distinguish int from long even on 32-bit machines where both map to the underlying type int32.

This extensibility comes in the following forms:

  • Every metadata object can carry custom attributes, and the metadata APIs provide a way to declare, enumerate, and retrieve custom attributes. Custom attributes may be identified by a simple name, where the value encoding is opaque and known only to the specific tool, language, or service that defined it. Or, custom attributes may be identified by a type reference, where the structure of the attribute is self-describing (via data members declared on the type) and any tool including the NGWS frameworks Reflection services may browse the value encoding. Refer to the Metadata API Specification for details.

CLS Rule 32: CLS-compliant tools are only required to deal with a subset of the encodings of custom attributes. The only types that can appear in these encodings are: System.Type, System.String, System.Char, System.Boolean, System.Byte, System.Int16, System.Int32, System.Int64, System.Single, System.Double, and any enumeration type based on a CLS-compliant base type.

CLS (consumer): Must be able to read attributes encoded using the restricted scheme. Must be able to attach attributes based on existing attribute classes to any metadata that is emitted. Must implement the rules for the System.AttributeUsageAttribute.

CLS (extender): Must be able to author new classes and new attributes, as well as all requirements for CLS consumer.

CLS (framework): Must externally expose only attributes that are encoded within the CLS rules and following the conventions specified for System.AttributeUsageAttribute

  • In addition to VOS type extensibility, it is possible to emit custom modifiers into member signatures. NGWS runtime will honor these modifiers for purposes of method overloading and hiding, as well as for binding, but will not enforce any of the language-specific semantics. These modifiers can reference the return type or any parameter of a method, or the type of a field. They come in two kinds: required modifiers that anyone using the member must understand in order to correctly use it, and optional modifiers that can be ignored if the modifier is not understood. Refer to the Type and Signature Encoding Specification for details.

CLS Rule 33: CLS-compliant tools shall not generate publicly visible required modifiers, and shall ignore optional modifiers they do not understand. They need not emit either kind of modifier, although they must be able to copy both kinds of modifiers should they exist in metadata that they import.

CLS (consumer): Must be able to read metadata containing optional modifiers and correctly copy signatures that include them. May ignore these modifiers in type matching and overload resolution. May ignore types that become ambiguous when the optional modifiers are ignored, or that use required modifiers.

CLS (extender): Must be able to author signatures that include optional modifiers.

CLS (framework): Must not use required modifiers in externally visible signatures unless they are marked as not CLS-compliant.


Globals, Imports, and Exports

The NGWS runtime type system does not have the notion of global statics: all VOS statics are associated with a particular class. Nonetheless, the metadata is designed to support languages that rely on static data that is stored directly in a PE/COFF file and accessed by its relative virtual address. In addition, while access to managed data and managed functions is mediated entirely through the metadata itself, the metadata provides a mechanism for accessing unmanaged data and unmanaged code through the PE/COFF file’s Import Address Table (IAT) and for exposing unmanaged entry points to managed code through the Export Address Table (EAT).

CLS Rule 34: Global static fields and methods are not CLS-compliant.


Scoped Statics

The NGWS runtime Type Model does not include a model for file- or function-scoped static functions or data members. However, there are times when a compiler needs a metadata token to emit into IL for a scoped function or data member. The metadata allows members to be marked so that they are never visible/accessible outside of the PE/COFF file in which they are declared and for which the compiler guarantees to enforce all access rules. More detail is provided in the Metadata API Specification.


Common Language Specification

The primary design challenge in creating a common language specification is choosing the right size subset – large enough that it is properly expressive and small enough that all languages can reasonably accommodate it.

Because the CLS is about language interoperability, its rules apply only to “externally visible” items. The CLS assumes that language interoperability is important only across the assembly boundary – that is, within a single assembly there are no restrictions as to the programming techniques that are used. Thus, the CLS rules apply only to items that are visible (see Visibility of Types) outside of their defining assembly and have public, family, or family-or-assembly accessibility (see Accessibility of Members).

It is useful to define some terminology:

  • A type is CLS-compliant if all its publicly accessible parts (those classes, interfaces, methods, fields, properties, and events that are available to code executing in another assembly) either
    • have signatures composed only of CLS-compliant types, or
    • are specifically marked as not CLS-compliant (see Marking Items as CLS-Compliant)
  • A tool is a CLS-compliant consumer tool if it can completely use any CLS-compliant type. That is, call any CLS-compliant method; create an instance of any type CLS-compliant, read and modify any CLS-compliant field, etc.
  • A tool is a CLS-compliant extender tool if it is a consumer tool and, in addition, can extend any (non-sealed) CLS-compliant base class, can implement any CLS-compliant interface, and can place any CLS-compliant custom attribute on all appropriate elements of metadata.

While many things contribute to the decision to exclude a VOS concept from the CLS, one should be explicitly called out: any construct that would make it impossible to rapidly verify code is excluded from the CLS. This allows all CLS-compliant languages to produce verifiable code if they so choose.


Marking Items as CLS-Compliant

The CLS specifies how to mark externally visible parts of an assembly to indicate whether or not they comply with the CLS requirements. This is done using the custom attribute mechanism to extend metadata. The class System.CLSCompliantAttribute indicates what assemblies are CLS-compliant and what types and type members are not CLS-compliant in a compliant assembly.

The constructor for System.CLSCompliantAttribute takes a Boolean argument indicating whether the item with which it is associated is or is not CLS-compliant. This allows any item (assembly, type, or type member) to be explicitly marked as CLS-compliant or not.

If no attribute is associated with an item, then:

  • an assembly is assumed not to be CLS-compliant.
  • a type is assumed to be CLS-compliant if and only if its enclosing type (for nested types) or assembly (for top-level types) is CLS-compliant.
  • other members (methods, fields, properties and events) are CLS-compliant if and only if the type in which it is defined is CLS-compliant.

CLS Rule 35: Types whose CLS-compliance differs from that of the assembly in which they are defined must be marked with the System.CLSCompliantAttribute. Similarly, members whose CLS-compliance differs from that of their class must also be so marked.


Identifiers

Languages that are either case-sensitive or case-insensitive can support the CLS. Since its rules apply only to items exposed to other languages, private members or types that aren’t exported from an assembly may use any names they choose. For interoperation, however, there are some restrictions.

In order to make tools work well with a case-sensitive language it is important that the exact case of identifiers be maintained. At the same time, when dealing with non-English languages encoded in Unicode, there may be more than one way to represent precisely the same identifier that includes combining characters. The CLS requires that identifiers obey the restrictions of the appropriate Unicode standard and persist them in Canonical form C, which preserves case but forces combining characters into a standard representation. See CLS Rule 2, in Collected CLS Rules.

At the same time, it is important that externally visible names not conflict with one another when used from a case-insensitive programming language. As a result, all identifier comparisons must be done internally to CLS-compliant tools using the Canonical form KC, which first transforms characters to their case-canonical representation. See CLS Rule 2, in Collected CLS Rules.

When a CLS-compliant language deals with a non-CLS-compliant language it must be aware that the VOS and VES perform all comparisons using code-point (i.e. byte-by-byte) comparison. Thus, even though the CLS requires that persisted identifiers be in Canonical form C, references to non-CLS identifiers will have to be persisted using whatever encoding the non-CLS language chose to use. It is a language design issue, not covered by the VOS or the CLS, precisely how this should be handled.


Overloading

The VOS Type System, while it describes inheritance, object layout, name hiding, and overriding of virtual methods does not discuss overloading at all. While this is surprising, it arises from the fact that overloading is entirely handled by compilers that target the VOS Type System and not the type system itself. In the metadata, all references to types and type members are fully resolved and include the precise signature that is intended. This choice was made since every programming language has its own set of rules for coercing types and the VES does not provide a means for expressing those rules.

Following the rules of the VOS Type System, it is possible for names to be defined in the same scope as long as they differ in either kind (field, method, etc.) or signature. The CLS imposes a stronger restriction for overloading methods. Within a single scope, a given name may refer to any number of methods provided they differ in any of the following:

  • Number of parameters
  • Type of argument

Notice that the signature includes more information but CLS-compliant languages need not produce or consume classes that differ only by that additional information:

  • Calling convention
  • Custom modifiers
  • Return type
  • Whether a parameter is passed by value or by reference (i.e. as a managed pointer or by-ref)

There is one exception to this rule. For the special names op_Implicit and op_Explicit described in Conversion Operators methods may be provided that differ only by their return type. These are marked specially and may be ignored by compilers that don’t support operator overloading.

Note that properties may not be overloaded by type (that is, by the return type of their getter method), but indexed properties may be overloaded with different number or types of indices (that is, by the number and types of the parameters of its getter method). Otherwise, the overloading rules for properties are identical to the method overloading rules.

CLS Rule 36: Fields and events may not be overloaded within a given class.

CLS Rule 37: Properties, instance methods, and virtual methods may be overloaded based only on the types of their parameters, except the methods named op_Implicit and op_Explicit which may also be overloaded based on their return type.


Operator Overloading

Operator overloading is not, strictly speaking, in the CLS. CLS-compliant producer tools are under no obligation to allow defining of operator overloading. CLS-compliant consumer tools do not have to provide some special mechanism to call these methods. This topic is addressed by the CLS so that

  • languages that do provide operator overloading can describe their rules in a way that other languages can understand, and
  • languages which do not provide operator overloading can still access the underlying functionality without the addition of special syntax.

Operator overloading is described by using the names specified below, and by setting a special bit in the metadata (mdSpecialName) so that they do not collide with the user’s name space. A CLS-compliant producer tool must provide some means for setting this bit. If these names are used, they should have precisely the semantics described here.


Unary Operators

Unary operators take one argument, perform some operation on it, and return the result. They are represented as static methods on the class that defines the type of their one operand. The following table shows the names that are defined.

Name | C++ Operator Symbol
op_Decrement | --
op_Increment | ++
op_UnaryNegation | - (unary)
op_UnaryPlus | + (unary)
op_LogicalNot | !
op_True | Not defined
op_False | Not defined
op_AddressOf | & (unary)
op_OnesComplement | ~
op_PointerDereference | * (unary)


Binary Operators

Binary operators take two arguments, perform some operation and return a value. They are represented as static methods on the class that defines the type of one of their two operands. The following table shows the names that are defined.

Name | C++ Operator Symbol
op_Addition | + (binary)
op_Subtraction | - (binary)
op_Multiply | * (binary)
op_Division | /
op_Modulus | %
op_ExclusiveOr | ^
op_BitwiseAnd | & (binary)
op_BitwiseOr | |
op_LogicalAnd | &&
op_LogicalOr | ||
op_Assign | =
op_LeftShift | <<
op_RightShift | >>
op_SignedRightShift | Not defined
op_UnsignedRightShift | Not defined
op_Equality | ==
op_GreaterThan | >
op_LessThan | <
op_Inequality | !=
op_GreaterThanOrEqual | >=
op_LessThanOrEqual | <=
op_UnsignedRightShiftAssignment | Not defined
op_MemberSelection | ->
op_RightShiftAssignment | >>=
op_MultiplicationAssignment | *=
OpPointerToMemberSelection | ->*
op_SubtractionAssignment | -=
op_ExclusiveOrAssignment | ^=
op_LeftShiftAssignment | <<=
op_ModulusAssignment | %=
op_AdditionAssignment | +=
op_BitwiseAndAssignment | &=
op_BitwiseOrAssignment | |=
op_Comma | ,
op_DivisionAssignment | /=


Conversion Operators

Conversion operators are unary operations that allow conversion from one type to another. The operator method must be defined as a static method on either the operand or return type. There are two types of conversions:

  • An implicit (widening) coercion must not lose any magnitude or precision. These should be provided using a method named op_Implicit
  • An explicit (narrowing) coercion may lose magnitude or precision. These should be provided using a method named op_Explicit

Conversions provide functionality that can’t be generated in other ways, and many languages will not support the use of the conversion operators through special syntax. Therefore, we recommend that the same functionality be made available using the more common ToXxx and FromXxx naming pattern.

Because these operations may exist on the class of their operand type (so-called “from” conversions) and would therefore differ on their return type only, the CLS specifically allows that these two operators be overloaded based on their return type. The CLS, however, also requires that if this form of overloading is used then the language must provide an alternate means for providing the same functionality since not all CLS languages will implement operators with special syntax.

CLS Rule 38: If either op_Implicit or op_Explicit is overloaded on its return type, an alternate means of providing the coercion must be provided.

CLS Rule 39: The functionality provided by op_Implicit or op_Explicit should also be provided through the FromXxx and ToXxx naming pattern.

CLS (consumer): Where appropriate to the language design, use the existence of op_Implicit and/or op_Explicit in choosing method overloads and generating automatic coercions.

CLS (extender): Where appropriate to the language design allow the definition of implicit or explicit coercion operators to produce the corresponding op_Implicit, op_Explicit, ToXxx, and/or FromXxx methods. Otherwise, if possible, provide a means for explicitly defining those methods including the possibility of overloading op_Implicit and op_Explicit based on return type alone.

CLS (framework): Consider providing coercion operations. When this is done, always provide functionality via FromXxx and ToXxx and consider providing op_Implicit and op_Explicit as well.


Naming Patterns

While the VOS does not dictate the naming of properties or events, the CLS does specify a pattern to be observed.

For Events:

An individual event is created by choosing or defining a delegate type that is used to signal the event. Then, three methods are created with names based on the name of the event and with a fixed signature. For the examples below we define an event named Click that uses a delegate type named EventHandler.

EventAdd, used to add a handler for an event
Pattern: void add_<EventName> (<DelegateType> handler)
Example: void add_Click (EventHandler handler);
EventRemove, used to remove a handler for an event
Pattern: void remove_<EventName> (<DelegateType> handler)
Example: void remove_Click (EventHandler handler);
EventRaise, used to signal that an event has occurred
Pattern: void family raise_<EventName> (Event e)

For Properties:

An individual property is created by deciding on the type returned by its getter method and the types of the getter’s parameters (if any). Then, two methods are created with names based on the name of the property and these types. For the examples below we define two properties: Name takes no parameters and returns a System.String, while Item takes a System.Object parameter and returns a System.Object. Item is referred to as an indexed property, meaning that it takes parameters and thus may appear to the user as through it were an array with indices

PropertyGet, used to read the value of the property
Pattern: <PropType> get_<PropName> (<Indices>)
Example: System.String get_Name ();
Example: System.Object get_Item (System.Object key);
PropertySet, used to modify the value of the property
Pattern: void set_<PropName> (<Indices>, <PropType>)
Example: void set_Name (System.String name);
Example: void set_Item (System.Object key, System.Object value);


Collected CLS Rules

The complete set of CLS rules are collected here for reference. Recall that these rules apply only to “externally visible” items – types that are visible outside of their own assembly and members of those types that have public, family, or family-or-assembly accessibility. Furthermore, items may be explicitly marked as CLS-compliant or not using the System.CLSCompilantAttribute.

  1. Boxed value types are not part of the CLS. Instead, use System.Object, System.ValueType or System.Enum, as appropriate.
  2. Languages must follow Annex 7 of Technical Report 15 of the Unicode Standard 3.0 (ISBN 0-201-61633-5) governing canonicalization of identifiers, available on-line at http://www.unicode.org/unicode/reports/tr15/tr15-18.html. While the CLS allows the use of case-insensitive comparison (canonicalization KC), identifiers must be persisted in their original case (canonicalization C) so that case sensitive comparisons work as expected.
  3. Languages must provide an escape mechanism to reference identifiers that happen to be keywords in their own language. In addition, extender languages must provide a mechanism for defining and overriding virtual methods with names that are keywords in their language. In order to override an inherited definition the VOS requires the precise encoding of the original declaration be used.
  4. All names introduced in a scope shall be distinct independent of kind. That is, while the VOS allows a single type to use the same name for a method and a field, the CLS does not.
  5. Fields and nested types must be distinct by identifier comparison alone, even though the VOS allows distinct signatures to be distinguished. Methods, properties, and events that have the same name (by identifier comparison) must differ by more than just the return type (although there is a special exception discussed in Conversion Operators).
  6. The underlying type of an enum must be a built-in CLS integer type.
  7. There are two distinct kinds of enums, indicated by the presence or absence of the System.FlagsAttribute custom attribute. One represents named integer values, the other named bit flags that can be combined together to generate an unnamed value. While languages are encouraged to distinguish these, they need not do so.
  8. Literal static fields of an enum must have the type of the enum itself.
  9. Accessibility must not be changed when overriding inherited methods, except when overriding a method inherited from a different assembly with accessibility family or assembly. In this case the override must have accessibility family.
  10. All types appearing in a signature must be CLS-compliant.
  11. The visibility and accessibility of types and members must be such that types in the signature of any member must be visible and accessible whenever the member itself is visible and accessible. For example, a public method that is visible outside its assembly must not have an argument whose type is visible only within the assembly.
  12. Typed references are not CLS-compliant.
  13. The varargs constraint is not part of the CLS, and the only calling convention supported by the CLS is the standard managed calling convention.
  14. Arrays elements must have a CLS-compliant type, a fixed number of dimensions, and all dimensions of the array must have zero lower bounds. While the abstract type System.Array is considered CLS-compliant it does have types that inherit from it but are not CLS-compliant.
  15. Pointer types are not CLS-compliant.
  16. CLS-compliant tools must deal with the fact that a single type may implement two interfaces and those interfaces may each require the definition of a method of the same name and signature. These methods are considered distinct and need not have the same implementation.
  17. CLS-compliant interfaces do not define static or instance methods, nor do they define fields.
  18. CLS-compliant classes, boxed value types, and interfaces must not require the implementation of non-CLS-compliant interfaces.
  19. An object constructor must call a base class constructor before any access occurs to inherited instance data.
  20. An object constructor shall not be called except as part of the creation of an object, and no object should be initialized twice.
  21. A CLS-compliant class must inherit from a CLS-compliant class. System.Object is CLS-compliant.
  22. The CLS restricts boxed value types to inherit directly from System.ValueType unless they are enumerations, in which case they must inherit directly from System.Enum.
  23. The methods that implement the getter and setter methods of a property must be marked mdSpecialName in the metadata.
  24. The accessibility of the property and of its accessors must be identical.
  25. The property and its accessors must all be static, all be virtual, or all be instance.
  26. The type of the property must match the return type of the getter and the last argument of the setter. The types of the parameters of the property must match the types of the parameters to the getter and the types of all but the final parameter of the setter. All of these types must be CLS-compliant, and must not be managed pointers (i.e. cannot be passed by reference).
  27. Properties must adhere to a specific naming pattern. See Naming Patterns.
  28. The methods that implement the event must be marked mdSpecialName in the metadata.
  29. The add and remove methods for an event must both either be present or absent.
  30. Events must adhere to a specific naming pattern. See Naming Patterns.
  31. CLS-compliant types must not use explicitlayout.
  32. CLS-compliant tools are only required to deal with a subset of the encodings of custom attributes. The only types that can appear in these encodings are: System.Type, System.String, System.Char, System.Boolean, System.Byte, System.Int16, System.Int32, System.Int64, System.Single, System.Double, and any enumeration type based on a CLS-compliant base type.
  33. CLS-compliant tools shall not generate publicly visible required modifiers, and shall ignore optional modifiers they do not understand. They need not emit either kind of modifier, although they must be able to copy both kinds of modifiers should they exist in metadata that they import.
  34. Global static fields and methods are not CLS-compliant.
  35. Types whose CLS-compliance differs from that of the assembly in which they are defined must be marked with the System.CLSCompliantAttribute. Similarly, members whose CLS-compliance differs from that of their class must also be so marked.
  36. Fields and events may not be overloaded within a given class.
  37. Properties, instance methods, and virtual methods may be overloaded based only on the types of their parameters, except the methods named op_Implicit and op_Explicit which may also be overloaded based on their return type.
  38. If either op_Implicit or op_Explicit is overloaded on its return type, an alternate means of providing the coercion must be provided.
  39. The functionality provided by op_Implicit or op_Explicit should also be provided through the FromXxx and ToXxx naming pattern.
  40. Exception objects must derive from System.Exception


VES: The Virtual Execution System

NGWS runtime provides an Execution Engine (EE) that creates the Virtual Execution System (VES) needed to implement the Virtual Object System. See the EE Architecture Specification for details of the runtime execution model as seen from a programmer's perspective.

The important components of the VES are:

  • An Intermediate Language (IL) that can be easily and effectively targeted by a wide range of compilers. See Intermediate Language (IL)
  • Loading managed code (i.e., code written in accordance to VES requirements) including resolving names, performing class layout, and creating stubs needed to transition between managed and unmanaged code. See Loading Managed Code.
  • Conversion of IL into native code, including when a method is first invoked (JIT compilation). See Conversion of IL into Native Code.
  • Verification of the type safety of methods written in IL as well as the integrity of the metadata describing those methods. See Verification of Implementation Code.
  • Services based on stack format: garbage collection of memory containing managed data, and initiation, propagation, and intercepting exceptions. See Services Based on Stack Format.
  • Insertion and management of security tests based on code identity, type safety, and administrative (trust management) controls. See Security Services.
  • Profiling and debugging services, including mapping native code instruction addresses back to IL addresses and lengthening the lifetime of local variables. See Profiling and Debugging Services.
  • Management of threads, contexts, and remoting. See Delegates
  • The VES provides the implementation for a set of types known as delegates. Delegates are the object-oriented equivalent of function pointers. Unlike function pointers, delegates are object-oriented, type-safe, and secure. Delegates are created by defining a class that derives from the base type System.Delegate. Each particular type of delegate provides a method named Invoke with appropriate parameters, and each instance of a delegate forwards calls to its Invoke method to a compatible static or instance method on a particular object. The object and method to which it delegates are chosen when the delegate instance is created.
  • In addition to an instance constructor and an Invoke method, delegates may optionally have two additional methods: BeginInvoke and EndInvoke. These are used for asynchronous calls and are described in a separate specification.
  • While, for the most part, delegates appear to be simply another kind of user defined class, they are tightly controlled. The implementations of the methods are provided by the VES, not user code. Also, the user can specify no additional members; the definition of a delegate is really no more than the signature of the method to which it forwards its calls.
  • Proxies, Contexts, and Remoting.


Intermediate Language (IL)

The Intermediate Language is intended to be easily targeted by compilers for a wide range of common programming languages and to be implemented by a range of IL-to-native code compilers, including Just-In-Time (JIT) compilers. It is the job of the JITter to convert from this Intermediate Language and execution model into the native instructions, calling convention, and execution model of a physical computer (see Conversion of IL into Native Code).

The virtual execution model consists of an evaluation stack used for pushing arguments to methods and built-in operations. At a method call the arguments are transferred from the control of the calling method to the control of the called method. Returned values of any instantiable type are pushed on the stack before method return. Instance and virtual methods are called by passing the this pointer as a “hidden” first parameter to the called method.

The IL instruction set can be divided into the following categories of instructions. See the IL Instruction Set specification for details

  • Arithmetic and Logical Operations. These take their operands off the evaluation stack and store the result back there. Arithmetic includes computation, comparison, and coercion. Computations can be performed with or without overflow detection. These operations implement the fundamental operations on the built-in VOS data types.
  • Control Flow.
    • conditional and unconditional branches within a method
    • jump table to known destinations within method (case dispatch)
    • calls and returns between methods (with fixed destination, computed destination, or instance-based destination, all with or without tail-call optimization)
    • throwing and catching an exception
    • breakpoint and no-op
  • Direct memory access. These include the ability to clear, copy, and load onto the stack arbitrary portions of memory. It is also possible to compute the address of static data, and load or store a built-in data type or Reference Type indirectly through a pointer.
  • Stack manipulation. The contents of the top of the stack can be duplicated or removed (popped), and constants (numbers, the null reference value, and strings) can be pushed onto the stack.
  • Argument and local variables. The values of arguments and local variables can be loaded onto the stack, the top of the stack can be stored into them, and their address can be loaded onto the stack. There is also support for typesafe reference to a variable number of input arguments.
  • Stack allocation. Memory can be allocated in the currently executing stack frame (alloca).
  • Object model. The IL directly supports the box and unbox operations for converting Value Types to and from their corresponding Reference Type. It also supports assignment compatibility testing.
  • Values of Instantiable Types. There are specific IL instructions to create and copy values of instantiable types (both Value Types and Reference Types) as well as to load them on the stack and initialize their bit sequences to a known initial state. There are instructions to access and modify member fields (static or instance) of values, as well as to compute the addresses of member fields.
  • Critical region. Every object (the value of a self-describing Reference Type) can be used as a synchronization point for methods and critical regions. Methods on objects can be marked synchronized to guarantee mutually exclusive access to their implementations. Static methods on a class or interface can also be marked synchronized and they use the class or interface itself as the synchronization point.
  • Arrays. There are IL instructions to deal with one-dimensional, zero-indexed arrays (vectors). These are used to create such arrays, extract their length, load or store individual elements, and calculate the address of an individual element. All arrays (not just vectors) maintain the information about their rank, bounds for each dimension, and underlying data type as part of their runtime description. Any attempt to store into the array or compute the address of an element of an array checks the bounds and the type at runtime (co-variant arrays).
  • Typed locations. The IL has instructions to create typed locations (used to implement the typed reference parameter passing convention of the VOS), load them on the stack, access their address part in a typesafe manner, and access their type part.


Loading Managed Code

The VES provides a class loader whose job is to load the implementation of a loadable type into memory and prepare it for execution. Implementations of types are supplied in assemblies, and the types are always referenced through their assembly. The class loader uses the name of the assembly, version information captured when the type reference was compiled, information about the application that is attempting to load the type, and administrative configuration to locate the assembly and its constituent PE files. A single PE File (see the File Format Specification) may contain the implementation of many types, but the implementation of a single type cannot span multiple PE files.

The class loader ensures the following:

  • For instantiable types, the amount of space required to hold an instance is known. For abstract types, the amount of space its subclasses must allocate for its use is known.
  • The “shape” of the space is known. That is, the offsets of each field and method in the appropriate data structures and the type of each.
  • Any references made by the loaded type to types that have already been loaded are resolved. That is, they have been checked for consistent usage and converted into a form that directly references the required information.
  • Any references that were made to the loaded type by types already loaded into the system are checked for consistency.
  • Any references made by the loaded type to types that have not yet been loaded are either
    • loaded and resolved immediately, or
    • recorded so that appropriate consistency checks will occur when the type is loaded
  • Stubs are created for all method implementations that are loaded. These stubs are used (at least) on the first call to the method. For methods implemented in IL the stub will call the JIT to generate native code and then update the stub. The stubs used by managed code to call unmanaged code (and vice versa) include an appropriate transition function.

The class loader is also part of the NGWS runtime security enforcement mechanism. It performs a variety of consistency checks including enforcement of certain accessibility rules (e.g., a child cannot override a final virtual method) and self-consistency requirements (a value class cannot contain an instance of itself as a field). This amounts to a minimal, mandatory, verification process that precedes the more complex (and optional) IL code verification process (see Verification of Implementation Code).

Implementations of classes and interfaces can include initialization code that must be run once before any active reference (such as creating an instance, calling a method, or referencing a field of the class) is made to the class. Implementations can be marked to allow the initialization to occur when the implementation is loaded or be deferred until the first active reference.


Conversion of IL into Native Code

The IL instruction set is designed as a target for compilers, but it is not designed as a traditional interpreted byte code or tree code. Converting IL to native code requires some per-method analysis, so it is truly a compiler intermediate language. A compiler back-end that accepts IL as input and produces native code is called, for historical reasons, a Just-In-Time Compiler or JITter. There is a standard interface between a JITter and the EE.

NGWS runtime will ship with three different classes of JITters (although all JITters may not exist for all architectures):

  • JIT, which is an optimizing compiler back end. The input to JIT is unrestricted sets of IL instructions. It performs a data flow analysis and produces high quality output code. It requires significant resources (memory footprint, working set, and time) to compile IL to native code.
  • EconoJIT, which is designed for high speed conversion of IL to native code, ease of porting to new architectures, and caching of the generated native code. The output code need not be of high quality. The code generation strategy makes it possible to discard the native code as desired, so that even large IL programs can be converted to native code and fit in a fixed-size code memory area.
  • PreJIT is based on the main JIT, but runs in a more traditional compilation model rather than just-in-time. It is used at installation time, by converting an entire assembly into managed native code. The resulting file loads and starts more quickly than code produced just-in-time.


Verification of Implementation Code

The Virtual Execution System is responsible for ensuring its security, and the security model (see the Security Specification) is built on a notion of trust levels. For some trust levels the security model requires that the code be proven typesafe. This is implemented by the VES using a verification algorithm that can examine the IL code for a method, perform a control flow analysis that uses the information from type signatures, and show that:

  1. The code is typesafe if the type signatures are correct.
  2. The implementation maintains the correctness of the type signatures.

A perfect verification algorithm (one that declares an implementation verified if and only if it is typesafe) is provably impossible to implement. Instead, we provide a conservative verification algorithm that runs efficiently and only declares an implementation verified if it is typesafe. By observing a set of rules, a compiler can ensure that the IL code it generates will be verified. Not all languages will follow these rules, however, since the language semantics may require more sophisticated proof techniques than are implemented in the verification process to show the type safety of the IL they generate.

Because a verification algorithm is built into the NGWS runtime JIT compiler, it must run on a method-by-method basis. This process cannot prove the third property that is required to ensure type safety of programs:

  1. All incoming values have the indicated type signature.

This property is ensured by a combination of the class loader, verification process, and the NGWS frameworks. The loader ensures that type signatures are consistent between the use and implementation of every method and field. The verification process ensures that this consistency is maintained within verified code. And the NGWS frameworks (which are not subject to verification at runtime) also preserve the same consistency.

As a pragmatic matter, not all code will be verified, so the VES by itself cannot guarantee type safety.


Services Based on Stack Format

The primary requirement for managed code is that it is possible to trace the stack at runtime. Different source languages may make different decisions about the layout of stack frames under their control, so the runtime uses a specific API, the code manager, to understand the format of individual stack frames. The code manager is specified either by the JITter or the source language compiler (persisted in the PE file format) that produces each native managed code method.

The code manager is used for four separate services:

  • Garbage Collection. Any source language that allows access to VOS objects is required to have a code manager that can report the locations in its stack frame that contain object references. The garbage collector supports both pointers to the start of an object or interior pointers (also known as managed pointers) that point inside the representation of an object. The garbage collector runs in two phases: the first to find all objects still active in the system and the second to copy those objects and modify references to them to point to their new locations. It uses the code manager both to locate the references on the stack and to arrange for execution to be suspended (“hijacking”) while the pointers are updated.
  • Exceptions. The VES supports an exception mechanism that is compatible with SEH (Win32 Structured Exception Handling). It uses the code manager to determine which exception handlers or filters are active in each stack frame. The code manager is free to use any mechanism it likes for this purpose. The code manager for the existing JITters uses an exception handler table that is part of the method header for this purpose.

CLS Rule 40: Exception objects must derive from System.Exception

CLS (consumer): Must allow exceptions derived from System.Exception to be caught and thrown.

CLS (extender): Must allow new exceptions to be defined by deriving them from System.Exception.

CLS (framework): Should, where reasonable, throw exceptions derived from System.Exception to signal error conditions, rather than returning status flags or success codes. Must not throw any value not derived from System.Exception.

  • Security. The implementation of code access security uses the code manager to examine the stack for special security markers that indicate permissions granted to executing procedures. These markers may be allocated on the stack by a JITter and are only accessible through the code manager.
  • Debugging and Profiling. Both debugging and profiling allow external processes to examine and modify the stack of a NGWS runtime process using the code manager.


Security Services

The VES is responsible for ensuring that all security checks are enforced, as described in the Overview of NGWS Runtime Security specification. Security is integral to the design of the NGWS runtime, and many parts of the VES include security-relevant parts. Overall, the VES is involved in the following areas.

Type Safety. Typesafe programs reference only memory that has been allocated for their use and access objects only through their exposed interfaces. From a security standpoint, the former allows multiple objects to safely share a single address space and the latter guarantees that security tests provided by those interfaces are not circumvented.
The VES provides type safety by combining strong typing in the metadata (parameters, members and array elements, return values of methods, static values) with strong typing in the IL (local variables and stack slots). This allows an efficient verification process to assure the type safety of many programs written in IL, independent of the language that produced them. The VES requires verification of code before it is run unless a specific (administratively controlled) security check is successfully performed.
Version Consistency. The VES, through the class loader and the assembly mechanism, is responsible for ensuring that the runtime environment meets assumptions built into code that is loaded. This is particularly important for early bound code, such as that generated by a native code compiler or through install-time JITting of IL. This is enforced by comparing version information gathered when the code was created with information available when the code is loaded.
Code Identity. There are only three ways for code to become executable within the VES: through the class loader, through Platform Invoke (legacy code), or through Unmanaged COM interop. All three of these are services provided by the VES and are part of the security perimeter. For example, the class loader maintains information about the source of every implementation that it loads. Therefore, the class loader is able to reliably provide some of the credentials required for validating code identity.
Accessibility. Application-specific security depends on enforcement of the accessibility rules that are part of a VOS type specification. These rely on the relationship between the implementation requesting access to a particular member or method of a type and the type itself. The class loader is responsible for connecting an implementation to the metadata about the type it implements. Therefore, the class loader can reliably specify whether an implementation requesting access to a type is: part of the implementation of that type; part of the implementation of a type that inherits from it; or from an unrelated type. In this last case, the class loader can also reliably specify an identifier for the type that is implemented by the code requesting access. This can be combined forwarded to the appropriate assemblies to determine whether access is granted.
Declarative Security. Metadata is capable of specifying permissions that are required in order to access methods, beyond the accessibility requirements of the type system. Calls to methods that have been marked as requiring permissions are intercepted at runtime (through stubs inserted by the linker, loader, or JITter) to guarantee that the caller possesses the correct permissions. This may involve walking the stack using the VES services.
Imperative Security. Imperative security is largely the responsibility of the application programmer. The VES supports imperative security by providing trusted methods to:
  • reliably access code identity information
  • locate permissions based on walking the stack
  • store permission information in the stack
  • access administrative information about security requirements


Profiling and Debugging Services

The VES provides the ability to both debug (observe and modify the behavior) and profile (measure resource utilization) of running programs. It does this by providing three underlying services, described in detail in the Debugging Specifications and the Profiling Specification:

  • Both profiling and debugging depend on information produced by the source language compiler. Typically this includes two kinds of information:
    • Maps from source language constructs to addresses in the instruction stream. These are used to translate between source code references (e.g. file name and line number) and addresses in the executing code.
    • Maps from addresses to locations in the stack frame. These are used to map between names (of parameters, local variables, or instance fields) and addresses on the stack.
If the source language compiler produces IL as its output, both of these maps must be recomputed based on the conversion of IL to native code. As with traditional compilation, the optimizations performed by a JIT compiler make it impossible to guarantee that the maps are maintained with 100% accuracy. For example, the JIT compiler may merge IL code sequences together so that a single native address corresponds to more than one source reference. It may also merge local variables with independent life times into a single stack frame slot (or, worse, split a variable so that it resides simultaneously in a register and on the stack).
The interface between the EE and the JIT compiler includes a mechanism for updating these maps. This can be used either to produce new maps that are persisted (for example, when a new PE file is created) or simply reported back to the VES-supplied debugging and profiling system (when JITting to memory for immediate execution).
  • An API for use by debuggers that handles registration for and notification of events in the running program. This includes the ability to:
    • Create a new process or enumerate existing processes and attach to one.
    • Set, remove, enumerate, activate, deactivate, and detect the occurrence of both data and execution breakpoints.
    • Detect the raising of both handled and unhandled exceptions.
    • Interrupt execution.
    • Walk the execution stacks (even in a multi-threaded application).
    • Locate, read, and modify instance fields, arguments, local variables and static data.
    • Read the executing code (IL or native).
    • Restart execution at a new address, single step program execution, or invoke methods or functions directly from the debugger.
    • Edit the program and continue execution (“Edit and Continue”), including the ability to read and modify the metadata associated with data and methods in the program.
    • Load and unload implementations (i.e. invoke the VES class loader) as well as intercept calls to the class loader that occur within the program.
  • An API for use by tools that do program profiling. The API supports profiling of managed native code (e.g. the output of a JITter) both with and without inserting specific profiling probes into the code.
Without modifying the code, the API provides tools information about:
  • Calls to the VES class loader or unloader.
  • Calls to any JIT compiler.
  • Calls to and returns from compiled methods or functions.
  • Metadata for the executing program.
  • Context crossings, exceptions, garbage collections, memory allocation on any garbage collected heap, security checks, object synchronization, thread creation and destruction.
By requesting modification of the executing code, the profiling tool can also obtain information about the dynamic context (chain of stack frames, executing thread, etc.) at entry or exit from a method or function.

Delegates

The VES provides the implementation for a set of types known as delegates. Delegates are the object-oriented equivalent of function pointers. Unlike function pointers, delegates are object-oriented, type-safe, and secure. Delegates are created by defining a class that derives from the base type System.Delegate. Each particular type of delegate provides a method named Invoke with appropriate parameters, and each instance of a delegate forwards calls to its Invoke method to a compatible static or instance method on a particular object. The object and method to which it delegates are chosen when the delegate instance is created.

In addition to an instance constructor and an Invoke method, delegates may optionally have two additional methods: BeginInvoke and EndInvoke. These are used for asynchronous calls and are described in a separate specification.

While, for the most part, delegates appear to be simply another kind of user defined class, they are tightly controlled. The implementations of the methods are provided by the VES, not user code. Also, the user can specify no additional members; the definition of a delegate is really no more than the signature of the method to which it forwards its calls.


Proxies, Contexts, and Remoting

A context is a grouping of objects that provides mutual isolation. In general, an object is created in a particular context and any attempt to access the object from another context is mediated by the context itself. That is, attempts to:

  • read the value of a (non-static) field of an object
  • modify the value of a (non-static) field of an object
  • invoke a (non-static) method

are mediated by both the context from which the attempt is made and the context in which the object exists.

A remoting boundary exists between contexts if it is not possible to share the identity of an object directly between the contexts. For example, if the contexts exist on physically separate machines that do not share a common address space, then a remoting boundary will exist between them. There are other administrative mechanisms for creating remoting boundaries. No single context can span a remoting boundary, hence crossing a remoting boundary always implies crossing a context boundary.

The VES provides a mechanism, called the application domain, to isolate applications running in the same operating system process from one another. Types loaded into one application domain are distinct from the same type loaded into another application domain, and instances of objects cannot be directly shared from one application domain to another. Hence, the application domain itself forms a remoting boundary.

The VES implements contexts and remoting boundaries based on the concept of a proxy. A proxy is an object that exists inside one context and represents an object in another context. The proxy forwards references to instance fields and methods to the actual object for interpretation. Proxies do not forward references to static fields or calls to static methods.

The NGWS runtime implementation of proxies is provided automatically for instances of types that derive from System.MarshalByRefObject. The Remoting Specifications provide information about how to control the behavior of the proxies, including how to specify where instances of the class should be created, what channel should be used to communicate between the proxy and the implementation, what serialization format should be used to pass method calls to the implementation, etc.

To create types whose instances can be created in different contexts, the type must derive from System.ContextBoundObject, which itself derives from System.MarshalByRefObject. The Context Specifications provide information about how to configure the proxies to pass information about the context in which the instance is to be created, the context properties that control the passing of information across context boundaries, and so forth.


Topic Not Found

No matching topic was found. The link may be out of date, incorrectly authored, or the topic may not be available in this release.

You may be able to find the topic by using the index or search tab instead.


Copyright

NGWS SDK Documentation

This is a preliminary document and may be changed substantially prior to final commercial release. This document is provided for informational purposes only and Microsoft makes no warranties, either express or implied, in this document. Information in this document, including URL and other Internet Web site references, is subject to change without notice. The entire risk of the use or the results of the use of this document remains with the user. Unless otherwise noted, the example companies, organizations, products, people and events depicted herein are fictitious and no association with any real company, organization, product, person or event is intended or should be inferred. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

Unpublished work. © 1999 - 2000 Microsoft Corporation. All rights reserved.

Microsoft, MS-DOS, Windows, Windows NT, Active Directory, ActiveX, Authenticode, FrontPage, IntelliSense, JScript , Outlook, Visual Basic, Visual Basic Scripting Edition (VBScript), Visual C++, Visual Studio, and Win32 are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries.

The names of actual companies and products mentioned herein may be the trademarks of their respective owners.