How MSMQ Authenticates Messages

MSMQ authenticates message at the request of the sending application. When the sending application indicates it wants a message authenticated, the MSMQ runtime code performs the following tasks:

Retrieves the internal certificate, external certificate certificate, or the security context information.
Note For applications using API functions, external certificates are provided in PROPID_M_SENDER_CERT and security context information is provided in PROPID_M_SECURITY_CONTEXT

For applications using ActiveX components, external certificates are provided in SenderCertificate and security context information is retrieved by AttachCurrentSecurityContext.
Extracts the public signing key from the internal certificate, external certificate, or from the security context information taken from the external certificate.
Extracts the matching private signing key. For internal certificates, MSMQ locates the private signing key internally. For external certificates, MSMQ searches for the private signing key in the Internet Explorer certificate store (this is why the external certificate must be registered in the certificate store).
Computes a hash value of the message using the algorithm specified by the sending application.
Note For applications using API functions, the Hash algorithm is specified by PROPID_M_HASH_ALG (the default algorithm is CALG_MD5)

For applications using ActiveX components, the Hash algorithm is specified by the MSMQMessage object's HashAlgorithm property (the default algorithm is CALG_MD5)

The following message properties (in the order shown here) are used to create the hash value:

API Functions	ActiveX Components
PROPID_M_CORRELATIONID	CorrelationId
PROPID_M_APPSPECIFIC	AppSpecific
PROPID_M_BODY	Body
PROPID_M_LABEL	Label
PROPID_M_RESP_QUEUE	ResponseQueueInfo
PROPID_M_ADMIN_QUEUE	AdminQueueInfo

Encrypts the hash value using your private signing key. This is the digital signature that will be attached to the message.
Attaches the certificate and digital signature to the message, then sends the message on to the target Queue Manager.

When the target Queue Manager receives the message, it performs the following tasks:

Computes the hash value of the message using the algorithm specified in PROPID_M_HASH_ALG or HashAlgorithm.
Extracts the public key from the certificate.
Decrypts the digital signature using the public key, obtaining the hash value sent with the message.
Compares the hash value computed by the Queue Manager to the hash value decoded from the digital signature.
If the hash values are the same, the queue then verifies the sender identifier, setting PROPID_M_AUTHENTICATED or IsAuthenticated to 1 if the sender identifier is valid.
The sender identifier stored with the certificate is retrieved from the MQIS and compared with the message's PROPID_M_SENDERID or SenderId property. (This is why the certificate should be registered with MSMQ.)
If the hash values are not the same, the message is discarded and a negative acknowledgment is returned to the sending application if one was requested.

Note MSMQ does not validate the external certificate. The receiving application performs any validation requirements on the certificate before using an authenticated message. MSMQ generates the digital signature of a message when it is sent and verifies the digital signature when the message is received, but does not validate the certificate itself.