Creating and Using IPsec Policies

Internet Protocol security (IPsec) is a framework of open standards for ensuring private, secure communications over IP networks through the use of cryptographic security services. The Microsoft Windows implementation of IPsec is based on standards developed by the Internet Engineering Task Force (IETF) IPsec working group.

IPsec establishes trust and security from a source IP address to a destination IP address. The only computers that must know about the traffic being secured are the sending and receiving computers. Each computer handles security at its respective end with the assumption that the medium over which the communication takes place is not secure. Computers that only route data from source to destination are not required to support IPsec unless firewall-type packet filtering or network address translation (NAT) is performed between the two computers.

You can use the IP Security Policy snap-in to create, edit, and assign IPsec policies on this computer and remote computers.

Note

This documentation is intended to provide enough information to understand and use the IP Security Policy snap-in. Information about designing and deploying policies is beyond the scope of this documentation.

About IPsec policies

IPsec policies are used to configure IPsec security services. The policies provide varying levels of protection for most traffic types in most existing networks. You can configure IPsec policies to meet the security requirements of a computer, organizational unit (OU), domain, site, or global enterprise. You can use the IP Security Policies snap-in provided in Windows Vista® and Windows Server® 2008 to define IPsec policies for computers through Group Policy objects (for domain members) or on the local computer or for remote computers.

Important

The IP Security Policy snap-in can be used to create IPsec policies that can be applied to computers running Windows Vista and Windows Server 2008, but this snap-in does not use new security algorithms and other new features available in Windows Vista and Windows Server 2008. To create IPsec polices for these computers, use the Windows Firewall with Advanced Security snap-in. The Windows Firewall with Advanced Security snap-in does not create policies that can be applied to earlier versions of Windows.

An IPsec policy consists of general IPsec policy settings and rules. General IPsec policy settings apply, regardless of which rules are configured. These settings determine the name of the policy, its description for administrative purposes, key exchange settings, and key exchange methods. One or more IPsec rules determine the types of traffic IPsec must examine, how traffic is treated, how to authenticate an IPsec peer, and other settings.

After the policies are created, they can be applied at the domain, site, OU, and local level. Only one policy can be active on a computer at one time. Policies distributed and applied using Group Policy objects override local policies.

IPsec Policy snap-in tasks

This section includes some of the most common tasks that you might perform using the IP Security Policies snap-in.

Creating a policy

Unless you are creating policies on only one computer and its IPsec peer, you will probably have to create a set of IPsec policies to fit your IT environment. The process of designing, creating, and deploying policies can be complex, depending on the size of your domain, the homogeneity of the computers in the domain, and other factors.

Typically, the process is as follows:

  1. Create IP filter lists that match the computers, subnets, and conditions in your environment.

  2. Create filter actions that correspond to how you want connections to be authenticated, data integrity to be applied, and data to be encrypted. The filter action can also be either Block or Permit, regardless of other criteria. The Block action takes priority over other actions.

  3. Create a set of policies that match the filtering and filter action (security) requirements you need.

  4. First, deploy policies that use Permit and Block filter actions and then monitor your IPsec environment for issues that might require the adjustment of these policies.

  5. Deploy the policies using the Negotiate Security filter action with the option to fall back to clear text communications. This allows you to test the operation of IPsec in your environment without disrupting communications.

  6. As soon as you have made any required refinements to the policies, remove the fall back to clear text communications action, where appropriate. This will cause the policies to require authentication and security before a connection can be created.

  7. Monitor the environment for communications that are not taking place, which might be indicated by a sudden increase in the Main Mode Negotiation Failures statistic.

To create a new IPsec policy

  1. Right-click the IP Security Policies node, and then click Create IP Security Policy.

  2. In the IP Security Policy Wizard, click Next.

  3. Type a name and a description (optional) of the policy, and then click Next.

  4. Either select the Activate the default response rule check box or leave it unselected, and then click Next.

    Note

    The default response rule can be used only for policies that are applied to Windows XP and Windows Server 2003 and earlier. Windows Vista and Windows Server 2008 cannot use the default response rule.

  5. If you are using the default response rule, select an authentication method, and then click Next.

    For more information about the default response rule, see IPsec Rules.

  6. Leave the Edit properties check box selected, and then click Next. You can add rules to the policy as needed.

Add or change a rule to a policy

To add a policy rule

  1. Right-click the IPsec policy, and then click Properties.

  2. If you want to create the rule in the property dialog box, clear the Use Add Wizard check box. To use the wizard, leave the check box selected. Click Add.

    The following instructions are for creating a rule using the dialog box.

  3. In the New Rule Properties dialog box, on the IP Filter List tab, select the appropriate filter list, or click Add to add a new filter list. If you have already created filter lists, they will appear in the IP Filter Lists list.

    For more information about creating and using filter lists, see Filter Lists.

    Note

    Only one filter list can be used per rule.

  4. On the Filter Action tab, select the appropriate filter action, or click Add to add a new filter action.

    For more information about creating and using filter actions, see Filter Actions.

    Note

    Only one filter action can be used per rule.

  5. On the Authentication Methods tab, select the appropriate method, or click Add to add a new method.

    For more information about creating and using authentication methods, see IPsec Authentication.

    Note

    You can use several methods per rule. The methods are attempted in the order in which they appear in the list. If you specify that certificates are used, put them together in the list in the order you want them to be used.

  6. On the Connection Type tab, select the connection type to which the rule applies.

    For more information about connection types, see IPsec Connection Type

  7. If you are using a tunnel, specify the endpoints on the Tunnel Settings tab. By default, no tunnel is used.

    For more information about using tunnels, see IPsec Tunnel Settings. Tunnel rules cannot be mirrored.

  8. When all the settings are complete, click OK.
To change a policy rule

  1. Right-click the IPsec policy, and then click Properties.

  2. In the Policy Properties dialog box, select the rule, and then click Edit.

  3. In the Edit Rule Properties dialog box, on the IP Filter List tab, select the appropriate filter list, or click Add to add a new filter list.

    For more information about creating and using filter lists, see Filter Lists.

    Note

    Only one filter list can be used per rule.

  4. On the Filter Action tab, select the appropriate filter action, or click Add to add a new filter list.

    For more information about creating and using filter actions, see Filter Actions.

    Note

    Only one filter action can be used per rule.

  5. On the Authentication Methods tab, select the appropriate method or click Add to add a new method.

    For more information about creating and using authentication methods, see IPsec Authentication.

    Note

    You can use several methods per rule. The methods are attempted in the order in which they appear in the list.

  6. On the Connection Type tab, select the connection type to which the rule applies.

    For more information about connection types, see IPsec Connection Type.

  7. If you are using a tunnel, specify the endpoints on the Tunnel Settings tab. By default, no tunnel is used.

    For more information about using tunnels, see IPsec Tunnel Settings.

  8. When all the settings are complete, click OK.

Assigning a policy

To assign a policy to this computer

  1. Right-click the policy, and then click Assign.

    Notes
    • Only one policy can be assigned to a computer at a time. Assigning another policy will automatically unassign the currently assigned policy. Group Policy on your domain might assign another policy to this computer and ignore the local policy.
    • For a computer-to-computer IPsec policy to be successful, you must create a mirrored policy on the other computer and assign that policy to that computer.
    • To assign this policy to many computers, use Group Policy.

See Also

  • IPsec Authentication
  • IPsec Connection Type
  • IPsec Tunnel Settings
  • Filter Actions
  • Filter Lists
  • IPsec Rules


Filter Lists

Except in the case of a block or allow rule, an IP filter list triggers security negotiations based on a match with the source, destination, and type of IP traffic. This type of IP packet filtering enables an administrator to precisely define which IP traffic is secured. Each IP filter list contains one or more filters, which define IP addresses and traffic types. One IP filter list can be used for multiple communication scenarios.

Internet Protocol security (IPsec) requires both an inbound filter and outbound filter between the computers specified in the filter list, except for Block or Permit rules. Inbound filters apply to incoming traffic and enable the receiving computer to respond to requests for secure communication or match traffic against the IP filter list. Outbound filters apply to traffic leaving a computer and conduct a security negotiation before traffic is sent.

By using the Mirrored check box, you automatically create two filters based on the filter settings: one for traffic to the destination and one for traffic from the destination. This allows two-way communications with other computers.

Filter list settings

Filter lists (and filter actions) can be defined when you create a policy or before you create a policy. Filter lists are available to any policy. To define a filter list, right-click the IP Security Policy node and select Manage IP filter lists and filter actions.

Each filter defines a subset of inbound or outbound network traffic that the filter action acts on by either securing traffic (using authentication, data integrity, or data encryption), blocking entirely, or allowing (without using authentication, data integrity, or data encryption). You must have a filter to cover any traffic to which the associated rule applies. A filter contains the following settings:

  • The source and destination addresses of the IP packet. You can configure any IP address assigned to the IPsec peer, a single IP address, IP addresses by DNS name, or groups of addresses to specify IP subnets.

  • The protocol over which the packet is being transferred. This automatically covers all protocols in the TCP/IP protocol suite. However, it can be configured for an individual protocol, including a custom protocol, to meet special requirements.

  • The source and destination ports of the protocol for TCP and UDP. By default, all TCP and UDP ports are covered, but this can be configured to apply to a specific TCP or UDP port only.

Important

DNS name resolution occurs only when the filter list is created and is not updated afterwards. So, if the IP address changes, the policy will not be updated. To update the IP address, you must edit the policy.

To create a filter list using the New Rule Properties dialog box

  1. In the IP Security Policy Properties dialog, select the correct IP Security rule and click Edit or you can create a new rule by clicking Add.

  2. On the IP Filter List tab, clear the Use Add Wizard check box if you want to create the filter list in the property dialog box. If you want to use the wizard, leave the check box selected. Click Add.

    The following instructions are for creating a filter list using the dialog box.

  3. On the Addresses tab of the IP Filter Properties dialog box, select a source (local) IP address and a destination (that is, an IPsec peer) IP address.

  4. On the Protocol tab, select the protocol type that the filter will match.

  5. (Optional) On the Description tab, type a description of the filter. This description can help you sort through filters and allows you to quickly identify the filter without having to open its properties.

  6. Click OK.

  7. Repeat steps 4 through 8 to add filters to the list.

  8. In the IP Filter List dialog box, type a descriptive name for the filter list. Click OK to add the filter list to the rule.

  9. In the New Rule Properties dialog box, select the filter list.
To create a filter list using the Manage filter lists and filter actions dialog box

  1. Right-click the IP Security Policy node and select Manage IP filter lists and filter actions.

  2. On the Manage IP Filter Lists tab, click Add.

  3. In the IP Filter List dialog box, clear the Use Add Wizard check box if you want to create the filter list in the property dialog box. If you want to use the wizard, leave the check box selected. Click Add.

    The following instructions are for creating a filter list using the dialog box.

  4. On the Addresses tab of the IP Filter Properties dialog box, select a source (local) IP address and a destination (that is, an IPsec peer) IP address.

  5. On the Protocol tab, select the protocol type that the filter will match.

  6. (Optional) On the Description tab, type a description of the filter. This description can help you sort through filters and allows you to quickly identify the filter without having to open its properties.

  7. Click OK.

  8. Repeat steps 4 through 8 to add filters to the list.

  9. In the IP Filter List dialog box, type a descriptive name for the filter list. Click OK to add the filter list to the rule.

See Also

  • Filter Actions


Filter Actions

A filter action defines the security requirements for the data transmission. Filter actions can be defined when you create a policy or before you create a policy. Filter lists are available to any policy. To define a filter list, right-click the IP Security Policy node and select Manage filter lists and filter actions.

A filter action can be configured to:

Permit traffic

Internet Protocol security (IPsec) passes this traffic to and from the TCP/IP driver without modification or the requirement for security. This is appropriate for traffic from computers that are not IPsec-capable. Be sure to limit the IP filter list to a minimal scope when using this type of filter action, so that you do not permit traffic that should be secured.

In some cases, you might temporarily allow ICMP traffic for troubleshooting purposes. In other cases, you might need to allow a computer that is not in your domain (for example, a consultant's computer) access to another computer in your domain. You can use the permit filter action to allow this access.

Important

The permit filter action allows access without authentication, data integrity, or encryption. Anyone using a computer with the IP address specified in the filter list is given the access. All traffic between the computers is done in plaintext; no integrity checks are performed.

Block traffic

IPsec silently discards blocked traffic. When using a blocking filter action, be sure to use an IP filter list that defines the correct scope of IP addresses. Using larger scopes increases the chance of blocking traffic between valid computers.

Negotiate security

If you enable the Accept unsecured communication, but always respond using IPSec option, IPsec negotiates security associations (SAs) and the sending or receiving of IPsec-protected traffic. However, if the peer cannot use IPsec, the communication will be allowed without IPsec protection. After you choose this filter action, you can also configure the following:

  • Security methods and their order. This list of methods defines in which order the methods will be attempted. The first successful method will be used and the remaining methods will not be attempted. Typically, the list should be ordered from highest security to lowest security, so that the most secure method is used.

  • Acceptance of initial incoming unsecured traffic (Accept unsecured communication, but always respond using IPSec). IPsec allows an incoming packet that matches the configured filter list to be unsecured (that is, not protected by IPsec). However, the outgoing response to the incoming packet must be protected. This setting is useful when you are using the default response rule for clients. When a group of servers are configured with a rule that secures communications with any IP address and accepts unsecured communication, responding with only secured communications, enabling the default response rule on client computers ensures that the clients will respond to the server request to negotiate security. To prevent denial-of-service attacks, this option should be disabled for secure computers connected to the Internet.

  • Enabling of communication with non-IPsec-enabled computers (Allow unsecured communication if a secure connection cannot be established). IPsec falls back to unsecured communication, if necessary. Again, you should limit the IP filter list to a minimal scope. Otherwise, if negotiation fails for any reason, any communications affected by the rule in which this filter action resides could result in data being sent without security. If you are concerned about unsecured communication, you might consider disabling these settings. However, communication with computers that cannot initiate IPsec, such as legacy systems, might be blocked. To prevent denial-of-service attacks, this option should be disabled for secure computers connected to the Internet.

  • Generation of session keys from new keying material (Session key perfect forward secrecy (PFS)). Enabling session key PFS ensures that master key keying material cannot be used to derive more than one session key. When session key PFS is enabled, a new Diffie-Hellman key exchange is performed to generate new master key keying material before the new session key is created. Session key PFS does not require Main Mode reauthentication and uses fewer resources than master key PFS.

IPsec security methods

Each security method defines the security requirements of any communications to which the associated rule applies. Creating multiple security methods increases the chance that a common method can be found between two computers. The Internet Key Exchange (IKE) component reads the list of security methods in descending order and sends a list of allowed security methods to the other peer. The first method in common is selected. Typically, the most secure methods appear at the top of the list; the least secure methods appear at the bottom of the list.

Predefined security methods

The following security methods are predefined:

Encryption and Integrity

Uses the ESP protocol to provide data confidentiality (encryption) with the triple Data Encryption Standard (3DES) algorithm, data integrity and authentication with the Secure Hash Algorithm 1 (SHA1) integrity algorithm, and default key lifetimes (100 megabytes (MB), 1 hour). If you require both data and addressing (IP header) protection, you can create a custom security method. If you do not require encryption, use Integrity only.

Integrity only

Uses the ESP protocol to provide data integrity and authentication with the SHA1 integrity algorithm and default key lifetimes (100 MB, 1 hour). In this configuration, ESP does not provide data confidentiality (encryption).

Custom security methods

If the predefined Encryption and Integrity or Integrity only settings do not meet your security requirements, you can specify custom security methods. For example, you can use custom methods when encryption and address integrity, stronger algorithms, or key lifetimes must be specified. When configuring a custom security method, you can configure the following:
Security protocols

Both AH (Data and address integrity without encryption) and ESP (Data integrity and encryption) can be enabled in a custom security method when you require IP header integrity and data encryption. If you chose to enable both, you do not need to specify an integrity algorithm for ESP.

Note

The AH protocol cannot be used over network address translation (NAT) devices because it uses a hash of the header. NAT devices alter the header, so the packet will not authenticate properly.

Integrity algorithm

Message Digest 5 (MD5), which uses a 128-bit key. This algorithm is no longer considered secure and should be used only when interoperability requires its use.

SHA1, which uses a 160-bit key. SHA1 is a stronger hash than MD5 and is compliant with the Federal Information Processing Standard (FIPS).

Encryption algorithm

3DES is the most secure of the DES combinations and somewhat slower in performance. 3DES processes each block three times, using three unique 56-bit keys.

DES uses a single 56-bit key and is used when the higher security and overhead of 3DES is not required. This algorithm is no longer considered secure and should only be used when interoperability requires it.

Session key settings determine when, not how, a new key is generated. You can specify a lifetime in kilobytes (KB), seconds, or both. For example, if the communication takes 10,000 seconds and you specify the key lifetime as 1000 seconds, 10 keys will be generated to complete the transfer. This ensures that, even if an attacker manages to determine one session key and decipher part of a communication, the attacker cannot decipher the entire communication. By default, new session keys are generated for every 100 MB of data or every hour. Any time a key lifetime is reached, the SA, in addition to the key refresh or regeneration, is also renegotiated.

To create a filter action using the New Rule Properties dialog box

  1. On the Rules tab of the IP Security Policy Properties dialog, clear the Use Add Wizard check box if you want to create the filter action in the property dialog box. If you want to use the wizard, leave the check box selected. Click Add.

    The following instructions are for creating a filter list using the dialog box.

  2. On the Filter Action tab of the Rule Properties dialog box, clear the Use Add Wizard checkbox and click Add.

  3. On the Security Methods tab, select the method (action) that the rule will use.

  4. (Optional) On the Description tab, type a description of the filter action. This description can help you sort through filter actions and allows you to quickly identify the filter action without having to open its properties.

  5. Click OK.

  6. Repeat steps 4 through 8 to add filter actions to the list.

    Note

    Although the rule can list several filter actions, only one filter action can be used per rule.

  7. On the Filter Action tab, select the appropriate filter action for the rule and click OK.
To create a filter action using the Manage filter lists and filter actions dialog box

  1. Right-click the IP Security Policy node and select Manage IP filter lists and filter actions.

  2. On the Manage Filter Actions tab, clear the Use Add Wizard check box if you want to create the filter action using the property dialog box. If you want to use the wizard, leave the check box selected. Click Add.

    The following instructions are for creating a filter list using the dialog box. The directions below do not use the wizard.

  3. On the Security Methods tab, select the method and click OK.

  4. If you selected the Negotiate security option, you can add multiple methods and specify the order they will be attempted. To do this click Add.

  5. (Optional) On the Description tab, type a description of the filter. This description can help you sort through filters and allows you to quickly identify the filter without having to open its properties.

  6. Click OK.

  7. Repeat steps 4 through 8 to add filter actions to the list.

See Also

  • Filter Lists
  • IPsec Authentication


IPsec Authentication

Each rule defines a list of authentication methods. Each authentication method defines the requirements for the way in which identities are verified in communications to which the associated rule applies. The methods are attempted by each peer in the order in which they are listed. The two peers must have at least one common authentication method or communication will fail. Creating multiple authentication methods increases the chance that a common method between two computers can be found.

Note

The order of these methods is also important because only the first common method is attempted; if it fails to authenticate, no other methods in the list will be attempted, even if these methods would have succeeded.

Authentication methods

Only one authentication method can be used between a pair of computers, regardless of how many are configured. If you have multiple rules that apply to the same pair of computers, you must configure the authentication methods list in those rules to enable the pair to use the same method. For example, if a rule between a pair of computers specifies only Kerberos for authentication and filters only TCP data and, in another rule, specifies only certificates for authentication and filters only UDP data, authentication will fail. Authentication methods are configured on the Authentication Methods tab of the Edit Rule Properties or Add Rule Properties property sheets.

  • The Kerberos version 5 authentication protocol is the default authentication technology. This method can be used for any computers running the Kerberos V5 authentication protocol that are members of the same or trusted domains. This method is useful for domain isolation using Internet Protocol security (IPsec).

  • A public key certificate should be used in situations that include Internet access, remote access to corporate resources, external business partner communications, or computers that do not run the Kerberos V5 authentication protocol. This requires that at least one trusted certification authority (CA) has been configured. Windows Server® 2008 supports X.509 Version 3 certificates, including CA certificates generated by commercial certifying authorities.

  • A preshared key can be specified. This is a shared, secret key that is previously agreed upon by two users. It is simple to use and does not require the client to run the Kerberos V5 authentication protocol or have a public key certificate. Both parties must manually configure IPsec to use this preshared key. This is a simple method for authenticating standalone computers or any computers that are not using the Kerberos V5 authentication protocol. A preshared key is for authentication protection only; it is not used for data integrity or encryption.

Important

The preshared key is stored in plaintext and is not considered a secure method. Preshared keys should be used for testing purposes only.

See Also

  • Filter Actions
  • IPsec Rules


IPsec Tunnel Settings

Tunneling

Internet Protocol security (IPsec) can perform Layer 3 tunneling for scenarios in which Layer Two Tunneling Protocol (L2TP) cannot be used. If you are using L2TP for remote communications, no tunnel configuration is required because the client and server virtual private networking (VPN) components of Windows Vista® and Windows Server® 2008 create the rules to secure L2TP traffic automatically.

To create a Layer 3 tunnel using IPsec, use the IP Security Policies or Group Policy snap-ins to configure and enable the following two rules for the policy:

  1. A rule for the outbound traffic for the tunnel. The rule for the outbound traffic is configured with both a filter list that describes the traffic to be sent across the tunnel and a tunnel endpoint of an IP address configured on the IPsec tunnel peer (the computer or router on the other side of the tunnel).

  2. A rule for the inbound traffic for the tunnel. The rule for the inbound traffic is configured with both a filter list that describes the traffic to be received across the tunnel and a tunnel endpoint of a local IP address (the computer or router on this side of the tunnel).

Note

You must create both an inbound and outbound for each tunnel connection. If a filter is created for only one direction, the rule will not be applied.

In Windows Vista and Windows Server 2008 you can specify either an IPv4 address or an IPv6 address. You must specify an endpoint for each side of the tunnel and the address protocol version must be the same for both sides. That is, if you specify an IPv6 address for the source side of the tunnel, then you must also use an IPv6 address for the remote side of the tunnel.

For each rule, you must also specify filter actions, authentication methods, and other settings.

See Also

  • IPsec Rules


IPsec Connection Type

Connection Types

For each Internet Protocol security (IPsec) rule, you must define to which connection types the rule will be applied. For example, if you specify that a rule to be applied only to remote access connections, then only these connections will match the rule.

Each rule has one connection type setting:

  • All Network Connections. The rule applies to communications sent through any of the network connections that you have configured on the computer.

  • Local Area Network (LAN). The rule only applies to communications sent through LAN connections that you have configured on the computer.

    Note

    Wireless connections are treated as local area network connections when polices are created using the IP Security Policy snap-in. If you want to specify that a rule is to be applied to a wireless connection only, you must create a policy using the new Windows Firewall with Advanced Security snap-in.

  • Remote Access. The rule only applies to communications sent through any remote access, virtual private network (VPN) connection, or dial-up connections that you have configured on the computer.


IPsec Rules

About IPsec rules

An Internet Protocol security (IPsec) policy consists of one or more rules that determine IPsec behavior. IPsec rules are configured on the Rules tab in the properties of an IPsec policy. Each IPsec rule contains the following configuration items:
Filter list

A single filter list that contains one or more predefined packet filters that describe the types of traffic to which the configured filter action for this rule is applied. The filter list is configured on the IP Filter List tab in the properties of an IPsec rule within an IPsec policy. For more information about filter lists, see Filter Lists.

Filter action

A single filter action that includes the type of action required (Permit, Block, or Negotiate Security) for packets that match the filter list. For the Negotiate Security filter action, the negotiation data contains one or more security methods that are used (in order of preference) during IKE negotiations and other IPsec settings. Each security method determines the security protocol (such as AH or ESP), the cryptographic algorithms, and session key regeneration settings used. The negotiation data is configured on the Filter Action tab in the properties of an IPsec rule within an IPsec policy. For more information see, Filter Actions.

Authentication methods

One or more authentication methods are configured (in order of preference) and used for authentication of IPsec peers during Main Mode negotiations. The available authentication methods are the Kerberos version 5 authentication protocol, use of a certificate issued from a specified certification authority (CA), or a preshared key. The authentication data is configured on the Authentication Methods tab in the properties of an IPsec rule within an IPsec policy. For more information, see IPsec Authentication.

Tunnel endpoint

Specifies whether the traffic is tunneled and, if it is, the IP address of the tunnel endpoint. For outbound traffic, the tunnel endpoint is the IP address of the IPsec tunnel peer. For inbound traffic, the tunnel endpoint is a local IP address. The tunnel endpoint is configured on the Tunnel Setting tab in the properties of an IPsec rule within an IPsec policy. You must create two tunnel rules, one for each direction that traffic will travel. For more information, see IPsec Tunnel Settings.

Connection type

Specifies whether the rule applies to local area network (LAN) connections, remote connections, or both. The connection type is configured on the Connection Type tab in the properties of an IPsec rule within an IPsec policy. For more information, see IPsec Connection Type.

Default response rule

The default response rule is used to ensure that the computer responds to requests for secure communication. If an active policy does not have a rule defined that is requesting secure communication, then the default response rule is applied and security is negotiated, if the default response rule is enabled. For example, the default response rule is used when Computer A communicates securely with Computer B, and Computer B does not have an inbound filter defined for Computer A.

The default response rule, which can be used for all policies, cannot be deleted, but it can be deactivated. You have the option of enabling it when you create new IPsec policies with the IP Security Policy Wizard.

Note

The default response rule will be ignored in a policy that will be assigned to a computer running Windows Vista® or Windows Server® 2008.

Authentication methods and the security methods can be configured for the default response rule. The filter list of <Dynamic> indicates that the filter list is not configured, but that filters are created automatically based on the receipt of IKE negotiation packets. The filter action of Default Response indicates that the action of the filter (Permit, Block, or Negotiate Security) cannot be configured.


IPsec Key Exchange

Key management

Before secured data can be exchanged, a contract must be established between the two computers. In this contract, called a security association (SA), both agree on how to exchange and protect information.

A key is a secret code or number that is required to read, modify, or verify secured data. Keys are used in conjunction with algorithms (a mathematical process) to secure data. In Internet Protocol security (IPsec), there are two phases or modes that use keys. Main Mode occurs first and generates a shared master key that the two parties can use to exchange keying information in a secure way. Quick Mode uses the master key to secure the establishment of one or more session keys that are used for data integrity or encryption.

Windows Server® 2008 and Windows Vista® handle key generation automatically and implement the following keying properties that maximize protection:

Important

The IP Security Policy snap-in can be used to create IPsec policies that can be applied to computers running Windows Vista and Windows Server 2008, but this snap-in does not use the new security algorithms and other new features available in computers running Windows Vista and Windows Server 2008. To create IPsec polices for these computers, use the Windows Firewall with Advanced Security snap-in. The Windows Firewall with Advanced Security snap-in does not create policies that can be applied to earlier versions of Windows.

Dynamic rekeying

IPsec uses a method called dynamic rekeying to control how often a new key is generated during communication. The communication is sent in blocks; each block of data is secured with a different key. This prevents an attacker who has obtained part of a communication and the corresponding session keys from obtaining the remainder of the communication. This on-demand security negotiation and automatic key management service is provided using Internet Key Exchange (IKE), as defined in RFC 2409.

IPsec allows you to control how often a new key is generated. If no values are configured, keys are regenerated automatically at default intervals.

Key lengths

Every time the length of a key is increased by one bit, the number of possible keys doubles, making it exponentially more difficult to determine the key. IPsec provides multiple algorithms to allow for short or long key lengths.

Key material generation: Diffie-Hellman algorithm

To enable secure communication, two computers must be able to gain the same shared key (session key), without sending the key across a network and compromising the secret.

The Diffie-Hellman algorithm (DH) is one of the oldest and most secure algorithms used for key exchange. The two parties publicly exchange keying information, which Windows Vista and Windows Server 2008 additionally protect with a hash function signature. Neither party ever exchanges the actual key; however, after their exchange of keying material, each is able to generate the identical shared key.

DH keying material exchanged by the two parties can be based on 768, 1024, or 2048 bits of keying material, known as DH groups. The strength of the DH group is proportional to the strength of the key computed from the DH exchange. Strong DH groups combined with longer key lengths increase the degree of computational difficulty involved in determining the key.

Note

On the IKE Security Algorithms dialog box, 768 bits corresponds to the Low (1) setting and 1024 bits to the Medium (2) setting.

IPsec uses the DH algorithm to provide the keying material for all other encryption keys. DH does not provide authentication. The Microsoft Windows implementation of IPsec authenticates identities after the DH exchange takes place, providing protection against man-in-the-middle attacks.

Key protection

The following features enhance the base prime numbers (keying material) and the strength of the keys for the master and session keys.

Key lifetimes

Key lifetimes determine when, rather than how, a new key is generated. Also known as dynamic rekeying or key regeneration, a key lifetime allows you to force a key regeneration after a specified interval. For example, if a communication takes 10,000 seconds and you specify a key lifetime of 1,000 seconds, 10 keys are generated to complete the communication. This ensures that even if an attacker is able to decipher part of a communication, the remainder of the communication is protected. Key lifetimes can be specified for both the master and session keys. Whenever a key lifetime is reached, the SA is also renegotiated. In addition, the key is refreshed or regenerated. The amount of data processed by a single key should not exceed 100 megabytes (MB).

Session key refresh limit

A session key refresh limit is used because the repeated rekeying from a session key can compromise the Diffie-Hellman shared secret.

For example, Alice on Computer A sends a message to Bob on Computer B, and then sends another message to Bob a few minutes later. Because an SA was recently established, the same session key material might be reused. If you want to limit the number of times this occurs, set the session key refresh limit to a low number.

If you have enabled master key perfect forward secrecy (PFS), the session key refresh limit is not used. Setting a session key refresh limit to 1 is identical to enabling master key PFS. If both a master key lifetime and a session key refresh limit are specified, the limit reached first causes the subsequent rekey. By default, IPsec policy does not specify a session key refresh limit.

Diffie-Hellman groups

Diffie-Hellman (DH) groups are used to determine the length of the base prime numbers (key material) for the DH exchange. The strength of any key derived from a DH exchange depends, in part, on the strength of the DH group on which the prime numbers are based.

Each DH group defines the length of the keying material to be used. Group 1 protects 768 bits of keying material; Group 2 protects 1024 bits; Group 3 protects 2048 bits. When a larger group is used, the resulting key that is determined from a DH exchange is larger and more difficult to determine by an attacker.

IKE negotiates which group to use, based on settings you configure on the IKE Security Algorithms dialog box, ensuring that there are not any negotiation failures that result from a mismatched DH group between the two peers.

If session key PFS is enabled, a new DH key is negotiated in the first Quick Mode SA negotiation. This new DH key removes the dependency of the session key on the DH exchange that is performed for the master key.

If the initiator is using session key PFS, the responder is not also required to use session key PFS. However, if the initiator is not using session key PFS and the responder is using session key PFS, negotiation fails.

The DH group is the same for both the Main Mode and Quick Mode SA negotiations. When session key PFS is enabled, even though the DH group is set as part of the Main Mode SA negotiation, it affects any rekeys during session key establishment.

Perfect forward secrecy

Unlike key lifetimes, PFS determines how, rather than when, a new key is generated. Specifically, PFS ensures that the compromise of a single key permits access only to data that is protected by it, not necessarily to the entire communication. To achieve this, PFS ensures that a key used to protect a transmission, in either mode, cannot be used to generate additional keys. In addition, if the key used was derived from specific keying material, that material cannot be used to generate other keys.

Master key PFS requires a reauthentication and is resource-intensive. When it is enabled, IKE must reauthenticate identities, increasing overhead for domain controllers when the Kerberos version 5 authentication protocol is used for authentication. It requires a new Main Mode negotiation for every Quick Mode negotiation that occurs.

Session key PFS can be used without a reauthentication and is less resource-intensive. Session key PFS results in a DH exchange to generate new keying material. It requires only four messages and no authentication.

PFS does not have to be enabled on both peers because it is not part of the SA negotiation. If the responder requires PFS and the sender's Quick Mode SA expires, it simply rejects the sender's message and requires a new negotiation. The sender causes the Main Mode SA to expire and then renegotiates. PFS can be individually set for both the master and session keys.

Key exchange

Before secured data can be exchanged, a contract between the two computers must be established. In this contract (SA), both agree on how to exchange and protect information.

To build this contract between the two computers, the Internet Engineering task Force (IETF) has established the IKE method of security association and key exchange resolution, which:

  • Centralizes security association management, reducing connection time.

  • Generates and manages shared, secret keys that are used to secure the information.

This process not only protects communication between computers, it also protects remote computers that request secure access to a corporate network. In addition, this process works whenever the negotiation for the final destination computer is performed by a security gateway.

Security association defined

An SA is the combination of a negotiated key, security protocol, and security parameters index (SPI), which together define the security used to protect the communication from sender to receiver. The SPI is a unique, identifying value in the SA that is used to distinguish among multiple security associations that exist at the receiving computer. For example, multiple associations might exist if a computer is securely communicating with multiple computers at the same time. This is a common occurrence when the computer is a file server or a remote access server that serves multiple clients. In these situations, the receiving computer uses the SPI to determine which SA is used to process the incoming packets.

Main Mode SA

To ensure successful and secure communication, IKE performs a two-phase operation. Confidentiality and authentication are ensured during each phase by the use of encryption and authentication algorithms that are agreed upon by the two computers during security negotiations. With the duties split between two phases, key creation can be rapidly accomplished.

During the first phase, the two computers establish a secure, authenticated channel. This is called the Main Mode SA. IKE automatically provides required identity protection during this exchange.

The following steps describe a Main Mode negotiation.

  1. Policy negotiation.

    The following four mandatory parameters are negotiated as part of the Main Mode SA:

    • The encryption algorithm (DES or 3DES).

    • The hash algorithm (MD5 or SHA1).

    • The authentication method (the Kerberos V5 authentication protocol, certificate, or preshared key authentication).

    • The Diffie-Hellman (DH) group to be used for the base keying material (768 bits Low (Group 1), 1024 bits Medium (Group 2), or 2048 bits High (Group 3)).

    • If certificates or preshared keys are used for authentication, the computer identity is protected. However, if the Kerberos V5 authentication protocol is used, the computer identity is unencrypted until encryption of the entire identity payload takes place during authentication.

  2. DH exchange (of public values)

    At no time are actual keys exchanged. Only the base information required by the DH key determination algorithm to generate the shared, secret key is exchanged. After this exchange, the IKE service on each computer generates the master key that is used to protect authentication.

  3. Authentication

    The computers attempt to authenticate the DH key exchange. Without authenticating the DH key exchange, the communication is vulnerable to a man-in-the-middle attack. Without successful authentication, communication cannot proceed. The master key is used, in conjunction with the negotiation algorithms and methods, to authenticate identities. The entire identity payload (including the identity type, port, and protocol) is hashed and encrypted using the keys generated from the DH exchange in the second step. The identity payload, regardless of which authentication method is used, is protected from both modification and interpretation.

    The sender presents an offer for a potential security association to the receiver. The responder cannot modify the offer. Should the offer be modified, the initiator rejects the responder's message. The responder sends either a reply accepting the offer or a reply with alternatives.

    Messages sent during this phase have an automatic retry cycle that is repeated five times. If a response is received before the retry cycle ends, standard SA negotiation begins. If allowed by IPsec policy, unsecured communications will begin after a brief interval. This behavior is known as fall back to clear. Even if the communication falls back to clear, secure communication negotiation is attempted at five-minute intervals.

    There is no limit to the number of exchanges that can take place. The number of SAs established is limited only by system resources.

Quick Mode SA

In this phase, SAs are negotiated on behalf of the IP Security driver.

The following steps describe a Quick Mode negotiation.

  1. Policy negotiation occurs.

    The IPsec computers exchange the following requirements for securing the data transfer:

    • The IPsec protocol (AH or ESP)

    • The hash algorithm for integrity and authentication (MD5 or SHA1)

    • The algorithm for encryption, if requested (DES or 3DES)

    A common agreement is reached and two SAs are established. One SA is for inbound communication and the other is for outbound communication.

  2. Session key material is refreshed or exchanged.

    IKE refreshes the keying material and new shared keys are generated for packet integrity, authentication, and encryption (if negotiated). If rekeying is required, either a second DH exchange (as described in Main Mode negotiation) occurs, or a refresh of the original DH key is used.

  3. The SAs and keys, along with the SPI, are passed to the IP Security driver.

    The second negotiation of security settings and keying material (for the purpose of securing data) is protected by the Main Mode SA. As the first phase provided identity protection, the second phase provides protection by refreshing the keying material before sending data. IKE can accommodate a key exchange payload for an additional DH exchange if a rekey is required—that is, master key PFS is enabled. Otherwise, IKE refreshes the keying material from the DH exchange completed in Main Mode.

    Quick Mode results in a pair of security associations, each with its own SPI and key. One SA is used for inbound communication, and the other for outbound communication.

    The retry algorithm for a message is similar to the process described in Main Mode negotiation. However, if this process times out for any reason during the second or higher negotiation off of the same Main Mode SA, a renegotiation of the Main Mode SA is attempted. If a message for this phase is received without an established Main Mode SA, it is rejected.

    Using a single Main Mode SA for multiple Quick Mode SA negotiations increases the speed of the process. As long as the Main Mode SA does not expire, renegotiation and reauthentication are not required. The number of Quick Mode SA negotiations that can be performed is determined by IPsec policy settings.

    Note

    Excessive rekeying from the same Main Mode SA might make the shared, secret key vulnerable to a known plaintext attack. A known plaintext attack is a sniffer attack in which the attacker attempts to determine the encryption key from encrypted data based on known plaintext.

SA lifetimes

The Main Mode SA is cached to allow multiple Quick Mode SA negotiations (unless master key PFS is enabled). When a key lifetime is reached for the master or session key, the SA is renegotiated. In addition, the key is refreshed or regenerated.

When the default timeout period elapses for the Main Mode SA, or the master or session key lifetime is reached, a delete message is sent to the responder. The IKE delete message tells the responder to cause the Main Mode SA to expire. This prevents additional new Quick Mode SAs from being created from the expired Main Mode SA. IKE does not cause the Quick Mode SA to expire because only the IPsec driver contains the number of seconds or bytes that have passed to reach the key lifetime.

Use caution when setting very different key lifetimes for master and session keys. For example, setting a master key lifetime of eight hours and a session key lifetime of two hours might leave a Quick Mode SA in place for almost two hours after the Main Mode SA has expired. This occurs when the Quick Mode SA is generated shortly before Main Mode SA expiration.