In my post just the other day I provided a link to the new guide covering Exchange publishing via Forefront TMG or UAG. One poster followed up by asking, “How do I set things up when I have both TMG/UAG with a web farm and a hardware load balancer?” It’s a great question, as these days hardware load balancers are becoming commonplace, and trying to get both Forefront TMG/UAG and the load balancer to work together is important to get right. As it happens, I had written something about this up too, and was saving it for a rainy day, so it looks like today, it’s raining. I hope this helps answer the question.
The introduction of the Client Access Server (CAS) role as the MAPI end point Outlook uses to connect to a mailbox has prompted many organizations to consider load balancing internal clients for the first time. The introduction of a load balancer to provide fault tolerance and sharing of load to client access, when combined with using a product such as Forefront TMG or UAG to publish Exchange, when those products can also provide load balancing, can be a source of confusion.
The most common question is whether Forefront TMG (Forefront TMG will be referred to throughout this section but the same is true of Forefront UAG in these scenarios) should be used to publish the Virtual IP address (VIP) created on the load balancer, as shown in the diagram below, or whether a farm of CAS should be configured on Forefront TMG, and that used as the destination for the publishing rule.
Figure 1 - All Connections through the Load Balancer
This approach of publishing the load balancer itself has both advantages and some disadvantages.
An obvious advantage is that a simple, common path now exists for both internal and external client connections, both via the load balancer. The disadvantage is that a single point of failure now exists for all client connections, though that will always be the case when concentrating connections to any form of hardware device and is usually mitigated by using redundancy in the configuration.
Another advantage is that a hardware load balancer usually has many more affinity methods available to it, and so that extra capability can be leveraged when balancing the load across the CAS.
One of the more subtle disadvantages is only clear when you consider how Forefront TMG views the health of the end point it is publishing – if the end point is a single load balancer, if there is an issue connecting to that load balancer the entire target is marked as down, whereas if Forefront TMG is treating the health of each member CAS on an individual basis, then any one member being down does not impact the entire service. This is similar to the previous case however, in that redundancy in the load balancer can help mitigate this risk.
A further issue that can cause problems in this scenario, though it is relatively easy to work around if the network configuration allows it, is that Forefront TMG typically uses its own IP address as the source IP in the TCP packets that reach the load balancer, effectively appearing to the load balancer as a single IP address, or client, which will impact the load balancers ability to distribute load based on source IP address. There are three mitigations to this problem;
- Configuring Forefront TMG to not replace the IP address of the client with its own IP address (though this requires Forefront TMG to be set as the default gateway (or used as the ultimate exit route from the network) on the load balancing hardware (if it is decrypting SSL) to ensure the packets route back through Forefront TMG), or on CAS, if SSL is being decrypted there.
- Configure the load balancer to use a form of affinity other than based on source IP – though this can be a problem for clients such as Outlook Anywhere where one client can create multiple SSL sessions, this can result in sessions from the same client being split across multiple CAS.
- Configure Forefront TMG to use Bi-Directional Affinity (available only in the Enterprise version of Forefront TMG) which allows Forefront TMG to manage this complex networking scenario. There are however some caveats to this approach, which are discussed in this blog post: http://blogs.technet.com/b/isablog/archive/2008/03/12/bi-directional-affinity-in-isa-server.aspx.
One last disadvantage to this solution is that publishing the load balancer itself rather than each individual server is that certain scenarios, any that involve Kerberos Constrained Delegation (KCD) for example (certificate based authentication and NTLM Outlook Anywhere are two Exchange scenarios), cannot be configured. KCD requires that Forefront TMG utilize the Service Principal Name of the delegated service, and since SPN’s cannot be configured on more than one machine in a domain, there is no way to configure KCD from Forefront TMG to CAS in this scenario at this time. In these scenarios, publishing a single virtual IP address, that of the load balancer, would prevent KCD from working altogether.
Another potential solution is to not use the hardware load balancer and simply point all client traffic at Forefront TMG and allow it to load balance all the connections. This is shown in the diagram below, and shows all internal and external client requests being made via Forefront TMG.
Figure 2 - Use Forefront TMG as only Sole Load Balancer
The problem with this suggestion is that Forefront TMG is unable to use a farm for any protocol other than HTTP. Accessing a mailbox from an Outlook client when connected to the same network is done using RPC, POP3 or IMAP4. Neither Forefront TMG nor UAG can load balance these protocols across a farm of servers. Therefore you should not use a name that ultimately uses Forefront TMG or UAG as the MAPI end point for your Outlook clients. Whilst it is technically possible to configure Forefront TMG to make the appropriate ports available, they can only be used to publish a single IP address. This single IP could be a single server, or a load balanced IP address, though if you have load balancing available, but choose to concentrate all your connections to Forefront TMG, you are negating all the benefit of having the load balancer in the environment.
Another alternative would be to force all your internal users into Outlook Anywhere mode, so all traffic is HTTPS and can therefore utilize the Forefront TMG/UAG web farm. Some customers without hardware load balancers have done this to solve this problem, and whilst it is certainly possible, it is not necessary if you do happen to have a hardware load balancer, as we will discuss.
Knowing that Forefront TMG cannot effectively load balance RPC requests, but can load balance HTTP based traffic, you may be tempted to force all your internal Outlook clients to connect using Outlook Anywhere, using HTTP, and then allow Forefront TMG to load balance this traffic to the CAS in the web farm being published. Whilst this would work in most cases, uneven load balancing is often seen as the number of source IP addresses seen by Forefront TMG is low, particularly if NAT is being used in any part of the network, and so the connections from Forefront TMG to CAS tend to be uneven. For this reason a dedicated software or hardware load balancer is the recommended approach for internal Outlook to CAS connections.
The opposite approach is to not use Forefront TMG at all, and instead only use the load balancer at the network edge (assuming the device is designed for and supported in this scenario).
Figure 3 - Use Only a Hardware Load Balancer
In this scenario you benefit from being able to use a multitude of affinity options provided by your load balancing device, and can use the same device for internal and external load balancing if the network supports it, but you do lose the ability to pre-authenticate traffic at the perimeter of the network, and scenarios involving KCD will require that CAS be responsible for terminating the SSL stream from the client.
A better solution is using a web farm for all clients accessing via Forefront TMG, and pointing all internal clients at the hardware load balancer. The diagram below outlines this design.
Figure 4 - Use Forefront TMG to Publish Each CAS and Point Internal Client at the Hardware Load Balancer
In this configuration, a web farm of CAS is created in Forefront TMG, containing the individual CAS servers, and used as the target for all publishing rules. A virtual array is also configured on the hardware load balancer containing those same CAS servers. The internalURL and DNS settings, used by clients connecting when inside the network point to the load balancing device, and the external settings resolve to the external interface of Forefront TMG.
The advantage of this approach is that clients benefit from being able to use the hardware load balancer for all protocols, including RPC, and Forefront TMG provides the load balancing for clients accessing from the Internet, and fully support scenarios such as certificate based authentication, by being able to delegate to specific CAS within the farm.
If you require POP3 and/or IMAP4 access from the Internet, this would be the only scenario where using Forefront TMG to publish the internal Virtual IP of the load balancer would be recommended, as Forefront TMG is unable to publish those protocols to a web farm, and using the VIP as a target gives additional availability to the solution.
The final presented solution is to simply place the load balancer at the network edge (assuming the device is designed for and supported in this scenario), and use it to publish any Exchange resources that you do not wish to pre-authenticate or for which you require KCD.
Figure 5 - Split the Edge Connections between Devices as Needed
This solution allows Forefront TMG to provide pre-authentication to the Outlook Anywhere users and perform KCD back to CAS (and could easily allow certificate based authentication with KCD for ActiveSync users), and enables the load balancer itself to be used for OWA and EAS access. There is no perimeter pre-authentication for these clients, which is a trade off, but this allows the full range of load balancer affinity types to be used for these clients, and avoids the routing complexities previously discussed. It’s an unusual configuration, requiring pre-authentication for Outlook Anywhere, but not for OWA, but some customers may choose this route as they are using some kind of custom security software on their CAS to provide strong authentication, and that software can’t be installed on Forefront TMG.
The choices available to you are summarized in the table below.
|
Depicted in Figure
|
Network Edge
|
Internal Clients
|
Advantages
|
Disadvantages
|
|
Figure 1
|
Forefront TMG publishes Hardware load balancer VIP
|
Hardware load balancer VIP
|
- Simple Configuration
- Ability to leverage multiple affinity types
|
- HW load balancer requires redundancy to avoid being a single point of failure and marked as down by Forefront TMG
- Network routing can be a problem
- Cannot be used if Certificate Based Authentication or NTLM Outlook Anywhere with pre-authentication is required
|
|
Figure 2
|
Forefront TMG balances load over each and every CAS
|
Route all traffic to TMG
|
- Removes the need for the additional cost of the load balancer
|
- Cannot provide resilient and load balanced RPC Client Access to internal Outlook clients
- Likely poor load balancing for internal clients due to small source IP pool
- Network configuration may make this difficult to implement
|
|
Figure 3
|
Load Balancer balances load over each and every CAS
|
Hardware load balancer VIP
|
- Removes the need for the additional cost of Forefront TMG
- Allows most affinity methods to be used
|
- No ability to pre-auth traffic entering via the load balancer
- Network configuration may make this difficult to implement
- Scenarios involving KCD require SSL termination and certificate validation to be done on CAS
|
|
Figure 4
|
Forefront TMG balances load over each and every CAS
|
Hardware load balancer VIP
|
- Certificate Based Authentication or NTLM Outlook Anywhere with pre-authentication is possible
- Hardware load balancer can balance Outlook RPC traffic effectively
- Ability to leverage additional affinity types
|
- Two load balancing pools to manage
|
|
Figure 5
|
Forefront TMG balances load over each and every CAS and
Load Balancer balances load over each and every CAS
|
Hardware load balancer VIP
|
- Certificate Based Authentication or NTLM Outlook Anywhere with pre-authentication is possible
- Ability to use multiple affinity types
|
- Multiple external namespaces required
- No ability to pre-auth traffic entering via the load balancer
|
Conclusion
The decision as to which of these solutions you should deploy will come as a result of understanding the scenarios you wish to support, and considering the network implications that can impact routing and load balancer effectiveness. It is important to understand that if you require pre-authentication of traffic in the perimeter network, then you need to deploy Forefront TMG, but if you don’t, you could simply use the load balancer to do load balancing for internal and external users. If you realize that you need to load balance RPC Client Access traffic, you need a hardware or software load balancer, as you cannot do that with Forefront TMG. If you ultimately want the best of both worlds, you may decide to deploy both, and use them for different purposes. As long as you carefully plan your requirements, you should be able to make the decision based on your needs, but always remember to keep one eye on the future. Things can change!