Exchange 2010 Site Resilience, Multiple DAG IPs, and Cluster Resources

Exchange 2010 allows us to have Database Availability Group (DAG) members in several AD Sites. For every subnet a DAG member’s MAPI NIC is in, we must obtain a DAG IP. This DAG IP is a separate IP than is located on the MAPI NICs themselves. We take this DAG IP to the DAG using the Set-DatabaseAvailabilityGroup command.

Multiple DAG IPs

Let’s take a look at an example of how the architecture may look.

Taking a look at the above Visio diagram, we have two sites, Primary Site and DR Site, with one node in each. The MAPI NIC in the Primary Site has an IP Address of 172.17.24.200. That means that we’ll need to have a DAG IP that lives in this same subnet. We choose a DAG IP of 172.17.24.120. The MAPI NIC in the DR Site has an IP Address of 172.16.24.200. That means that we’ll need to have a DAG IP that lives in this same subnet. We choose a DAG IP of 172.16.24.120.

In order to add these MAPI IP Addresses, we’ll need to run the following the command.

Note: IPs on Replication NIC’s subnet do not get added to the Database AvailabilityGroupIPAddresses. Only MAPI NIC Subnets get added.

Keep in mind, when adding additional IPs in the future, it is important that you include all existing DAG IPs. The Set-DatabaseAvailabilityGroup -DatabaseAvailabilityGroupIPAddresses property is not additive.

To verify the DAG IPs were added successfully, let’s check out our DAG Properties.

In Exchange 2010 SP1, we have the ability to add our DAG IPs via the GUI. If we go to the DAG Properties, we now see we can manage our Witness Server and Alternate Witness Server.

This allows us to do our IP Address configuration right from the GUI instead of needing to use Set-DatabaseAvailabilityGroup with the DatabaseAvailabilityGroupIPAddresses property and needing to worry about all previous IP Addresses being included since the property isn’t additive.

Cluster Resources

So, let’s take a look at what really happens to the cluster resources and what determines which DAG IP is active. Let’s open the Failover Cluster Manager. Start > Administrative Tools > Failover Cluster Manager.

After selecting our DAG, let’s take a look at the cluster resources. We can see from here that we have two Network IP Resources.

But let’s take even a deeper look.

Select the DAG from within the Cluster Core Resources > Right-Click > Choose Properties.

Now let’s take a look at the Dependencies Tab.

As we can see, the two DAG IPs are set up with an OR dependency which means that the cluster can activate either DAG IP at any given time. As we saw earlier, the 172.16.24.120 IP is the existing DAG IP that is online which means the DRSiteNode’s DAG IP is currently the online Network IP resource.

Let’s run a cluster command so we can failover the default “Cluster Group” from one cluster node to another.

We now see the PrimarySiteNode is the node that has the “Cluster Group.” Let’s go ahead and take a look at the Cluster Resources again and see which Network IP Resource is online.

Looks like the PrimarySiteNode’s DAG IP is now Online instead of the DRSiteNode’s DAG IP. This means that the Network IP Resource that is online depends on which DAG Node has the “Cluster Group.” If you recall from my previous articles, the DAG Node that has the “Cluster Group” is the DAG Node that acts as the Primary Active Manager. The Primary Active Manager is the DAG Node responsible for choosing what databases get activated in a failover. For more information on Active Manager, click here.

Elan Shudnow :: Sep.27.2010 :: Exchange, Exchange 2010 ::

191033 Responses2010-09-27+13%3A27%3A01Elan+Shudnow to “Exchange 2010 Site Resilience, Multiple DAG IPs, and Cluster Resources”

on 27 Sep 2010 at 9:08 am1Exchange 2010 Site Resilience, Multiple DAG IPs, and Cluster Resources | Elan Shudnow’s Blog « JC’s Blog-O-Gibberish

[...] Exchange 2010 Site Resilience, Multiple DAG IPs, and Cluster Resources | Elan Shudnow’s Blog Posted on September 27, 2010 by johnacook http://www.shudnow.net/2010/09/27/exchange-2010-site-resilience-multiple-dag-… [...]
on 01 Oct 2010 at 7:49 am2Chris Lehr

Good article. The cluster.exe command needs to be documented more. However, your article starts with a MAPI and a DAG network, and then your DAG networks only cover/address one of the networks you started the article with.
on 10 Nov 2010 at 6:42 am3Johnson

I was searching for this document. Thanks alot for the explanation.
on 03 Jun 2011 at 2:03 am4Tariq.Muhammad

Nice posting…… this is helpful document……. Thanxxxxs.
on 09 Jul 2011 at 5:47 am5admin2010

Thanks for the article. I would like to know that the MAPI NIC ip is belongs to the LAN network and Replication NIC is belongs to the heartbeat???? File sharing option should be enabled on heartbeat connection or not???
on 18 Jul 2011 at 2:09 am6Guest

Nice, this is a real good article and saved lot of my time…Excellent work.
on 05 Aug 2011 at 6:35 pm7Elan Shudnow

Well in Exchange 2010 both NICs do heartbeating. In fact, the Exchange 2010 documentation wants you to ACL the network so MAPI NIC on Node A cannot talk to MAPI NIC on Node B to prevent any heartbeat crosstalk. Here is a good article on NIC configuration: http://www.howexchangeworks.com/2010/05/network-a…
on 14 Aug 2011 at 2:03 am8Ian Salgado

Hi, I refer u to your diagram above

Are the 2 Nic for MAPI & Replication 2 different physical NIC’s?

Cheers

Ian
on 15 Aug 2011 at 6:48 am9Elan Shudnow

Ian, they are two different physical NICs. One Physical NIC for the MAPI Network and a separate Physical NIC for the Replication NIC.
on 16 Aug 2011 at 12:45 am10Ian Salgado

Hi Elan,

Thank You for your quick response.

On the MAPI NIC – Obvisouly traffic between the different MAPI connections needs to be open, right ? so that mail flow can occur.

On the REPLICATION NIC –

1. Does this network segment needs to be SEPARATE FROM the MAPI NIC segment ? ie: MAPI-NIC=192.168.1.xxx & REP-NIC=192.168.2.xxx
2. Again all traffic between DAG member on this REP-NIC's will need to be open ?

Thank You

Regards

Ian
on 16 Jan 2012 at 5:31 am11MOahmed

Now i have previous DAG should i create new dag to the DR or Branch site or add th DR site Mailbox to the primary site dag.

Thank You

Regards
Mohamed
on 16 Jan 2012 at 8:20 am12Elan Shudnow

Without knowing much about your business requirements and what the conceptual design is, it's hard to be definitive. But typically, if you have a Primary Site and a Failover Site you would use the same DAG for both locations so you can replicate databases from Server in Primary Site to Server in Failover Site.
on 18 Jan 2012 at 2:00 am13Mohamed

Thank you very Much Elan
on 18 Jan 2012 at 2:19 am14Mohamed

So Elan im starting get confused let me tell you what i have

DC1

Domain 2008

Cas Array 2010 using WNLB.

HUB on 2 server's Using Fail-over.

Dag01
DC1 all ready installed on it certificate after reading information i think will need to buy new San certificate hold primary and secondary site names also auto discovery.

the Second DC2
i think i will do the followin ti apply Active Passive Scenario wit the same name

Install Additional A.D In the Second Data-center.
add database copy from the primary Data center to the second Data Center ……. here now
i will run this command

Set-DatabaseAvailabilityGroup -DatabaseAvailabilityGroupIPAddresses [ Primary Dag IP only !!!!! ]

that what i think only one DAG with one witness share …… that wht i understand from you

so what i do next

thank you

Mohamed,
on 26 Jan 2012 at 12:11 pm15Luis Chavez

Thanks Elan, Excellent Article really this is helpful document.
on 29 Feb 2012 at 8:35 am16Confluence: Exchange 2010

Questions for Bill…

Exchange 2010 questions WMWare rosala: After some further investigation I want to suggest we consider turning on DRS and leaving it in manual mode…….
on 01 Mar 2012 at 5:05 am17Sean O'Farrell

Super post. Thanks.
on 25 Jul 2012 at 5:24 pm18Adrian

So if I have a three node DAG across two subnets and I have NOT configured multiple DAG IP's is my configuration incorrect?
on 25 Jul 2012 at 9:08 pm19Elan Shudnow

Not necessarily. If no IP Addresses are entered, it will use DHCP to obtain a cluster IP for each segment just as long as DHCP is available to on the same subnet that hosts the MAPI IPs. Many environments don't have DHCP on the subnets. Since your DAG is working, it sounds like DHCP is available. But while it may be available for the MAPI Network that is hosted in one site, it may still not be available in the other site that hosts that MAPI Network.

I typically assign static IPs.
on 26 Jul 2012 at 3:57 pm20Adrian

Thanks Elan, here is my setup with single DAG IP set statically. Will I need to set multiple DAG IP Addresses?

server1 at site A IP address: 172.29.0.98/23 and GW 172.29.0.1
server 2 at site A IP Address 172.29.0.99/23 and GW 172.29.0.1
server 3 at site B IP Address 172.29.8.47/23 and GW 172.29.8.1
on 26 Jul 2012 at 4:37 pm21Elan Shudnow

Site B is a different network. Therefore, you need one DAG IP in the subnet located at Site A and another DAG IP in the subnet located at Site B. The way it's set up right now is potentially incorrect. The reason I say potentially is the same as the reason I gave in my previous comment to you.
on 07 Aug 2012 at 5:55 am22prakash

Hi Elan

my self prakash..am facing a prob ..in OWA am unable to delete move search the mail.. but in outlook it was fine
we r running windows2008R2 with exchange 2010sp1 updated rollup1…but on that day on wards am facing these probs
on 24 Sep 2012 at 8:22 am23JazManUni

Hi Elan,
I f I have a small site of 2000 users, I want to have 2 cas/hub servers in 1 cas array and 2 additional mailbox servers in DAG can I put the FSW on one of the CAS/HT servers?
Is there an issue putting the FSW on a cas array member?
Alternatively, if I create a third mailbox server instaed will this be overkill for 2000 users?
Thanks for your help
on 26 Sep 2012 at 9:55 pm24Elan Shudnow

Sure, you can have a FSW on a HUB/CAS. In fact, if you don't manually designate a specific server/share for the FSW, a HUB Server in the same site will automatically be chosen to put the FSW on that HUB in C:\. If you had a separate HUB Server and a separate CAS Server, you could also choose to put it on a CAS.

I would honestly be reluctant to tel you adding a third server would be overkill. It depends on your business requirements in regards to data retention, high availability, DR, etc… But in simple fashion, a single server can easily handle 2000 users if you spec it to have the necessary cpu, memory, and disk requirements.

And by the way, I would opt to deploy multi-role servers with a hardware load balancer. I'm not a fan of separation of roles. Either is Microsoft.
on 04 Oct 2012 at 1:42 pm25James

Hi Elan

I think I have this almost figured out. I just have two questions. I have 2 mailbox servers in subnet A and both are members of DAG1. I am adding a 3rd mailbox server in subnet B. Do I add the DAG Ip address for subnet B then add the third server to the DAG? or add the server to the DAG and then add the subnet B address to the DAG. Second question, do I add the subnet B DAG IP to DNS? We use Netbackup which does query DNS for the DAG IP address.
on 06 Oct 2012 at 6:46 am26Elan Shudnow

Add the DAGIP beforehand. You do not need to manage DNS manually for your DAG. When the DR Server becomes the Primary Active Manager (Default Cluster Group is on DR DAG Server), DNS is updated to point to that Cluster IP. But, clients don't connect to that Cluster IP like they did in Exchange 2007. They connect to the CAS Server's RPC Client Access FQDN which then makes the appropriate MBX connections.
on 08 Oct 2012 at 2:06 am27Adrian

Hi James, we use arcserve r16 and when backing up the DAG it's best to use the DAG DNS name although you can use IP addresses or HOSTS file when backing up pre-prod DAG via a production backup server.
on 11 Dec 2012 at 3:38 pm28Confluence: IT

Provisioning uk-virt0…

4GB USB Stick Software Requirements: bootable ES…
on 11 Jan 2013 at 11:56 pm29Varun

Hi, we hv the same simillar setup that ha shown above in diagram. We hv problem whenever network Links to DR unstable then the entire cluster is unstable and all exchange DB's are getting dismounted and getting mounted.. is there any way we can configure all servers in production within cluster and DR shld be used only for replication.
on 30 Mar 2013 at 12:05 pm30Nate

Varun had an excellent question which has sadly remained unanswered. I am experiencing the exact same issue as Varun, and would like to know if there is anything that can be done about it.

Any response at all would be greatly appreciated.

Thanks,
Nate
on 31 Mar 2013 at 9:01 am31Elan Shudnow

There can only ever be 1 MAPI Network. The DAG chooses the NICs that are configured to register in DNS and have a valid DNS record. All the replication NICs need to have DNS registration disabled. If multiple NICs are registering in DNS, that can potentially cause the DAG to have some issues.

For proper NIC/Network configuration, see the following link: http://technet.microsoft.com/en-us/library/dd6381…
on 31 Mar 2013 at 9:01 am32Elan Shudnow

I responded to Varun. Hope that information helps.
on 03 Apr 2013 at 2:14 am33Ganesh

Hi Elan,

Nice Article..

I have a query here, i have site A and Site B and streched DAG members, i dont want database to automatically failover to site b server even if site a server is down. I want only DB & Logs to be copied to another site server.

thnks

Trackback this post | Feed on Comments to this post

Elan Shudnow's Blog

Just another IT guy!