Elan Shudnow’s Blog

Welcome to Part 2 of this article series. In Part 1, we started off by discussing the goal of this lab. That goal is how to deploy a two-node Exchange 2010 RTM Database Availability Group (DAG) on Windows Server 2008 R2. We then prepared our Operating System with the Exchange 2010 Prerequisites which includes software prerequisites as well as modification to hardware configuration such as our Network Interface Cards (NIC)s.

In this Part, I will go over the installation of one of our Exchange 2010 Servers which will include the Mailbox, Client Access, as well as Hub Transport Server Roles.

Part 1

Part 2

Part 3

Part 4

Installation

With Exchange 2010, we still have the setup.com for unattended mode installations using the Command Line Interface (CLI) as well as setup.exe for attended mode installations using the Graphical User Interface (GUI). We’ll be using the GUI for purposes of this lab.

After running setup.exe, we’ll be presented with the following screen:

We can see that the first two steps are already taken care of.  If you recall from Part 1, we used PowerShell to take care of the prerequisite installations.  So, let’s proceed to Step 2 and choose our language.  For me, it will be English.

When clicking on the language option, we get a couple choices.

If you choose the first option, Install all languages from the language bundle, you will be provided with an option to download the language pack or use an already downloaded language pack.  For purposes of this lab, we’ll choose the second option as we’ll only be using English.

It’s finally time to choose Step 4 and Install Exchange!

So let’s go ahead and choose Step 4 and let’s begin installing Exchange.

After some initializing, we’re provided with the Installer GUI.  The first page, as you guessed it, an Introduction Page.  Read the Introduce Page and Click Next to Continue.

You are now provided with the License Agreement.  After reading the agreement, select “I accept the terms in the license agreement.” Click Next to Continue.

You are now provided with the Error Reporting page.  I like to choose Yes for this option.  The reason why is when you call into Premiere Support Services (PSS), they will have some error reporting information from your servers that may assist with the troubleshooting/fixing of your server.  Choose whichever option best fits your needs.  Click Next to Continue.

You are now provided with the Installation Type.  Previously, in Exchange 2007 CCR/SCC, you could only install the Mailbox Server role.  Now, with DAGs, you can have HUB/CAS/MBX/UM all on the same server.  We’ll be choosing the Typical Exchange Server Installation for this lab which includes HUB/CAS/MBX as well as the Exchange Management Tools.  A nice tip to note is that you can have both the Exchange 2007 Management Tools as well as the Exchange 2010 Management Tools installed on the same box.  Click Next to Continue.

You are now provided with Client Settings.  If you have Outlook 2003 or Microsoft Entourage, click Yes.  This creates a Public Folder Database and modifies some Exchange options such as OAB Distribution for Public Folders to provide support for these clients.  As a side note, there was an msexchangeteam.com blog post that stated that Entourage is getting updated to support Exchange Web Services (EWS) so in the future, you may only have to do this for Outlook 2003 clients and not Entourage.  For purposes of this lab, we will not be using Entourage or Outlook 2003.  Click Next to Continue.

Your first server is most likely going to be an internet facing CAS server.  Because of this, I specified our Internet-facing FQDN.  This will modify your -ExternalURL parameters for this Exchange 2010 CAS.  Pretty nifty and an installation option I very much welcome.  Click Next to Continue.

You are now provided with the Customer Experience Improvement Program.  I always like to join these things to provide information to Microsoft to help make the product better. Click Next to Continue.

Finally, it’s time for some Readiness Checks.  We can see that the organization will need to be prepared with a /PrepareAD which will prepare the schema, forest, and domain.  Make sure the person running this installation is a part of the Enterprise Admins and Schema Admins in order to update the schema.

We also see that we need the Filter Pack.  I didn’t include this in Part 1 as Microsoft updates their Setup Prerequisite files and this (links/files/requirement)  may change in the future.  So go to the link here to download the filter pack.  Make sure you download and install the x64 version.  You can install the Filterpack while the Exchange setup is still running.  Once the Filterpack is finished installing, Click Install in the Exchange Setup.

So now you’re presented with the installation.  It only took 10-15 minutes for the install to complete.  Pretty fast!  Click Finish to Finish.

Summary

Well folks, that is all for Part 2 of this article. For Part 3, I will go through the DAG creation process (including File Share Witness on a non-Exchange Server) as well as add our first node to the DAG.

Elan Shudnow :: Nov.03.2009 :: Exchange, Exchange 2010, Server 2008 R2 :: 13724 Comments »http://www.shudnow.net/2009/11/03/exchange-2010-rtm-dag-using-server-2008-r2-%e2%80%93-part-2/Exchange+2010+RTM+DAG+using+Server+2008+R2+%E2%80%93+Part+22009-11-03+13%3A26%3A09Elan+Shudnow

Now that Exchange Server 2010 is RTM and Server 2008 R2 is RTM, I thought it would be nice to create a multi-part article on how to use create a Database Availability Group (DAG) on two Exchange Server 2010 RTM nodes utilizing Server 2008 R2 as their Operating System. This article is to guide you through the entire process from preparing Server 2008 R2 for Exchange 2010 RTM, installing Exchange 2010 RTM, creating databases, creating a DAG, adding our nodes to the DAG, and then replicating our databases between both servers.

Part 1

Part 2

Part 3

Part 4

Lab Setup

Guest Virtual Machines

One Server 2008 R2 Enterprise (Standard can be used) RTM x64 Domain Controller.

Two Server 2008 R2 Enterprise (Enterprise Required) RTM x64 (x64 required) Member Servers where Exchange 2010 RTM will be installed with the Mailbox, Client Access Server, and Hub Transport Server roles.

One Server 2008 Enterprise (Standard can be used) RTM x64  server that will be our File Share Witness (FSW) Server.  This box will not serve any other purpose in this lab other than FSW.

Assumptions

• You have a domain that contains at least one Server 2003 SP2 Domain Controller (DC).
• You have configured the IP settings accordingly for all servers to be on the same subnet which includes the public NICs for both Failover Cluster nodes. I have provided the IP scheme of my lab below, but this will vary depending on your needs and VMware configuration.

Computer Names

DAG Node 1 – SHUD-EXC01

DAG Node 2 – SHUD-EXC02

Domain Controller – SHUD-DC01

FSW Server – SHUD-OCSFE01

Configuration of  Exchange 2010 DAG Nodes

Processor: 4

Memory: 1024MB

Network Type - MAPI NIC (MAPI Network)

Network Type - Replication NIC (Replication Network)

Virtual Disk Type – System Volume (C:\): 50GB Dynamic

Storage Note: In a real-world environment, depending on the needs of the business and environment, it is best practice to install your database and logs on separate disks/spindles; both of which are separate from the spindles that the C:\ partition utilize. We will be installing Exchange 2010 RTM databases/logs on the same disks/spindles for simplicity sakes for this lab.  While Exchange 2010 databases move a lot of the IO for databases to sequential IO, there’s still quite a bit of Random IO occurring and is still recommended to place the database and logs on separate spindles.

Network Note: A single NIC DAG is supported.  It is still recommended to have at least one dedicated replication network.  If using only a single NIC, it is recommended for this network to be redundant as well as gigabit.

Configuration of  Domain Controller

Processor: 4

Memory: 512MB

Network Type - External NIC

Virtual Disk Type – System Volume (C:\): 50GB Dynamic

IP Addressing Scheme (Corporate Subnet otherwise known as a MAPI Network to Exchange 2010 DAGs)

IP Address – 192.168.1.x

Subnet Mask – 255.255.255.0

Default Gateway – 192.168.1.1

DNS Server – 192.168.1.150 (IP Address of the Domain Controller/DNS Server)

IP Addressing Scheme (Heartbeat Subnet otherwise known as a Replication Network to Exchange 2010 DAGs)

IP Address – 10.10.10.x

Default Gateway – 10.10.10.x

Subnet Mask – 255.255.255.0

LAB Architecture

Some notes about this architecture:

• Exchange 2010 DAGs remove the limitation of requiring Mailbox Only Role Servers as existed with Exchange 2007 Clustered Servers
• Exchange 2010 is no longer Cluster Aware and only utilizes very few pieces of the Failover Cluster Services such as Cluster Heartbeat and Cluster Networks.  More on this in an upcoming part.
• UM is supported on these two DAG nodes but is recommended to be installed on separate servers
• For HTTP publishing, ISA can be utilized.  For RPC Client Access Server publishing (which ISA cannot due as it publishes HTTP traffic only) with CAS Servers on the DAG nodes, you must use a hardware load balancer due to a Windows limitation preventing you from using Windows NLB and Clustering Services on the same Windows box.  Alternatively, you can deploy two dedicated CAS Servers and utilize Windows NLB to load balance your RPC Client Access Server traffic.
• Two node DAG requires a witness that is not on a server within the DAG.  Unlike Exchange 2007, Exchange 2010 automatically takes care of FSW creation; though you do have to specify the location of the FSW. It is recommended to specify the FSW to be created on a Hub Transport Server.  Alternatively, you can put the witness on a non-Exchange Server after some prerequisites have been completed.  I will be deploying the FSW on a member server (which happens to be my OCS Server in my lab) and will display the prerequisite process for achieving this.

Preparation of Exchange 2010 RTM DAG Nodes

Network Interface Card (NIC) Configuration

First thing we will want to do is configure the IP Configuration of both the MAPI NIC and the Replication NIC.

We will want to rename our MAPI NIC connection to MAPI and our Replication NIC connection to Replication. To do so, go to Start > Right-Click Network > Properties.

Once in the Control Panel, Choose Change Adapter Settings.

Now you will be presented with the Network Connections window. This is where you can modify the network properties for each NIC in your server. For your Internal Corporate Connection which is also your MAPI Network, rename your Local Area Connection to Internal. Likewise, for your Private Heartbeat Connection which is also your Replication Network, rename your Local Area Connection to Replication. After you have done this, it will look something similar to the following:

Network Interface Card (NIC) Configuration

First thing we will want to do is configure the IP Configuration of both the MAPI and Replication NIC.

Part of the assumptions earlier in this article as that you have a properly configured TCP/IP Network where all nodes are properly connected to the TCP/IP Network. Because of this, I will skip the Public TCP/IP Configuration and proceed to configuring the Private Heartbeat NIC.

Important: When configuring the MAPI NIC, you can leave IPv6 enabled if you are using Server 2008 R2.  There is an issue with Server 2008 (still exists in SP2) that prevents IPv6 from listening on port 6004 that prevents Outlook Anywhere from working. You can read more about that here. Again, Server 2008 R2 does not have this issue.  So if you happen to be installing Exchange 2010 on Server 2008, disable IPv6 as discussed below.  If using Server 2008 R2, feel free to leave IPv6 enabled.

Note: You can, if you’d like, disable File and Printer Sharing for Microsoft Networks.  In Exchange 2007 SP1, Microsoft provided the ability to allow for continuous replication to occur over the private network.  Because Exchange 2007 utilizes SMB for log shipping, it is required to have the File and Printer Sharing enabled.  Exchange 2010 no longer utilizes SMB and now utilizes TCP.  More on this later in an upcoming Part.

In addition to disabling IPv6 from the NIC Properties, I would follow these instructions here to fully disable IPv6 on your Exchange 2010 system as disabling it on the NIC itself doesn’t fully disable IPv6.  While the article is based on Exchange 2007, it’s a Windows based modification and will apply to a system running Exchange 2010 as well.

Double-Click or Right-Click > Properties on the Replication NIC to begin configuration.

Uncheck the following:

• Internet Protocol Version 6 (TCP /IPv6) – Disable IPv6 in the registry as well as noted above.

Select Internet-Protocol Version 4 (TCP /IPv4) and press the Properties button. For NodeA, the only TCP/IP configuration we will need, is the IP Address and Subnet Mask. NodeA’s IP configuration will be 10.10.10.152/24 while NodeB’s IP configuration will be 10.10.10.153/24.

Go into the Advanced NIC configuration settings by clicking the Advanced button. From there, you will navigate to DNS tab and de-select “Register this connection’s addresses in DNS.”

Select the WINS tab and de-select “Enable LMHOSTS lookup” and configure the NetBIOS setting to “Disable NetBIOS over TCP/IP.”

Once you are done configuring the Advanced settings, press OK three times and you will be back at the Network Connections screen. From here, choose Advanced and select Advanced Settings…

You will be presented with the Binding Order for your current NICs. Ensure that the MAPI NIC is on top by selecting MAPI and pressing the green up arrow key on the right-hand side of the dialog.

Exchange 2010 Operating System Prerequisites

Server 2008 SP2 and Server 2008 R2 prerequisites are quite different.  Because our servers are going to be deployed on Server 2008 R2, we will follow the guidance for deploying on Server 2008 R2.  You can see the prerequisite requirements here.

We will be doing our prerequsite installations via PowerShell.  You can open PowerShell by going to Start > Run > PowerShell.

You will first have to import the module for ServerManager.  Afterwards, depending on the roles that are installed, different prerequisites are required.  Because we are going to be installing HUB/CAS/MBX, the command we would run is the following:

Note: The installation documentation does not have you include Failover-Clustering in the above command.  I add it anyways since we’ll be using it for the DAG.  I you don’t add it in the above command, you can just add it below when you enable the NetTcpPortSharing.  If you don’t add it below, when you add the first node to the DAG, Failover Clustering will be installed anyways.  I like to install it beforehand though.

Finally, we’ll want to modify the NetTcpPortSharing service to start automatically.

Summary

Well folks, that is all for Part 1 of this article. For Part 2, I will go through the installation process of one our DAG nodes that will contain the Client Access Server, Hub Transport Server, as well as Mailbox Server roles.

Elan Shudnow :: Oct.29.2009 :: Exchange, Exchange 2010, Server 2008 R2 :: 130238 Comments »http://www.shudnow.net/2009/10/29/exchange-2010-rtm-dag-using-server-2008-r2-%e2%80%93-part-1/Exchange+2010+RTM+DAG+using+Server+2008+R2+%E2%80%93+Part+12009-10-29+15%3A07%3A20Elan+Shudnow

Exchange has an Online Maintenance task that runs every night.  In the Exchange Management Console (EMC),  go to Organization Configuration > Mailbox > Database Management Tab > Right-Click our Database > Properties >  Maintenace Tab. We can then see our Maintenance Schedule.

In Exchange 2010, this will appear as:

As you can see, in Exchange 2010, there is a new option that is enabled by default.  This option is the “Enable background database maintenance (24 x 7 ESE scanning).  This option is not Online Defragmentation, but is rather Database Checksumming.  More on this later…

In Online Maintenance, there’s several tasks that run such as dumpster cleanup, purging mailboxes based on retention, etc…  You can see a full list of these tasks here.  When these eleven tasks successfully finish, an Online Defragmentation (OLD) process runs.  Microsoft explains OLD as, “The intention for online defragmentation is to free up pages in the database by compacting records onto the fewest number of pages possible, thus reducing the amount of I/O necessary. The ESE database engine does this by taking the database metadata, which is the information in the database that describes tables in the database, and for each table, visiting each page in the table, and attempting to move records onto logically ordered pages.”

There is also a process called Online Maintenance Database Checksumming.  Checksumming checks the integrity of the database by looking through every database page since there was no guarantee OLD would successfully look through every page to ensure there is no corruption.  This process is entirely sequential and doesn’t cause a performance problem on the database.  The issues with this method in Exchange 2007 RTM is that this process ran at the end of Online Maintenace and because of that, resiliency is effected as these processes temporary suspend continous replication.  In Exchange 2007 SP1, Microsoft provided registry keys to allow you to run background checksumming.  You can read more about these processes and the registry keys at the bottom of this article here.

In Exchange editions prior to Exchange 2010, we can monitor OLD by checking out the available 70x Event IDs in the Event Viewer’s Application Log.  Similarly, you can verify the amount of whitespace that has been created in the database by viewing the 1221 Event ID.   The list of Event IDs for Exchange versions prior to Exchange 2010 is as follows:

• 700 – Starting
• 701 – Completed
• 702 – Resuming
• 703 – Completed Resumed Pass
• 704 – Interrupted and Terminated
• 1221 – Whitespace Amount

This has all changed quite a bit in Exchange 2010.  OLD2 is the new version of Online Defragmentation and no longer occurs at the end of the Online Maintenance Schedule.  Instead, it runs 24 x 7 on a database. It is throttled so it does not negatively affect performance.  You cannot modify OLD2 to run as OLD did in earlier versions of Exchange.  OLD2 is not configurable. Because of this, the need to troll the above Event IDs is no more.  Instead of trolling 70x Event IDs, Exchange 2010 will only notify you if something goes wrong with Online Maintenance.  That way all the 70x error codes do not appear as spam.  If you see a 70x in Exchange 2010, you know there is a problem. Keep in mind though, that this is all in regards to Mailbox Databases.  Queue Databases still have 70x Event IDs.

If you need to check available whitespace, you can now do this via the Exchange Management Shell (EMS).  If your database is called Database1, the command would be:

Note: The -Status switch is required when you need to contact the database directly for the following pieces of information:

• BackupInProgress
• Mounted
• OnlineMaintenanceInProgress
• Available free space in the database root

As stated, OLD2 is throttled and doesn’t negatively affect performance.  If you have an interest in monitoring performance of OLD2, you can do so by using the following perfmon counter set: MSExchange Database -> Defragmentation Tasks.

In Exchange 2010, there are two ways you can configure Online Database Scanning (checksumming).  The first is the default option shown in the first image in this article.  By default, it runs as a 24 x 7 process on the Active Database.   You can uncheck this option which will then revert Online Database Scanning so that it runs after all online maintenance tasks are completed.  Because most online maintenance tasks complete within an hour, this process works reasonably fine for smaller databases (under 500GB).  Microsoft now supports up to 2TB databases.  Anything larger than 500GB should definitely have the default set which is to run this process 24 x 7 to ensure it completes.  Exchange 2010 was designed with the mindset that Online Database Scanning should complete at least once every three days.  If it does not, Exchange 2010 will provide a warning event in the Event Logs.

Thanks to Matt Gossage, Program Manager for Storage at Microsoft for providing much of this information. You can see Matt Gossage’s level 300 webcast on Exchange 2010 Storage here.

Elan Shudnow :: Oct.25.2009 :: Exchange, Exchange 2010 :: 1384No Comments »http://www.shudnow.net/2009/10/25/exchange-2010-24x7-online-defragmentation-and-online-database-scanning/Exchange+2010+24x7+Online+Defragmentation+and+Online+Database+Scanning2009-10-26+02%3A12%3A30Elan+Shudnow

I ran into an issue where we had two OCS 2007 R2 Front End Servers behind an F5 Load Balancer.  We kept getting “This service is temporarily unavailable” from our Communicator Clients after we configured the Communicator 2007 R2 Response Group Tab.  For those that are unfamiliar with this tab, it is a web based extension to the Communicator interface that allows users to log in and out of groups.  For more information about the Response Group Tab, click here.

If we take a look at the F5 documentation for OCS 2007 R2 here, we see the following configuration requires for the Response Group Service (RGS):

What this is saying, is that our client will be connecting over 5071 TCP to our Load Balancer in order to communicate with the RGS.  This is incorrect!  5071 TCP is indeed used for Response Group Service communication, but this is only for Front End to Front End server communication.  The RGS has something called a matchmaking service.  The service on the client is just a website that communicates to the Load Balancer over port 443.  So, the million dollar question… Why do we get a service unavailable?

When you’re dealing with the RGS, each Front End Server has a Matchmaking service.  From the Technet Documentation:

Each Front End Server has a Match Making service, which is an internal service that is responsible for queuing calls and finding available agents. Only one Match Making service per pool is active at a time–the others are passive. If a Front End Server with the active Match Making service becomes unavailable, one of the passive Match Making services becomes active. The Response Group Service does its best to make sure that call routing and queuing continues uninterrupted. However, there may be instances when active calls are lost as a result of the transition. Any calls that are in transfer when the service transition occurs are lost. If the transition is due to the Front End Server going down, any calls currently being handled by the active Match Making service on that Front End Server are also lost.

The Match Making service is what utilizes 5071 TCP.  But as stated earlier, this is only for Front End Server to Front End Server communication.  Our Front End Servers need to be able to communicate with each other without traversing the load balancer.  This means that that each server must be able to contact DNS, get the IP of the other server, and then communicate with that IP over 5071.  This is key as to why we’re encountering the issue.

Sometimes, depending on the environment, servers behind load balancers will have multiple IPs assigned.  One for connectivity from the load balancer and another IP for other server operations such as management.  These Front End Servers had their default gateways set to the F5 and each Front End IP that was used for the F5 were on different segments.  The problem here is when one Front End tried to communicate to the other Front End, it would query DNS and get the IP and it would route to the F5.  The F5 would then route it back but the Front End Server saw it coming from the load balancer and think it’s an unauthorized server for RGS requests.  This is why Communicator would see the service as unavailable.

There are a few ways to fix this issue:

• Modify hosts file on each Front End Server so they are communicating to the correct IP which are on the same segment
• Rework your load balancing configuration so the Front End Servers only use 1 IP which is where the load balancer sends the traffic and have the Front End IPs be able to directly talk to each other.
• Modify DNS so all the traffic destined to the FQDN of the Front End Server would go directly to the Front End IP which is on the same segment as the other Front End IP.
• If you must keep both Front End Servers on separate subnets and have them route through the load balancer, if possible, modify the load balancer so the requests appear to be coming from the original host that sent the request instead of the load balancer.

When it comes down to it, you just need to make sure that when 1 Front End Server talks to another, it needs to appear that it is coming from the other Front End Server instead of the load balancer so that it is an authorized host for RGS requests over 5071 TCP.

Elan Shudnow :: Oct.19.2009 :: OCS :: 13072 Comments »http://www.shudnow.net/2009/10/19/ocs-2007-r2-load-balancing-response-group-service-unavailable/OCS+2007+R2+Load+Balancing+-+Response+Group+Service+Unavailable2009-10-19+16%3A09%3A22Elan+Shudnow

I ran into an issue today where Exchange Unified Messaging’s Outlook Voice Access (OVA) feature was not playing old voice mails.  For example, when you dial into Outlook Voice Access, you will hear the following:

“You have no new voice messages and no new e-mail messages.  Please say voicemail, e-mail, etc….”  A tip here is that you can press 1 if you want OVA to revert to a touch tone (DTMF) interface.

So in our scenario, say “Voice Mail.”  OVA will say that you have no voice messages. Now here is where the problem is.  We definitely do have voice messages in our folders, but they are no longer in our Inbox.  For some users, it would play their voice mails just fine even if they weren’t in the Inbox, but for others, it would not. For the user’s that didn’t work, this is what our Voice Mail Search Folder displays.

If we Right-Click our Voice Mail Search Folder and Choose Customize this Search Folder, we may see the following:

If you were to click Browse, you would see the following:

The reason I say may, is because the “Mail from these folders will be included in this Search Folder” by default.  What I noticed was, is that when I looked at some mailboxes, they had it set to Inbox and when I looked at other, it had it set to the following which had no problems with OVA playing back the voice mails:

If you were to click Browse, you would see the following:

The key thing here, is that when you dial into OVA, it plays back your voicemails based on what the Search Folder displays.

The difference between the two, is that the checkmark was set at the Mailbox and search all subfolders, clicking on your Search Folder would find all voicemails throughout your entire mailbox.  But there’s a catch here though if your Voice Mail Search Folder utilizes the Inbox selected method for its search parameters instead of the checkmark being at the Mailbox level.

You may think to yourself, “This should work even if I only have Inbox selected and set it to search in all subfolders.”  After all, that would be a logical though since your subfolders are subfolders based off of the parent Inbox. That’s what I thought.  But it just wouldn’t work.  And it wasn’t just this one user, it was all the users who utilized the checkmark at the Inbox level even if it was set to search subfolders.  Another thing you may think to yourself is, “This should work then if I change deselect the Inbox and then choose the entire Mailbox and search in subfolders.”  This didn’t work either if you initially had a checkmark at the Inbox level instead of at the Mailbox level.  What I ended up realizing, is that I had to re-create the Voice Mail Search Folder completely for things to work properly.   It’s as if something was entirely wrong with the Voice Mail Search Folder.

The interesting thing here, is that none of these users customized this search folder but they were UM enabled at different times.  I’m thinking an Exchange patch modified the way the Search Folder parameters were configured. The reason why I think it’s an Exchange patch and not an Outlook patch, is because we saw this behavior the same with a mix of Outlook 2003 and Outlook 2007 clients.  Add in the fact the the way this Voice Mail Search Folder gets created, is when a user is UM enabled.

So onto re-creating our Voice Mail Search Folder…  During troubleshooting this, I discovered a bug that was confirmed with Microsoft which you should know when deleting Search Folders.  There is an Outlook 2007 and Outlook 2010 bug that occurs when deleting Search Folders in Cached Mode.  So in order to get our Search Folders re-created as how they really should be, I set Outlook in Online mode, deleted the Search Folder, sent the user a new Voice Mail, and saw the Voice Mail folder get re-created.  The Search Folder was set to search the entire mailbox.  When I dialed into OVA, it would play all voice mails no matter where they were stored.  I then re-set Outlook back into Cached Mode.

Once this Voice Mail Search Folder was re-created, the search parameters were displayed as such:

Now if I look at my Voice Mail Search Folder, I successfully see my voice mails stored in my Saved Voicemails subfolder:

And because my Search Folder is finding my voice mails from subfolders, when I dialed into OVA, it successfully played back all those voice mails as well.

Elan Shudnow :: Oct.15.2009 :: Exchange :: 1290No Comments »http://www.shudnow.net/2009/10/15/exchange-unified-messaging-ova-not-playing-voicemails/Exchange+Unified+Messaging+-+OVA+not+playing+voicemails2009-10-16+03%3A46%3A53Elan+Shudnow

Exchange 2010 introduced a new feature called Client Throttling.  The goal of this functionality is to ensure that no one user can tax your Exchange Server.  I’m sure many of us have had to troubleshoot Exchange performance issues and one of the tools that we utilize is the Exchange Server User Monitor (Exmon) tool.  Often, you will find a couple users who are using utilizing the system quite a bit more than other users.  We become worried and suspicious that something may be going on with this user.

With the new Client Throttling functionality, we can feel more confident of the following:

• Users aren’t intentionally taxing our Exchange Server
• Users aren’t unintentionally taxing our Exchange Server
• Users are proportionality utilizing the resources on our Exchange Server

Policies

By default, Exchange 2010 utilizes a Default Policy to manage all of the users on your Exchange 2010 Server.  Should you decide that different users need to have a different throttling behavior, you can create different policies for these different user groups.  Or should you decide that you want all users to have have the same throttling behavior but different parameters than what the Default Policy contains, you can modify the Default Policy to fit the needs of your organization.

The four PowerShell cmdlets that assist you in defining your policies include:

• New-ThrottlingPolicy
• Remove-ThrottlingPolicy
• Get-ThrottlingPolicy
• Set-ThrottlingPolicy

Exchange Servers Throttling Applies To

The parameters that are configured in a policy apply to the following Exchange functionality:

• Activesync
• Web Services
• IMAP
• POP
• Outlook Web App (OWA)
• PowerShell
• RPC Client Access Server

Throttling Parameters

The parameters that you will configure are:

• MaxConcurrency
• PercentTimeInCAS
• PercentTimeInAD
• PercentTimeInMailboxRPC

By default, out of these 4 parameters, MaxConcurrency is the only parameter that has a value specified.  It is also the parameter you will most likely configure should you have a need to modify your throttling behavior.

Each Parameter will be prefixed by each type of Exchange Service.  For example, Activesync will be EAS.  Because of this, the parameter for Max Concurrency for Activesync would be EASMaxConcurrency.  The acronyms for each service are:

• Exchange Activesync = EAS
• Exchange Web Services = EWS
• IMAP = IMAP
• POP = POP
• Outlook Web Access = OWA
• RPC Client Access = RCA
• PowerShell = PowerShell

The parameters for PowerShell are a bit different and include the following:

• PowerShellMaxConcurrency
• PowerShellMaxCmdlets
• PowerShellMaxCmdletsTimePeriod
• PowerShellMaxCmdletQueueDepth

Let’s take a look at configuring Activesync Throttling.  We can do this by running:

Get-ThrottlingPolicy | FL IsDefault,EAS*

As you can see, since we haven’t created a policy, this will show us our Default Policy as you can see by the parameter IsDefault being set to True.  For all Exchange services that can utilize throttling, you will see that only MaxConcurrency is configured.

CPUStart

It is important to note the CPUStartPercent parameter.  It is only when Exchange reaches this CPU utilization that the policy will be enforced.  If an Exchange Server is adequately sized, your user’s should very rarely ever get throttled.  But throttling is good to have in the situation where your Exchange Server does encounter high CPU utilization in the circumstances noted at the beginning of this article, intentional high utilization and unintentional high utilization which the throttling policy will ensure that users are utilizing your Exchange resources proportionality.

MaxConcurrency

Maximum amount of connections that a user can have open on an Exchange Server at any given time.  So by this being set to 10, once an Exchange Server utilizes 75% usage, the Exchange Server will begin throttling new connections.  Keep in mind that this will only throttle “new” connections.  So if a user has let’s say 15 connections open, 5 of those connections should not be disconnected.  Again, you can modify the Default Policy to contain a different amount.

Example: Modify EAS MaxConcurrency

Let’s say we wanted to modify EAS to throttle and allow 12 connections.  Our organization has a lot of EAS activity and we determined we want to bump up this value just a little bit. We would run the following cmdlet:

Set-ThrottlingPolicy -Identity IdentityHere -EASMaxConcurrency 12

In order to obtain the name of your Default Throttling Policy, run the following command:

Get-ThrottlingPolicy | Where-Object ($_.IsDefault -eq “True”) | FL Identity

In our case, the Identity of our Default Throttling Policy is DefaultThrottlingPolicy_4f681051-74b6-42e4-a751-48b71e1cce22.  Easy enough to remember eh?

This would change the above command to:

Set-ThrottlingPolicy -Identity DefaultThrottlingPolicy_4f681051-74b6-42e4-a751-48b71e1cce22 -EASMaxConcurrency 12

Remaining Parameters

As stated above, it is important to note that most of the time, you will only configure the MaxConcurrency parameter.  Should you find the need to further tweak your policies, you can visit the following Technet Articles to learn more about those parameters: here and here.  The articles will also show you how to create a new policy as well as assign the new policy to a specific user or a group of users.

Elan Shudnow :: Sep.26.2009 :: Exchange, Exchange 2010 :: 12525 Comments »http://www.shudnow.net/2009/09/26/exchange-2010-client-throttling/Exchange+2010+Client+Throttling2009-09-26+22%3A54%3A02Elan+Shudnow

There are several ways in which we can utilize Audio/Video streams in Office Communications Server.  While this article is based off of R2, the same “should… but not verified” work the same in OCS 2007 R1.  There aren’t really any places out there that describe how the media session works in different circumstances.  For example, what servers and ports are utilized when doing Audio/Video through the Live Meeting 2007 client when connected to the On-Premise Web Conferencing feature in OCS 2007 R2?  How about when you do a peer to peer with both users being internal to the network?  How about both users being external to the environment and connecting through the Edge?  How about when you do a peer to peer with one user being internal and one user being external?  Want to know?  Read on…

Media Ports and Restricting Amount of Ports Being Used

The first thing to understand is that in OCS 2007 R2 (the same applies for R1), when a user attempts to activate any type of audio and/or video, they first attempt a peer to peer session.  The ports utilized here are TCP/UDP 1024-65535.  You can see what ports are used for OCS 2007 R2 here and here.  This port range is utilized mainly for users who are internal to the network.  If you want users to utilize peer to peer audio while internal to the network, you must ensure that this port range is open even if users are in different sites.

But what if you don’t want this entire port range open between your sites?  You can utilize Group Policy to limit the amount of ports that are being used.  These two settings include:

• PortRange/MaxMediaPort
• PortRange/MinMediaPort

The above group policy settings modify the following three registry keys:

• HKEY_LOCAL_MACHINE\Software\Policies\Microsoft\Communicator\Portrange\Enabled REG_DWORD 1
• HKEY_LOCAL_MACHINE\Software\Policies\Microsoft\Communicator\Portrange\MaxMediaPort REG_DWORD 40039 (for example)
• HKEY_LOCAL_MACHINE\Software\Policies\Microsoft\Communicator\Portrange\MinMediaPort REG_DWORD 40000 (for example)

Note: Both clients must have these registry keys set in order for the modified port range to take effect.

You must have at least 40 ports open which is an IETF Interactive Connectivity Establishment (ICE) protocol requirement to ensure that Audio/Video port negotiation works for peer to peer while internal to the network.  ICE is a protocol that provides a mechanism for firewall or NAT traversal.  The RFC draft for this protocol can be found here.

Audio/Video Connectivity Scenarios

Two Users Internal to the Network (media ports open)

When these two users are internal , they will attempt peer to peer.  Because they can successfully connect to each other, they utilize peer to peer media.  This is why OCS scales pretty high; because a lot of connections are one to one which means that peer to peer media connections are never bridged through the server.  Because these users connect directly to each other for media, they have no need to connect to the Edge.

Note: The red arrow signifies a successful media connection only.  The diagram does not reference any other signaling such as SIP.

Audio/Video through the Web Conferencing Server

When users are connected through On-Premise Web Conferencing and activate Live Meeting (when internal or external… doesn’t matter), they are connecting directly through the Front End’s Conferencing MCU’s.  Because of this, even when it’s two users, the user’s are still connecting to the Front End MCUs.  If both user’s are external, they still connect through the Front End MCUs but are proxied through the Edge Server.

Note: The red arrow signifies a successful media connection only.  The diagram does not reference any other signaling such as SIP.

Two Users Internal to the Network and Any Users External to the Network

As previously stated, any time you have more than two users, peer to peer is no longer utilized and users always connect directly to the MCU on the Front End Servers.  This means that both users internal to the network will connect to their Front End server(s) and the external user will connect to the Front End server as well utilizing the Edge Server for proxying to the Front End.  There is absolutely no peer to peer connectivity in this situation.

Note: The red arrow signifies a successful media connection only.  The diagram does not reference any other signaling such as SIP.

Two Users on the Internet

When these two users are external , they will attempt peer to peer.  Because they can successfully connect to each other, they utilize peer to peer media.  This is why OCS scales pretty high; because a lot of connections are one to one which means that peer to peer media connections are never bridged through the server.  Because these users connect directly to each other for media, they have no need to connect to the Edge for Audio/Video.  You will still see the user connected to the Access/Edge over port 443 and/or 5061 (if these are your remote access port and federation port if you are using federation).  When users are connected through On-Premise Web Conferencing and activate Live Meeting, they are connecting directly through the Front End’s Conferencing MCU’s.  The Front End will have a certificate that contains the Pool Name and will/can contain SAN names for additional SIP domains that you may contain.  Because of this, SAN names are supported on Front End Servers.

Note: The red arrow signifies a successful media connection only.  The diagram does not reference any other signaling such as SIP.

Two Users Internal to the Network (media ports closed)

When these two users are internal , they will attempt peer to peer.  During their ICE negotiation, as previously stated, they will know the Internal Edge NIC in case their peer to peer connectivity fails. Because they fail to connect to each other, they will connect to the internal Edge NIC over either UDP 3478 or TCP 443.  UDP 3478 is tested first during ICE negotiation, and if that fails as a candidate, ICE will return TCP 443.  If you anticipate on blocking ports between your users, make sure your Edge Server can scale high enough to deal with the amount of Audio/Video connections it will be handling.  To block one of your sites from doing peer to peer with other sites, block the peer to peer port range (discussed at the beginning of this article) from that site and block that site from communicating over UDP 3478 and TCP 443 to the Edge Server.  This will prevent clients from doing any type of media communication from user’s outside of their own site.  If you want to allow them to do peer to peer for users in some sites, modify the firewall ACLs accordingly for those sites.

Note: Diffserv markings for Quality of Service (QoS) are lost through an Audio/Video Edge Server.

Note: The red arrow signifies a successful media connection only.  The diagram does not reference any other signaling such as SIP.

One User External and One User Internal

When one user is internal and one user is external , they will attempt peer to peer but not in the same sense as in two internal users. The external user will hit TCP 5061 to the Access Edge Server and will be provided with either UDP 3478 or TCP 443 for the Audio/Video Edge.  As stated earlier, UDP 3478 is preferred, but if that fails during the  ICE negotiation process, TCP 443 will be used as the ICE candidate.  If you attempt a telnet edgeserver.domain.com over the Internet, telnet will fail to connect.  This is because telnet uses TCP.  You can do a netstat -an to see your server listening on UDP 3478 and utilize a different program such as netcat which can attempt telnet to UDP by using netcat -u host 3478.  More information on netcat here.

Moving on… we see that the user will connect to the A/V Edge over UDP 3478 or TCP 443, but what about the internal user?  Because this is technically peer to peer, the internal user will NOT connect to the MCU on the Front End but will instead connect directly to the A/V Edge Server’s Internal NIC over UDP 3478 or TCP 443 as well.  The Front End A/V MCU will not be used in this scenario.  When you add a 3rd person to the conversation, the external user will connect to the Front End Server’s A/V MCU in which the A/V Edge will proxy this data for the user, and the internal users will connect to the Front End A/V MCU instead of the Internal Edge NIC.

Note: Diffserv markings for Quality of Service (QoS) are lost through an Audio/Video Edge Server.

Note: The red arrow signifies a successful media connection only.  The diagram does not reference any other signaling such as SIP.

Issue to be aware of

Certain certificate vendors like to add www automatically to the CN of your certificate request with consent from the person requesting the certificate.

So for example, our certificate request for a regular certificate (non SAN) was CN=Servername.domain.com.  These vendors provided the following certificate:

• CN=Servername.domain.com
• SAN=www.Servername.domain.com

Now this is where you can run into an issue when doing peer to peer and falling back to the internal Edge NIC or when one person is on the Internet and one person is internal while using Communicator.  Some certificate vendors like to assign a www as a SAN name to your regular (non-SAN) cert.  So if you requested a CN of Servername.domain.com, these vendors will automatically add www in front of your CN.  This messes up peer to peer audio/video negotiation.

This is important to remember when getting a 3rd party certificate for the Internal Edge NIC.  It is not supported to have anything other than a CN for this certificate.  The reason why is ICE negotiation will not work properly.  Because the www.servername.domain.com will be there as a SAN, the client will be provided the www.servername.domain.com name for ICE connectivity to the internal Edge NIC which will obviously fail.  Because of this, the scenarios above which utilize the Internal Edge NIC will fail.  To recap, the following two scenarios end up using the Internal Edge NIC:

• One user internal to the network and one user external to the network
• Internal users attempting to do peer to peer (media ports being closed) and falling back to the Internal Edge NIC

There are a few ways to fix/workaround the problem:

• Use a Windows PKI implementation for Internal Edge NIC
• Deal with the SAN name but use a CNAME for www.servername and redirect that to the servername HOST/A record using Internal DNS to do this as you are dealing with the Internal Edge NIC.
• If the vendor doesn’t add www to their SAN certs, you can get a SAN cert only with the CN filled out
• Go with a different vendor that doesn’t automatically add www to their regular certificates.

Elan Shudnow :: Aug.29.2009 :: OCS :: 120326 Comments »http://www.shudnow.net/2009/08/29/office-communications-server-2007-r2-audiomedia-negotiation/Office+Communications+Server+2007+R2+Audio%2FMedia+Negotiation2009-08-29+22%3A27%3A30Elan+Shudnow