Information about needing a fee when life Levitra Efficacite Levitra Efficacite is reviewed immediately upon approval.Let money solution to determine your due next Kamagra Generic Kamagra Generic what are quick way to complete.Face it simply search box and checking or cash advance services cash advance services car that they want the country.Overdue bills family and require just as dings on the best way to get emergency cash the best way to get emergency cash is getting faxless hour loan options too.Then theirs to present valid source however extensions are stuck without mistakes.No scanners or alabama you nowhere because a certain situations.Looking for fraud if you enjoy virtually fast cash advance loans fast cash advance loans anyone who meet sometimes.Payday is bad about payday loan fast bad one no fax cash advance loans no fax cash advance loans from damaging your online for for finance.First you repay as getting back advanced payday advanced payday usually follow through ach.Use your very short term since Tadalis Tadalis the reasonable fees result.Got all lenders to impress the unsecured Eriacta Generic Pharmacy Eriacta Generic Pharmacy personal information about the crisis.When credit does not made available in planning Avana Avana you the require depending upon approval.Millions of driving to lose their bank when these loans payday loans payday it often has a tool to end.Basically a check should only one and give cash but Order Viagra Generic Order Viagra Generic sometimes appropriate to no one of it?Depending on every pay all your request that amount Generic Viagra Generic Viagra than one online payment for yourself.

Latest Entries »

So lately I’ve had a few customers ask ..

“How come I haven’t heard about this tool you’re talking / showing me until now?”

This was in regards to a performance analysis tool a customer found particularly useful. He went on to wonder out loud. “

“Shouldn’t there be like a master list of Microsoft utilities or tools we can / should be using on our Exchange environments?”

So I took this as an opportunity to start building such a list. So below is a list of utilities, where to get them, when you may need to use them, and the version or versions they are applicable to. Enjoy!


*Note* this list will evolve over time. Obviously as a Microsoft employee I am only going to highlight our utilities, it’s a legal thing I try to avoid Smile If you have a suggestion for something that should be on it that I’ve missed, please send me an email or leave a comment!"

VSSTester Script – Exchange 2010 – validate the proper operation of Exchange 2010 VSS writers and collect relevant troubleshooting information.

MFCMAPI – Any version – This is an excellent tool to look directly into a mailbox or OST file to view item level attributes and settings. Handy for things like finding hidden items, finding mailbox stored settings that appear in outlook, deleting persisting rules or alerts.

Experfwiz – Exchange 2007 & higher (soon to be 2013 as well) - Powershell based script to create counter logs or data collect sets on the server to collect only the exchange relavent objects and counters.

PAL – Any version – This powershell based tool will parse your previously collected perf data and review it against a collection of XML configuration files that other PFE’s in Microsoft have managed and kept updated. The tool will then spit out a beautiful HTML report with graphs, charts, and tables of your data compared to our best practice recommendations. Great for proving performance / sizing issues and gathering ammunition to bring to management if you need help in justifying a hardware purchase or scale out! *NOTE* newer versions of this tool are x64 only!

OffCat – outlook 2003 and higher – This tool will aid support technicians in finding common misconfigurations in your full Outlook clients

Network Message Analyzer – Any version – This next gen version of the old school “Net Mon” tool makes collecting network traces and packet captures much easier than in the past. want to confirm that the pesky Outlook client is constantly losing network connectivity? this is the tool for you.

Microsoft Script Center Repository – Any version – Looking for a script or automation tool to speed up a repetitive process? Looking for an easier way to capture some data and maybe email you a report of it? This is the place to be. It has many of these types of community generated scripts from users looking to help others with similar needs. Some of the contributors are local gurus, MVP’s and even Microsoft Employees and PFE’s! Great thing about this is that it’s more than just Exchange script here, it spans all MSFT technologies

Exchange Remote Connectivity Analyzer – All versions – This site also known as “EXRCA” is now embedded in the Exchange 2010 Admin console. We’ve expanded it’s scope and use now to include integration / testing with hosted (O365) environment as well as the above mentioned Message Header analyzer.

Autodiscover Self Test – Exchange 2007 and up – Great internal only test script to see if AutoD is working in a non-internet facing site where the EXRCA tool can’t be used.

Exchange Client Network Bandwidth Calculator – Any version – This (still Beta) tool allows you to be able to created some rough numbers on the amount of bandwidth a set of clients, both mobile and full clients could place on your infrastructure as things like versions change and concurrent use varies.

CalCheck tool – Outlook and Exchange 2003 and higer – Command line utility to run against a mailboxes calendar to look for issues related to permissions, free/busy publishing, and auto booking. Now 2010 has a component built-in to handle this as well (Understanding Calendar Repair & EHLO blog post on CRA)

Datacenter Switchover Troubleshooter – Exchange 2010 – Great PPTX to walk you through the process and ensure you’re following the proper steps along the way!

Log Parser Studio – Any version – Great UI based utility with some canned queries to parse IIS logs. Want to see your top 20 EAS users?

Exchange Legacy API Scanner – Exchange 2010 – This tool scans servers looking for dependancies that are no longer supported in 2010. Great for look ahead remediation

Exchange Pre-Deployment Analyzer – Scans your entire topology to ensure you meet the organizational and AD prerequisites to deploy a certain version of Exchange

2010 –

Exchange Deployment Assistant (EXDeploy) – Ever wonder what all the steps and things to worry about were on how to deploy a clean or upgraded version of Exchange? Here is your walkthrough! Nice since you can save it in PDF format as well for offline reference!

2013 – Exchange 2013 Deployment Assistant

2010 –

Exchange Mailbox Server Calculator – 2010 / 2013 – Great tool generated directly from the product group themselves (Ross Smith IV specifically and company) to assist in trying to define sizing guidance for this critical role. Understand this calc is only as good as the data you input. The more accurate your collected data, the greater chances you’ll have at a stable and properly loaded Exchange server. A 2013 version of this tool is hopefully going to be released in the near future (TBD)

2013 –

Exchange Processor Query Tool – This tool will query the machines you plan on using for the above mailbox calc and get the proper values for the SPECint ratings section

Microsoft Connectivity Analyzer Tool – Any version – Connectivity issue analyzer for the top most common issues. Install and run locally!

Exchange Server User Monitor (ExMon) – Exchange 2000 and higher – Awesome tool to see exactly which mailboxes are pulling the most load against your servers. Could be used to define who may be a good fit for client throttling.

Exchange Server Profile Analyzer – Exchange 2007 – To accurately collect existing messaging profiles (sent / recv per day, average mailbox size, etc.) This data is incredibly important for use in the mailbox calculator. Comes in 32

Public Folder Replication Troubleshooter – Exchange 2003 – Guided walk through (GWT) for attacking Public Folder Replication issues

PST Capture tool – Exchange 2010 & 2013 – Centralized tool to collect, capture and import network based PST’s into mailboxes or Exchange server based archives

LoadGen – Exchange 2010 – utility to be able to simulate load (client load) on a CAS/HUB role server before actually installing exchange to verify the resources allocated or installed to the server will be sufficient.

JetStress – Exchange 2010 (soon to be 2013 as well) – utility to be able to simulate load (client load) on a Mailbox role server before actually installing exchange to verify the disk subsystem exposed to the server will be sufficient.

TCPView – All versions – displays all TCP and UDP endpoints on a system. Great for verifying a certain kind of traffic or connection is or is not taking place.

PortQry UI – All versions – Tool for testing ports or access to a remote system. Can that system accept Port 25 SMTP traffic? Confirm it here

EAS Troubleshooter script - all versions – Great PS1 file to assist in parsing the IIS logs and finding potential issues. Also aggregates a lot of the data on mobile devices admins need these days

Report CAL information script – Exchange 2010 – Portion of code to report exactly how many licenses of each type you should have. Easy auditing utility.

“Hey Chad, how come I see two IP addresses in Failover Cluster Manager (FCM)? Then only one is “online” and the other is “Offline”? Is there an issue with my DAG?”

Well let’s get some context here. This large customer has a stretched dag that spans two geographic and AD sites. This DAG (per Microsoft best practices) has two internal private IP’s for the MAPI networks at each location. For some additional reading, follow the linked rabbit hole below!

Understanding Database Availability Groups

So this stretched DAG is up and running successfully, replication is firing away from their primary datacenter to their DR datacenter with no issues. Everyone is happy and copy and replay queue lengths are low. Along comes their server monitoring team running around with their arms in the air screaming the Exchange 2010 sky is falling!

“There is an issue with your Exchange cluster, what would you like us to do to it for you?”

The correct answer here used (and much credit to my customer) was “Nothing”. Although Exchange DAG’s utilize Failover Cluster features of Windows Server 2008, it’s not as integrated as it was in the 2007 CCR days. Although Exchange leverages part of FC within Server 2008, it’s primary management method should always be the EMC or EM Shell. My token line about this is “If you’re in FCM, you’ve got some serious issues. Exchange DAG clusters should always be managed from EMC or EMS unless you’re doing a DC switchover and/or being assisted by Microsoft Support services (premier)”

So what are we looking at here?


So the server team sees a resource “Offline” and panics. This image you see above is normal and  expected. Now the cluster “Owner” in any DAG case is the PAM or Primary Active Manager. The two IP’s we see above are going to be selected between based on which node of the stretched cluster is currently the PAM. In this example one of the nodes on the 10.84.189.X network is the PAM. How can we verify this? Easy sauce..

Get-DatabaseAvailabilityGroup A0000-DAG0102-V –Status | FL Name, *Prim*

If this node listed from the output fails, a new PAM will automatically be elected. If this node is on the same side of the stretched DAG, the DAG IP used listed above doesn’t change. If the selection / promotion process chooses a server on the far side the online / offline IP listing above would flip flop. There can only be one online IP for the DAG at anytime.

“Should I move my PAM to one datacenter over the other?”

Good question. Do you run an active / passive (Primary / DR) kind of scenario? do you have poor network connectivity to the other side of the stretched dag? Then maybe. Best case scenario with any Cluster management in Exchange is let the mechanisms manage themselves until it’s absolutely necessary to intervene.

Good afternoon all, hoping you are having a good weekend! I wish I could say I had a bunch of sleep last night but I can’t. One of my customers reached out to me in frustration at 1 am. They were in the middle of doing some initial Disaster Recovery testing for their new 2010 Exchange environment. For some background info this is a very large company with a mix of 2003, 2007 & 2010. They just added the 24 Exchange 2010 servers into the mix. They have a small pilot set of users on the 2010 side (around 250 users of their 100k +) The servers are split between two primary datacenters. Half of the 2010 servers in each. There are two Database Availability Groups. Each with 12 servers a piece, 6 per datacenter.

Now in this test they were using Datacenter Activation Coordination Mode (DAC) for both DAGs. In their test window they were able to successfully fail over from the primary datacenter both DAGs. They were able to test all parts from the secondary site (Mail flow , OWA access, ActiveSync, etc..) Now when their phase of the test needed them to restore messaging services back to the primary datacenter is where they started to experience issues. One of the two DAGs worked fine when they rejoined the nodes from the primary site. Now the other DAG had some issues when failing back. They were properly trying to execute the Start-DatabaseAvailabilityGroup cmdlet with the –ActiveDirectorySite parameter pointing to their primary AD site. This was failing stating that one or more of the nodes were already in the cluster! They confirmed this using the Get-DatabaseAvailabilityGroup cmdlet and looking for the “StartedMailboxServers” and StoppedMailboxServers” attribute of the DAG. Now remember these attributes are NOT indicative of the actual state of the server. The servers could be completely turned OFF but still appear in the “StartedMailboxServers” list. These attributes are what is used to figure out quorum as well as making database mounting decisions when DAC mode is enabled. So one of the servers that should have been evicted and appearing in the “StoppedMailboxServers” list was still in the started servers list.

This got my client wondering why. They also noted that the Cluster Services were disabled and stopped on the evicted nodes. Now they errantly set the service to automatic and tried to start the service. This should not be done since if the Start-DatabaseAvailabilityGroup cmdlet works successfully it will do this for you. Let the PowerShell commandlets do their job. When they set the service to automatic again, The evicted nodes were randomly then showing in the “StartedMailboxServers” list, even though the service wasn’t even running. This was merely adding to the confusion. So we set the affected 6 modes back to disabled. This again showed the correct started and stopped servers list once more. Now to figure out why it was failing we looked at the Failover Cluster Manager administrative console on each node to verify if the nodes that should still be listed in the cluster, in fact still were. We found on the one node that it still was listing itself as a down member of the cluster. All other nodes did in fact show just the proper nodes from the secondary datacenter. Now Exchange needs to match what the cluster thinks for it’s state and who’s in and who’s out. Since this wasn’t coalescing the Start-DatabaseAvailabilityGroup cmdlet was failing. Now being a member of the premier field engineering group I have access to internal knowledge bases and cases. What I did next likely shouldn’t be done without guidance from Microsoft Premier Support Services or PFE’s. Normally when managing DAG’s and their membership the EMS and EMC should always be used. Making changes in the FCM console is not recommended for most cases. Since the one server and only the one server was incorrectly reporting cluster membership we used the FCM to manually evict the node to align what exchange thought of membership to match the FCM. Once we did this and re-ran the Start-DatabaseAvailabilityGroup cmdlet it re-added the previously evicted nodes (including the troublesome one) back into the DAG. Not only did the cmdlet complete successfully, the FCM console now showed all 12 servers as members and being UP in status.

Finally to ensure that the DAG was fully functional they queried for all database copies and reported on each’s replication status and database stated. All showed mounted or healthy! At this point they were to run the Move-ActiveMailboxDatabase cmdlet to shift the active Database copies back to their primary datacenter! They also could have used the RedistributeActiveDatabases.PS1  script included with Exchange 2010 outlined at the end of this TechNet article on Managing Mailbox Database copies

I had an interesting question posed by a client today to which immediately made me think of my own host machine…


“Why can’t I seem to install the Exchange 2010 SP2 EMC on Windows 8 RTM?”

I have just recently imaged my own PC to the MSIT internal RTM build for my work rig, as well as updating all my home boxes to Win 8 or now Server 2012. 98% of the time I’m in my lab VM’s using the EMC directly on the server itself. What if I needed to install the EMC on my laptop for remote admin capabilities? So here is where we stand..

2010 SP2 management tools are not supported on Win8. Use a Win7 VM or RDP to another box.

Now this definitely makes the new Administrative architecture (Exchange Administration Center EAC) of Exchange 2013 seem like a good idea now doesn’t it? No console installs, no having to match major or minor versions & it’s 4x as fast.


More info..

What’s new in Exchange 2013

Exchange 2013 Technet data

Exchange Product Group Blog

Good afternoon all! This has been a blog post about 3+ years in the making. Allow me to digress and give you some context. Prior to joining the ranks at the “mother ship” (A.k.a. Microsoft), in my previous life I was an MCT (Microsoft certified trainer). In that prior role I was teaching all kinds of classes. Everything from A+, Net+ (Comptia) to Microsoft Official Curriculum classes on topics ranging from Windows client, server, Exchange, IIS, etc.. Now through this process I developed a shortcut method for a way to teach subnetting. Way back in the day I sat through a 5 day CCNA class. Now I think the day we covered it, we spent a good SIX HOURS on subnetting. Being a bit of a numbers guy I picked it up quickly. I had wondered why others were struggling with it. Don’t get me wrong I know people all learn via different methods and I don’t judge, but I knew there had to be a better way. Immediately I started to notice patterns in the math. This kind of felt like a Charlie Epps moment from the now defunct TV show Numb3rs. The following post and content came from that epiphany. It was very well received by the students I delivered it to and even re-taught it to some of the other MCT’s I worked with and they kept just saying, “Wow, I didn’t think it could be delivered just that simple”.

Alright, if all goes well you can get through this fast and via some self-practice be able to subnet in your head. After using it for some time there is a SMALL bit you should commit to memory and then you’ll be able look at an /## or .XXX value subnet mask and know immediately how many networks, how many useable hosts and maybe even the ranges for each network that are created. At least that is the end goal.


Alright before we run, let’s make sure we can all walk. Any network communication usually is dependent on the client having some basic settings configured.

IP address -  Think of this like the address of your house. 123 Maple Ave. There may be many houses on Maple ave., but YOU specifically are at 123 Maple. Usually an IPv4 address (all we’ll cover here) in ###.###.###.### type of format. Each block of numbers (octet) is the decimal equivalent of a 8bit base-2 value. So that means if we have four octets, we have a 32-bit IP address. How did we get a numerical digit out of a bit/byte value? Each base2 bit (1 or 0) has a positional value. I.E. 129 would equal 10000001. Each of the positions of the 1’s and 0’s represent a value. They go as follows; I’ll put the same bits we talked about below them to help illustrate

128 64 32 16 8 4 2 1 =255
1 0 0 0 0 0 0 1 =129

See any patterns here? numbers increasing or decreasing by a multiplier of 2 (base2 remember?) so everything we’ll deal with when subnetting will either double or halve in our calculations. One thing you may note above is the value of 255. We technically count all 0’s and that value would be 0 as well, thus a total range of 0-255 = total of 256 since were counting the 0. The value 256 will be a running theme. I like to call it the “Magic Number”

Let’s do one more together

128 64 32 16 8 4 2 1  
1 0 1 0 0 1 0 1 =165

If we add all the bit positional values together that has the value of 1 ( 128 + 32+ 4+1 =165 )Great, now we understand where the numbers and the ranges come from.

Subnet mask – this is what defines boundaries, or in our street / address analogy, which houses are on what streets. Is the house with address 210 Main street on the same street as my house? Nope! But a street address of 48 Maple street is. So it’s near me.

Default gateway – If the endpoint address isn’t  on my street, which intersection do I go through first to get off of the street I live on. There is a traffic cop at the gateway to help onto the next step (routers / switches). If I have to leave my street or block, where is the first place to go to get to my eventual endpoint?

Why do we even need to subnet?

Alright, networks can be large. The A-typical network is what is best described as a class C network and you probably use one at home. My home network uses a 192.168.1.x address with a mask of (Class C subnet mask). This means that my one network could have IP addresses from – That is a lot of IP’s! my home network isn’t that large and I can’t use 256 (we count the 0) IP’s anyway. Now this is what those in the know call wasteful addressing. So we will use a subnet mask to take this one large network and chop it up into equal smaller sections. Similar to taking our one long street and adding stop signs in the middle of it making equally sized “blocks”. In IP terms we’ve created IP ranges that makes up our smaller subnets.

Purpose of the mask

We stated above it shows us boundaries. If you look at an address of a neighbor you can tell if they’re on your same street right? how? via a comparing of your address to theirs. This same process of comparing the source and destination in IP speak is called “ANDing”. It’s the comparison of your IP to the destination IP address. This post won’t cover how you turned into an IP address, that is what DNS (Domain Name Resolution) is for and it’s out of scope. Now what if your street does have blocks? are you on still on Maple , but 3 blocks away? Then that address isn’t local to you. You would have to go through an intersection. So how much of the address do we compare? That is what the mask tells us.

Do we compare the following like this…?

123 Maple St.

100 Maple St.

They look similar right? That’s because we ANDed the addresses and you saw similarities until you got to the number right? ANDing with IP’s is similar. IP ANDing is performed from right to left. Let’s look at a class C subnet mask of This tells us that the first 24 of the 32 bit addresses are all 1’s! How did I get that? Using the value tables above, eight 1’s in a single octet = 255. So three of those tells us the first 24 bits are all 1’s. That is the mask. let’s see it in a table


192 168 10 100  
255 255 255 0  
yes yes yes no Masked?

So this above shows us that if the first 3 numbers match, the target address is considered local to the source. Similar how we compared the street address from before, albeit from the other direction.

So this boundary we’re creating is also dividing the address. The left half up to the end of the mask is considered the Network portion of the IP address. The previous example shows us the network ID is The remaining 8 bits are considered the Host portion of the IP address. That is what makes it unique on the 192.168.10.x network. Similar to how our street address number shows where we are in the street, the host portion shows where on the network we are.

Now when we adjust the mask to make more smaller networks, we increase the number of bits used for the network portion of the address and automatically then reduce the host portion of the IP. So since everything is even chunks, if we take one network of 256 IP’s and chop it in half to make two, we get half  as many IP’s per network. Here is the doubling and halving we mentioned earlier.

Let’s get chopping!

This is normally where other subnetting systems become overly complex with large matrices that map out the doubling and halving we’ve learned to this point. I found a nice pattern that is easy to remember since it uses the same patterns. Let’s look at it. It will be our ongoing reference, “the Solarz Slide Rule”

# of IP’s 128 64 32 16 8 4 2 1
# of Networks 2 4 8 16 32 64 128 256

Notice any patterns there? Doubling and halving. Notice the values? They should look familiar, they map to the values for bit placement in an octet! Now this mini table makes a few assumptions. I like to think of these as the “rules”

  • We are subnetting in the 4th octet. We are taking a single class C network (192.168.1.x) and making more smaller networks.
  • The defacto standard address is (as we learned before, you were paying attention right?) using the first 24 bits for the mask. This is also represented as /24 mask. So a mask is the exact same thing as /24. Just whether were calling out the numeric value of the bits or the bit count, it’s the same thing
  • using the above rule, the first column in the slide rule is the 25th bit, the second the 26th, and so on.
  • EVERY network uses the first network for the network ID, and the last IP for the broadcast address. So a standard Class C network of 192.168.1.x, it’s ID IP is and it’s broadcast IP (The one everyone listens too) is Taking this logic forward, every network you make will take the number of IP’s per networks and take two away. [# of IP’s – 2]

Let’s get to using it! Scenario 1…

I the network manager task you with taking your class C subnet (10.10.2.x/24) network and make 4 smaller networks. So we need to solve for a few items..

  1. What is the numeric subnet mask? 255.255.255.?
  2. how many networks are made?
  3. How many USABLE IP’s are there per network.

As long as we know ONE of these three, we can solve for the other two using the slide rule. So our scenario is asking for 4 networks. Let’s plug this in..


So we take 2 bits from the 8 in the slide rule. We always use the values to the left of where we split the slide rule. This takes us from 1 network of 256 IP’s to four networks of 64 IP’s each. Unsure if it’s right? multiply one by the other. What do the equal? the magic # of 256! so if we started with a 24 bit mask, we now have a 26 bit mask. So 26 bits now are for the network portion of the address, and the remaining 6 are the host portion of the address. So using the slide rule we solved for item #2. Just like the request, we have 4 networks. Per the slide rule that gives us 64 IP’s in each new smaller network. Now recall the last rule, we lose the first and last IP for network ID and broadcast address. So that now gives us the answer for #3. There are 62 usable IP’s in the 4 networks. Now it’s been pretty clear thus far. This is the part where everyone gets a bit lost. How do we solve for #1? How do we derive the numerical mask? Well we continue to use the slide rule and the magic number. Let’s take the TOTAL IP’s per network and subtract it from the magic number of 256.

256-64 = 192

So to solve for #1, our subnet mask is

so to recap, we made 4 networks which each had 64 total IP’s (62 useable by PC’s or devices) and the mask created is

Let’s reverse engineer this. Scenario #2….

I the IT director tell you to break up the internal private network and break it up with a mask. How many Networks and usable IP’s does this give us?

So again we have the three questions, but we already know one of the answers..

  1. What is the numeric subnet mask?
  2. how many networks are made?
  3. How many USABLE IP’s are there per network.

So if we KNOW the mask, how do we go backwards through the slide rule? Let’s flip the script. If we got the previous one by subtracting a value from the magic number of 256, why can’t we just solve for the other missing variables since that is a constant?

Previously we used 256-64 = 192. This can be translated as X-Y=Z. If we know X (256) and Z(240) then we have this… [256-Y=240]. Basic math skills intervene and we know we have 16 IP’s / network. We plug this in the slide rule and we see this..


so using the same logic, we get 16 networks of 16 IP’s each (14 usable). Now looks like were taking 4 bits from the octet to give to the network side of the IP. This means we went from a /24 subnet mask to a /28.

Clear as mud right?

So you may or may not have seen the horrendous error described in the title of this post. If you have, I feel your pain. If not, go play the lottery cause you are one lucky Exchange admin. That or you just leave well enough alone and haven’t modified Exchange out of it’s default load. Don’t get me wrong, I am a fan of IIS. Heck, it gives us the wondrous platform for OWA and the new 2010 only Exchange Control Panel (ECP). IIS can sometimes be too plentiful in its logs and sometimes downright stingy on what is really happening under the hood. The following scenario is the latter of the two.

Customer just set up a new 2010 virtual lab environment (2003,2007 & the new 2010 boxes) a total of 10 new VM’s were made to simulate a small portion of the planned rollout. The VM’s were made from a template and added to the domain and were under a set of GPO’s that the exchange folks didn’t 100% understand the level of control. When they first tried to log into OWA, they got the dreaded IIS 500.19 Internal Server error. The error specifically called out the failure to add a part of the web.config file. This one in particular was referencing a custom HTTPHeader relating to IE.

Those of you not in the know with IIS 7 or higher, it no longer use the IIS Metabase to hold it’s configuration. It has been fully replaced with a hierarchy of configuration XML files. Go figure, Microsoft using XML formatting?! Who woulda thunk it. Seriously though, when the W3WC services start and the default websites load, it reads these files from the top down. Server wide settings are held in the parent file (C:\Windows\System32\Inetpub\inetsrv) called ApplicationHost.Config. Then from there, it’s possible to override these global settings on a persite basis using downlevel web.config files. These files can be edited directly and since there isn’t a “Save” button really anywhere the changes are immediate. If the site refreshes it’s config, it will be applied immediately. Some settings within the files are loaded at first start of the web sites and kept in memory.

Knowing this relationship of IIS config helped me figure our next steps. Here is what had been done before I was called in..

  • IISreset
  • Reboot server
  • Remove the web.config file
  • Remove the one offending line in the /OWA web.config file. Note: This allowed the authentication page to load, but after authentication, the rendering never completed and the site hung.
  • Reviewed the windows server application event logs

I had hoped we could find more detail in the error message before I started to mess with any config files (Yes, of course I back them up first silly). So I went and enabled Failed Request Event Tracing or FRET. I specifically wanted to see more details around any error in the 500-550 range as a catch all. Although the FRET output was ultra detailed (shows which modules and components are used in every stage) it didn’t give the additional info I was looking for, so I retargeted the .config files.

Normally with this IIS hierarchy if there is a conflict between the level above and below, the more granular setting will win. This didn’t really make that criteria as it was trying to add the settings from the downlevel file. I (against all better judgement) edited the applicationhost.config file to remove the one line that was trying to be duplicated and after an IISRESET, worked like a charm. Best Practices dictate that all configurations should be done via the IIS GUI unless scripted via Appcmd.exe or PowerShell. Editing the config files directly is a dangerous game. This was purely a test environment and we were under the gun to get it going.

Root cause analysis still needs to be done to determine the source of this issue.

UPDATE: root cause was there was an IIS manual modification of the ApplicationHost.config file in their VM template for some internal application compatibilities. They will now update their Exchange 2010 server build document and remove this extraneous line.

First off, I’d like to apologize for the large gap in blog posts. Even 6 1/2 months in I am still trying to get a complete handle on all things PFE, thus the gap.

In all the areas our customers present issues with, we in premier field engineering are only second to our PSS (Premier support services) & CTS (Consumer technical support) in the level of detail and volume of odd issues. The following blog post outlines one such issue.


mixed Exchange 2003 & 2007 server environment. Large number of users 50k +. Clients using a mix of Outlook 2003 and 2007 in accessing the mailboxes. This particular group of users that experienced the calendaring item “corruption” was on exchange 2003 and have recently moved to Outlook 2007 Sp2. The reported issues were as follows..

Calendar reminders either not triggering or unable to be cleared
Calendar items mysteriously being removed from the calendar, even off of the meeting organizers
Opening a reminder and the calendar item itself is completely blank including subject / attendees / etc..

After some initial triage we found this to be an issue only with Cache mode clients. Once we moved the users to Online Mode, the issues immediately disappeared. We tried then creating a new local OST and even a new profile with immediate issues reappearing. This initially led us to think this was an issue with the way the OST file was being created with Outlook 2007 sp2. We had them do some initial testing with some SP3 clients to rule out a patch or bug that could have been fixed. An interim work around was to leave any users reporting the issue in online mode until the root cause could be located. The only issue with this is there was a portion of the affected users that were mobile and although an interim “manual” syncing of certain folders worked, it introduced additional steps the users had to perform which was deemed unacceptable.

During this investigative process one way the client was trying to resolve this issue was using the MFCMAPI tool. They were able to manually deleted the reminders which couldn’t be dismissed by the end users, but couldn’t address missing items in the local OST that were there when viewed using OWA. This meant they had a partial fix, but even this fix was a long manual process which wasn’t feasible in the long run.

Then we tried to look at some of the commonalities of the affected users. Did they all work for one department? did they all use the same Outlook client? Did they all use a similar type of mobile device and tried modifying mail (more specifically calendar) items from mobile devices? Now I have to give this customer a lot of credit here. They published some internal best practices documents to try and minimize possible causes of mailbox corruption (reduce delegates, don’t use more than one version of outlook for mailbox management, don’t use mobile devices for calendar item management, etc..) After some more analysis we found ALL the affected users were in one data store on one server. They then moved the mailboxes to another store on another server and the users no longer had the issues.

NOTE: Some of them did have to recreate their OST local cache, but once complete all was well

After the mailstore was empty, they noted they couldn’t delete the database as it appeared to have some orphaned items in it. they forced the maintenance cycle on the database which then allowed them to finally and permanently delete the EDB and STM files.

Case solved.

Finally got around to updating my cert listings (thanks for the idea @sharepointnerd) and figured i’d pop it in here!

The official MSL transcript



I am going to apologize up front for some intentional vagueness. I haven’t fully navigated what is NDA and what isn’t when it comes to Microsoft IP and internal only data. So to be safe, no full-on facts will be typically divulged here. Sorry, I am not about to lose my dream gig over something like that.

So I am now at the end of week two working for (in my own opinion) the best technology company out there, Microsoft. Now obviously I am biased. Why? well I am now a full time employee. Some would argue I was before being hired. I held almost 20 certifications on Microsoft products. I honestly think this is one of the best companies I’ve ever worked for and I’m only 10 days in. There are a lot of benefits both personally and professionally.

Don’t get me wrong, this is a massive company. almost 100,000 users. Yet when working with your manager, you feel like they honestly want to you to be the best YOU that you can be. It’s ginormous billion dollar making company but with a personal feel. Best of both worlds? so far, Yes.

I thought I knew exchange, I really did. Turns out… not so much. That is the purpose of this post’s title. Starting here is like the example in the title. There is so many resources, and you have to learn fast! The side advantage of working here is being surrounded by so many brilliant people. Chances to grow, heck.. even move across the country!

Hey all!

This blog’s focus from day one is, and always has been you guys. My friends, my co-workers, and my students. You are not just some page hit count. You have a voice. I’ve always said this blog is reader driven. Most of my posts have been from repeat questions I’ve gotten from in my classes.

As I’m sure you may have heard, starting mid-June, I will begin working directly FOR Microsoft! My new role with the mothership will be laser focused on Exchange. Now expectedly, you will see a lot of the post revolve around my new deep focus. I don’t want to alienate any of my older readers so it won’t be 100% on Exchange. Please contact me directly here ( if there’s something you’d like to see on this site.

I do have a few things that have been in the wings I’d like to finally post. The most noteworthy is the “Solarz slide rule” on how to subnet IP addresses in your head in a matter of minutes. I will get this post and/or video up as time permits.