Over the last month or so at my day job, we ran into an unnecessarily tricky scenario trying to get free/busy working between our on-premises Exchange environment and a federated hybrid environment. My main purpose in writing up this experience is because I hope people can find it via a web search and not have to commit hours upon hours of time with Microsoft solving it.
The problem was that when a user in the cloud went to invite an on-prem user to a meeting, the Outlook client (desktop client or OWA) would show an error: “The recipient’s server could not be determined”. Not entirely helpful, as that’s extremely generic.
We looked at the IIS logs on the Exchange front-ends to see if that revealed any clues. Whenever we attempted to view an on-prem user’s free/busy info from the cloud, the log would show something like this:
2015-04-09 18:33:02 10.101.101.104 POST /autodiscover/autodiscover.svc/WSSecurity &CorrelationID=<empty>;&cafeReqId=be5979f1-ed29-04b9-a131-254aeff795a4; 443 - 10.100.100.100 ASAutoDiscover/CrossForest/EmailDomain//15.01.0130.019 - 401 0 0 140
Notice the HTTP 401 error at the end of that line. It seemed like the cloud server was not authorized to view information on the on-prem server. At this point, we opened a ticket with Microsoft. (side-note: this was frustrating… we had an engineer working with us up to this point. Why did he not have the tools to troubleshoot, or at the very least do the research and get back to us?)
The new support engineer first delegated us to a TMG engineer, who then proceeded to tell us it wasn’t a TMG problem (the request was making it all the way to our front-end - TMG was not getting in the way). We got another on-prem Exchange support engineer, and after doing the typical Microsoft support case song and dance, he finally sent us this link:
Free/busy lookups stop working in a cross-premises environment or in an Exchange hybrid deployment
The problem was revealed by the Test-FederationTrust cmdlet, which produced this output similar to this (I’ve replaced the irrelevant RunspaceID GUIDs):
> Test-FederationTrust RunspaceId : <guid> Id : FederationTrustConfiguration Type : Success Message : FederationTrust object in ActiveDirectory is valid. RunspaceId : <guid> Id : FederationMetadata Type : Error Message : The federation trust doesn't contain the same certificates published by the security token service in its federation metadata. RunspaceId : <guid> Id : StsCertificate Type : Success Message : Valid certificate referenced by property TokenIssuerCertificate in the FederationTrust object. RunspaceId : <guid> Id : StsPreviousCertificate Type : Success Message : Valid certificate referenced by property TokenIssuerPrevCertificate in the FederationTrust object. RunspaceId : <guid> Id : OrganizationCertificate Type : Success Message : Valid certificate referenced by property OrgPrivCertificate in the FederationTrust object. RunspaceId : <guid> Id : TokenRequest Type : Success Message : Request for delegation token succeeded. RunspaceId : <guid> Id : TokenValidation Type : Error Message : Failed to validate delegation token.
The key problem was the TokenValidation. That url referenced above suggested that the metadata information in our environment had become stale, and it proposed that we run
> Get-FederationTrust | Set-FederationTrust –RefreshMetadata
in our Exchange powershell. Magically, after we ran this, the cloud users could then retrieve our free/busy information.
As I mentioned above, I hope this write-up helps someone down the road. We searched high and low for a solution to this problem, and for whatever reason we never came across the article that our support engineer linked us to. Maybe it’s because we searched for this related to our HTTP 401 error on the Exchange front ends. That article doesn’t mention anything like that - just that our metadata was stale.
What makes this situation more ridiculous is that Microsoft suggests in the article to set up a Scheduled Task to run this cmdlet regularly. If they know this is a problem, then why can’t that run as part of some process under the hood of Exchange? I digress…
Finally, I should also add that this is probably one of a hundred different problems you could encounter while making free/busy work between cloud and on-prem environments. Even still, I hope this helps someone.