Companies are moving their email to cloud in droves.
Let's face it, administering Microsoft Exchange is one of those jobs that when everything goes right, no one knows you exist. And when things go wrong, everyone knows you exist. The good news is that many companies are offloading their Exchange to Microsoft through the use of Microsoft Office 365. If you doubt that Office 365 is big, consider that in July of this year Office 365 online workplace tools brought in more revenue than the traditional version of Office that’s installed on people’s computers. When you think about it, e-mail server replacement is the perfect SaaS application. It's well defined without huge deviation from one organization to the next, scales well across multiple servers, needs to be accessible from anywhere and often needs permanent retention of records. All things that the cloud is good at.
Moving to the cloud means I'll never have to worry about email again, right?
It's important to remember that while moving to the cloud alleviates your responsibility for the servers that run e-mail, you still are responsible for monitoring the e-mail itself and your company's connectivity to the cloud. Monitoring cloud-based applications is different than monitoring on-premises applications. Where you may have been concerned with memory and disk capacity on your servers, or server-to-server communication in the past. Those are not concerns with SaaS. But some potential issues still exist. Here are just a few of the metrics you may need to be concerned with in an Office 365 environment:
- Portal Access - Rather than server availability, it's important to know portal availability. This includes the user portal, the administration portal and the billing portal. These may each be used by different users in your company but are all important.
- Forwarded Exchange Users - Are these mailboxes really necessary? Are they violating company or government policies? What if a healthcare worker is forwarding messages containing patient information to a personal account, for example?
- Inactive Exchange Users - While sometimes you may keep a user's mailbox for a period of time after they are gone, sometimes you just forget to delete them and are paying for unneeded accounts.
- Groups Accepting External Email - Do you really want external entities to be able to bulk mail these groups?
- Top Senders - This is a handy metric for telling if your accounts have been hacked and are being used by spammers.
- Administrative Roles - Did the number of administrators change unexpectedly?
- License Usage - Get a handle on how quickly your license usage grows. How many licenses are being used? What percentage of my total? You still need capacity planning for SaaS, just a different type of capacity.
- Last Password Change - Number of users with a password that is 90 days old or more. How many users have a password that never expires?
- User Mailbox Security - How many users have access to a large number of mailboxes? Should they?
Earlier this year, in collaboration with Loop1 Systems, we developed a set of templates for Microsoft Office 365 to monitor these and much more. The templates have been very popular with customers, but there are a few things you can do to improve their implementation and function. Since these templates monitor Software as a Service, they aren't exactly like other templates that we typically provide.
Microsoft Office 365 is Software as a Service and it doesn't run on any of your servers. What node should you apply the template to?
Since these templates are PowerShell scripts that run against a Microsoft URL, the best solution is to create an external node and apply the templates to it. You can use "outlook.office365.com" as the node. This is the URL for the mail API requests. Technically for the Portal, Subscription, Security Statistics and License Statistics templates the scripts use "api.admin.microsoftonline.com", but splitting the Office 365 templates between two nodes can be confusing and forces the SAM user to understand which components of the service reside on each node.
You can also use an ICMP node rather than an external node
External nodes don't report status. By using an ICMP node, you will get a rudimentary status indication on the node icon based on a ping of the URL. External nodes give no status and always display a purple "arrow" icon without status. However the URL "api.admin.microsoftonline.com" doesn't seem to respond to ping requests so it will always appear to be down if you point an ICMP node there. Here is the external icon vs. the ICMP node icon.
Get a real picture of Office 365 availability with NetPath
Another way to determine the responsiveness of the Office 365 application is to set up a NetPath service for "outlook.office365.com". If you have NetPath, you can use it to get a detailed view of the bottlenecks between your site and the application portal.
Improving responsiveness to queries by polling less frequently
Depending on the number of mailboxes in your environment and the number of templates implemented, you can experience throttling of your API requests from the Office 365 API. If you are throttled, the choices are to either run less component monitors or reduce your polling frequency on some templates. Most users can actually reduce the polling frequency substantially on most or all Office 365 templates since the majority of the metrics don't change frequently. One thing to keep in mind is that if you want to ensure enough data points to avoid gaps in history, you might want to use less than an hour for your polling frequency, so try setting the frequency to 1200 (20 minutes) rather than the default of 300 (5 minutes). If you want to know more about Microsoft API throttling, see Avoid getting throttled or blocked in SharePoint Online | Microsoft Docs for a description. The article is about Sharepoint but the concept is the same for Office 365.
I don't like the output of the detailed data from the templates. Can I make it more readable?
The data comes back from the API in a comma-delimited format which is great for programming but not so readable. To make the data more readable, you can modify your own copies of the scripts as follows:
[string]::Join( ", ", $users)
[string]::Join( "< br/>", $users)
NOTE: You should be aware that this modification is injecting HTML directly into the output from the PowerShell script. When viewed on the SAM console it will display correctly. However, this change could create unexpected results in other areas of SAM that are not displayed on a web server, such as reports.
Comparing Exchange 2013/2016 templates with the Office 365 template. They are both Exchange, why are they so different?
Since Office 365 is SaaS, many of the metrics in our previous Exchange templates is either not available or not meaningful. Metrics like disk I/O and disk latency aren't available for a cloud service where the hardware is abstracted away from the user. Similarly attempting to monitor processes and services on the hosts is not possible. Primarily with Office 365 we monitor application data, which is available through the Office 365 API.
There was a MAPI round trip template available for Exchange. Can I run this template against Office 365?
The MAPI round trip template was intended to check connectivity between multiple Exchange servers. Since Office 365 is SaaS, you don't control the physical servers that are used for your accounts. With cloud-based applications, you should check connectivity between your network and the Office 365 website. You can get a sense of this connectivity by using the portal templates and the ICMP option discussed above. Also as mentioned above, you can use NetPath to show the actual path your connections take to Microsoft. Another option is to use Web Performance Monitor to record a typical mail transaction and get perspective on each part of the session.
A comprehensive approach to monitoring Microsoft Office 365
Hopefully, this post has given you some ideas about why and how to monitor Office 365. SolarWinds offers many tools to help you from SAM templates to network tools to user simulation.