On May 17th, 2013, IU approved Policy IT-28 Cyber Risk Mitigation. This policy states that all units of IU must use the services provided by UITS to the greatest extent practicable. This policy has far-reaching impact on the computing environment at IU and one of the affected services provided by the SoIC units (CS, Informatics, and ILS) has been email. Since UITS provides email services for IU, we are now obligated to use them. As a result, we have begun the process of migrating the users of our email servers to the UITS servers with the ultimate goal of decommissioning our mail servers.
Please note that we expect this change to have minimal impact on the current users for the following reasons:
- The existing email addresses that people have been using for many years (eg. @cs.indiana.edu, @slis.indiana.edu) will continue to work.
- In that majority of cases, people will still be able to use the same email client programs they are currently using to read their email. The Exchange server talks IMAP so will work with any IMAP-capable email client.
- Users will be able to get email disk quotas on par with the levels provided on the current SoIC servers.
- The vast majority of SoIC users are already using the IU servers so are not impacted by this change.
We have identified the following advantages to making this change:
- It will bring us in compliance with IU policy IT-28.
- It will be a cost saving measure since the anti-spam/virus software in use on the CS email server is a commercial product we renew annually.
- System staff will save time by not having to administer these email servers.
- Currently, email sent from IU servers to our servers can be encrypted by the IU auto-encryption service because our servers are outside the IU circle of trust. Once this migration is complete, we will be inside the circle of trust so inter-IU email will not be encrypted.
- Once the migration is complete, the anti-spam filtering can be more efficient and, hopefully, more accurate. When email is sent to an @cs address and then forwarded to the IU servers (as is common for most of our users) the IU spam scanning software sees the message as coming from our servers which renders certain types of IP-based relay filtering ineffective.
However, we fully understand that there will also be some disadvantages to making this change:
- It will take valuable time from both the current users and system staff to complete the migration
- Anyone using procmail filtering on the CS server will need to convert those rules to Exchange filters. In most cases, the Exchange filters can do what you need but there are a few features of procmail that cannot be done on Exchange.
- It is possible that a very small number of users will be currently using email client programs that are not capable of using IMAP and will no longer work. While it is possible to download email from the Exchange server into a local file so email clients requiring direct filesystem access to the mail will work, this is going to be cumbersome and is not recommended. The mail clients that fall into this category include Elm, /bin/mail, and MH/nmh/MH-e.
We are targeting April 2015 as the time at which our local email servers will be taken out of service. However, we have already migrated many users off of our server and we hope to migrate the majority of the remaining users well in advance of the April 2015. This change impacts approximately 35 faculty and staff still receiving email on the SoIC servers. We plan to reach out directly to each of these people to find out about their email environment and ways we can work to facilitate the transition.
We plan to contact all current users of the SoIC mail server directly to help with this transition. This section gives detailed instructions about how to perform the migration but we will be available to help with this on a one-on-one basis.
This section gives detailed instructions on how to migrate your email from the CS server to the IU Exchange server. Please make sure you complete these steps in the order presented. There are dependencies between the steps so doing them out of order could result in your email bouncing and you losing email.
- Create Your Exchange Account (or see if you already have one) - If you need to create an Exchange account, you can do that via the IU Account Management Service. From that page click on "Manage My IU Computing Accounts" and then "View your accounts". If you have an Exchange account it should show up as "Microsoft Exchange" in the account listing. If you don't have an account yet, click on "Create more accounts" and you should see the option to create the account. If you do not already have an account and you don't have the option to create one, then send an email to email@example.com asking them to create it for you.
- Verify Your Exchange Account - There can be a couple hour delay between when you first create your Exchange account and when it becomes active. Before you continue with the migration, you should log into the Outlook Web App (OWA) interface and verify that you can get logged in. If you try to log into the web interface and get a message about the web site being 'out of date' that means you haven't waited long enough so please try back later.
- Turn Off Exchange Forwarding - If you already had an Exchange account and you were forwarding that email elsewhere (like to your CS account) you will have to remove that before you proceed. You can check that and/or remove any forward you have in place via the Outlook Web App. Log in and go to Options > Create an Inbox Rule to view any filtering rules you may have in place. Just check any you need to remove and click the delete (X) button.
- Change Your IU Email Forwarding - Once you have verified that your Exchange account is receiving email properly, you need to change the forwarding for your @indiana.edu (or @iu.edu) address. You can do this by going to itaccounts, clicking on 'Manage my IU computing accounts', then 'Manage your e-mail', and then 'Forward your e-mail'. In the 'Forward your IU email' section of this page it will show where your email is currently being forwarded. You will want to check "Set delivery for this address to users UITS mail account on ..." and select "Exchange" from the pull-down menu. Once that is done, just click the "Forward email " button near the bottom of the page. Please note that it usually takes 1-2 hours for this change to take effect although it could take as long as overnight. You should verify that email sent to firstname.lastname@example.org goes to your Exchange account before proceeding.
- Change Your CS Email Forwarding - Once email@example.com is properly forwarded to your Exchange account you are ready to update your forwarding on the CS systems. To do this, you will need to log into any of the CS linux systems (such as tank.soic.indiana.edu) and create a .forward file in your home directory that contains a single line containing "firstname.lastname@example.org" (being sure to replace username with your IU username). This will take effect immediately so you should send yourself a test email to email@example.com and confirm that you receive this test email in your Exchange account.
- Add Exchange To Your Email Client - The program you use to read your email (eg. Thunderbird, Apple mail, Outlook) is commonly referred to as your email "client". If you are using an IMAP based email client then you can simply change the IMAP server from mail.cs.indiana.edu to imap.exchange.iu.edu and be off and running. However, we recommend that for one of your email clients that you actually add the Exchange account as a new email account in your email client so that both accounts are accessible at the same time. This will make it easier to move email folders from the CS server to Exchange. There are instructions in the UITS KB on setting up various email clients to work with Exchange. You can use the UITS KB page Reading Your Exchange Email to locate the instructions for setting up many common email clients. If your email client program is not listed and you aren't sure how to set it up, please contact the ITG for help.
Transfer Your Inbox - When new email arrives, it is typically delivered into a folder called your Inbox. You can move all the email from your old CS inbox into your new Exchange inbox using your email client (if you set things up per step 7 so that both accounts are accessible). The method you use to transfer email between folders varies with email clients but in Thunderbird you can just highlight the messages in a folder and then right-click, select "Move To...", and then select the desired target folder. You can move all the messages from your CS inbox to your Exchange inbox if you want but you might consider creating a new folder on the Exchange server (called something like "CS Inbox") and then move all your messages there to keep them separate from your new Exchange inbox. It is entirely up to you so just move them as desired so your old CS inbox is empty.
Message Size Limit
There is one serious gotcha you need to be aware of when moving messages from a CS folder (including, but not limited to, your inbox) to Exchange. The UITS mail systems have a per-message size limit of 20MB but the CS mail system allows larger messages. If you attempt to move a message larger than 20MB to Exchange, the results will be unpredictable and will almost certainly fail, sometimes in a dramatic or confusing fashion (including hangs and crashes). To avoid this problem, we recommend that you sort the messages in any mail folder you plan to move by size and delete those larger than 20MB. The process of sorting messages by size is mail client specific but we can help if you don't know how to do it. For example, here are instructions for sorting messages by size in Thunderbird.
Transfer Other Email Folders - You almost certainly have a number of email folders other than your primary inbox and all those folders will have to be moved to your exchange account. If you do not transfer them, they will still be accessible to you as files in your ~/mail directory (and via any file-based mail client like pine) but they will not be accessible via IMAP. Granted, this is a tedious process but some email clients (like Thunderbird) make moving folders between servers relatively simple. In Thunderbird you can simply drag-and-drop folders from CS to Exchange but they must be done one folder at a time as we have not had luck with any multi-folder copy tool or plugin for Thunderbird. There is an export/import feature in Microsoft Outlook that we have used to transfer an entire folder hierarchy that is an option if moving folders one at a time is not feasible. Let us know if you would like to use this tool and we can provide further instructions. However, please note the above Message Size Limit warning which applies no matter which method of transfer you elect to use.
Email Quota Warning
The default quota on your new Exchange account will be 2GB but it can be increased. This default may be less than the total of all your CS email folders so you need to take care when moving your folders. You should keep an eye on your Exchange quota as you move folders to be sure you do not run out of space. Please see the UITS KB page How do I check my storage space on the IU Exchange server? for details on how to check your usage. If you see that you are approaching your limit, please contact the ITG and we will help you get your quota increased.
- Transfer Any Procmail Rules - If you are using procmail to do automatic processing of incoming email (eg. automatically filing emails into folders) then you will need to recreate these rules in Exchange since procmail will no longer be supported. If you are using procmail, you should have a file named .procmailrc in your linux account home directory that contains various rule sets called recipes. Using the Outlook Web App interface you can add rules by going to Options > Create an Inbox Rule. You will typically need to create an inbox rule for each procmail recipe in your .procmailrc file.
Tips, Caveats, and Pitfalls
This section addresses some issues you may run into during this migration issue. As always, if you have problems you can contact the ITG and we will try to help
- Sent vs. Sent Items - A large percentage of the mail systems in the world default to using a folder named Sent to save copies of the email you send. In the Exchange world they use a Sent Items folder instead and this can cause some confusion since the message you send via your email client may end up in Sent and the message you send using the Outlook Web App interface will end up in Sent Items. If you want all of your sent messages to end up in the same folder, you probably should plan to configure your email client to use Sent Items.
- Trash vs Deleted Items - Many systems use a folder named Trash into which emails are moved when they are deleted but in Exchange they use a Deleted Items folder instead. If you configure your email client so that messages are moved to a Trash folder when deleted you can change that so they are moved to the Deleted Items folder instead. Please note that if you do use the Deleted Items folder then you have the added advantage that once messages are purged from this folder that you still have up to 15 days to recover them. For more information, see the KB page In Outlook for Windows, how can I recover deleted items if I am an IU Exchange user?
- Junk E-Mail - As with the Sent folder, Exchange uses a different naming convention for the Junk/Spam folder than some other systems. The automatic spam filtering that is done in Exchange will result in message it flags as spam going into the Junk E-Mail folder. This email is only retained for 5 days before being auto-deleted so you should check this folder frequently. Do note that (per the above bullet item regarding the Deleted Items folder) it is possible to recover deleted messages for up to 15 days so that gives you a longer time window in which to recover auto-deleted junk email.
- Whitelisting/Blacklisting - If you find that non-spam email is ending up in your Junk E-Mail folder, you can add specific senders to your Safe Senders list. For more information on your junk email settings, please see the UITS KB document At IU, how can I view an online summary of my Exchange account's Junk E-mail settings? There is also a "Don't move e-mail to my Junk E-Mail folder" option there if you want to opt out of the automatic spam filtering as well as a "Blocked Senders" list.
- Outlook Filtering and Disappearing Email - It is important to note that the Outlook mail client (but not the Outlook Web App) will do junk email filtering which is independent of the server-side filtering that is done on the Exchange server itself. If you never run the Outlook mail program, then you don't have to worry about this. But, if you use Outlook plus some other IMAP mail client (like Thunderbird, Android K-9, Apple Mail, etc) you may occasionally see a message show up in your inbox for a few seconds and then disappear. This is probably Outlook's mail filtering at work so you should check your Junk E-Mail folder for the message. Furthermore, there are some users who have reported that Outlook's filtering is moving obviously non-spam messages to Junk E-Mail even when using the default Outlook settings. So, if you are an Outlook user and are finding that many non-spam messages are ending up in your Junk E-Mail folder, you may want to turn this feature off in Outlook via Home > Junk > Junk E-mail Options... > No Automatic Filtering. When you look at a message in your Junk E-Mail folder in Outlook, there will be some extra information in the header and if Outlook's filtering was responsible for moving the message to your Junk E-Mail folder it will say "This message was marked as spam using the Outlook Junk E-mail filter". If you don't see that message, then it was probably moved to that folder by the server-side filtering on the Exchange server or by a custom rule you have set up.
One final note regarding this Outlook filtering is that Outlook will do this filtering when it first starts up (or even just when you log in without even starting Outlook!) on all messages Outlook has not yet scanned. So, for example, if you were to run Outlook only once a month, it will look at all messages you have received in the past month and may move any of those messages to Junk E-Mail. Likewise, if you were to move a mail folder to Exchange, even if it contains old messages, Outlook will scan all the messages in that folder and move any messages it identifies as junk to Junk E-Mail.
- Tags Not Supported - With some mail clients (most notably Thunderbird) you can use Tags to mark messages with values like Important and Personal. The CS mail server supported these tags server-side which means that if you used Thunderbird on your office computer and set a tag it would be visible on, say, your home computer. The Exchange server doesn't support this feature so tags are local to the system where you set them and are not visible on other systems.
- Exchange Filtering Rules and "Received From" - When you create Inbox Rules to automatically filter your email, one of the conditionals is "It was received from...". If a message has a Sender: header it will use that and not the From: header. So, if you find that your filtering is not working as expected, look at the full message headers and see if there is a Sender: header. If there is, then you will have to use the address listed there in your rule.
If you run into any other oddities please contact the ITG and we will look into it and add it to this page if it might be of general interest.