News Stay informed about the latest enterprise technology news and product updates.

Miscues abounded in Boston email retention policy, practices

Boston CIO Bill Oates talks about measures taken to tune an email retention policy after disconnects between user practice and system capability set off a political firestorm.

Email retention policy became part of the public discourse recently in Boston. A records request from The Boston Globe revealed that a top aide to the mayor was double-deleting messages in apparent violation of a state records requirement to save everything for two years. The practice was chalked up to a misunderstanding, but an investigation is ongoing -- and drives home to all CIOs the importance of having an email retention policy and communicating it clearly to users.

We spoke with city of Boston CIO Bill Oates on the day the city published an email policy guide to how the city did things before and the measures taken since – including journaling -- to clarify its email retention policy.

Here is an edited and condensed version:

More on email archiving
For CIOs, email deletion scandal shows need for email retention policy

E-discovery rules double-edged sword for CIOs

Email and messaging management

Email archiving strategies: Five best practices
Were you surprised when you read, along with everyone else, that this mayoral aide had been cleaning out his inbox by double-deleting and that this has turned out to be a controversy for the current administration?
Oates: Most of our activity managing the mail system is concerned with capacity. We spend probably more time looking at the high users as opposed to looking for low levels of messages in people's mailboxes.

In this instance we had a records request. Normally, the result of such requests is volumes of data. So as soon as it came to our attention that there was a low number of retrievable emails we dug deeper and discovered that our users expected that all their mail was being saved. We reacted to that right away. But in the normal course, our technology support team would not be out there actively looking for folks with small numbers of messages in their mailboxes.

The other key issue is that what we're hearing now is a requirement to save everything, no matter what the content, for a minimum of two years. I think that if we understood that as a clear requirement we would have had a system in place that would support that requirement.

More email retention
policy details
Does Boston have a formal email retention policy?
Oates: There really has not been a clear retention policy, but there has been a lot of discussion over the last couple of years. A task force in place this year with legal, the city archive management folks and our IT team has been looking into the Federal Rules of Civil Procedure and some of the Massachusetts requirements. We definitely had a disconnect between what the users were expecting was getting backed up versus what was actually getting backed up. Legal has just released a policy to make sure everybody understands what the policy is.

Do I understand correctly that there was no email retention policy in place before the mayoral aide's emails were requested?
Oates: We certainly had an email use policy that is consistent with our IT policies that has been up there for a number of years. But from the retention perspective, I don't know that there was a single email retention policy that had been officially adopted citywide. Last May we released to everybody an email policy that is also up on our employee portal. I don't believe prior to that that there a specific email retention policy.

What explains a top mayoral aide's lack of knowledge about email retention best practices or policies?
Oates: We found an expectation around retaining emails that was not consistent with what the systems were doing at that time.

We now have put a safety net in place by enabling journaling in our Exchange environment, which creates a copy of every message that comes in or leaves the city mail environment, and this copy is saved separately from the email that would actually go into a user's inbox. In addition, we back up our systems every night. We retain all of those backups; we start on a Data Domain backup system and then it moves off to tape. So, for three months we have backup of our entire Exchange environment, which includes all messages. After the three months, messages are automatically thrown into an archive, where they stay indefinitely. --L.T.

So you were not aware of the two-year rule?
Oates: I do not believe that there is a requirement by any public-sector agency for us to do that. We value how important the retention of public records is for the city, and today we have an email system that has more than 10 million objects in it. We have a robust server and storage environment to support our mail system. We are very diligent about making sure that we provide the capacity in the systems to do those kinds of things.

But the idea that literally every message that comes into our city environment, whether it has business-related content or not, be saved and held on to without giving our users the ability to do some level of content filtering is not ideal. Our public records folks were telling our users, in a policy statement posted in May, to look at the content of their mail and if there is no business value to that information to get rid of messages when you don't need them. So it was very clear that the email system was not viewed as a record-keeping system.

We are going to be meeting regularly with the secretary of state's office, because we want to make sure our policies and our systems are supporting whatever the state and federal requirements are on public records. We want to comply; all I am saying is that if compliance is interpreted as the need to hold onto every single message that comes into and leaves our environment and that these messages be kept for two years, I am not sure that is the proper interpretation of what requirements are.

Is the goal to be working with the secretary of state and get the law changed?
Oates: I don't think we are talking about any law change. I think what I am saying is making sure we have clear guidance on the interpretation and application of that law. In our situation, we clearly had a disconnect between the expectations of our users and what our systems were doing in terms of email retention. I am not saying that I believe anything we were doing concerning message processing and retention was in violation of any statutes.

We are doing journaling right now because we want to make sure we are not losing any message that comes into our environment, and we will continue to do that as long as we need to. But I am not sure that that would be the regular practice anywhere else, but that would be the only way to hold on to all these messages and literally not give the user an ability to delete even an inconsequential message.

How much time have you spent on this the last couple of weeks?
Oates: More than we normally would! We have a lot of technology initiatives going on around the city. But we certainly understand this is critical, so we have been spending a lot of time on it. The good news is that we have spent a lot of time and effort over the past year in building up our system -- we have migrated our Exchange platform over the last year, which dramatically increased our server and storage capacity in the city to support the growth of all these things we are talking about.

Have you consulted with other cities on how they handle what is clearly a thorny issue -- email retention?
Oates:Yes, we have. I think that how we have managed our environment is very much in keeping with how other cities run things.

Disconnects and miscommunications are the Achilles' heel of IT. Do you plan to get out there and talk to users?
Oates: Absolutely. Take a look at our web site,, and click on the email policy link, which is a summary of our policy, including a couple of diagrams that show how we operated before and after the change. You'll see all the training and education of all the employees, on the new email policy issued by our corporation counsel.

Let us know what you think about the story; email: Linda Tucci, Senior News Writer

Dig Deeper on Enterprise systems management

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.