Database Archiving for the Future

by Trevor Eddolls
June 12, 2008

An August 2007 survey of IT executives conducted by eMedia on behalf of BridgeHead Software found that 59 percent of respondents complained that the volume of data they’re forced to back up is disrupting business operations or will do so eventually. And 93 percent said their routine backup volumes are continuing to increase. The survey identified several advantages to reducing the volume of data routinely backed up:

• Less IT time devoted to backup and other business class processes (69 percent)
• A reduction in the impact of backup and replication on network utilization and capacity (60 percent)
• A reduction in disk resources devoted to data snapshots, replication, and mirroring (58 percent)
• Reduced disruption to the live application environment (45 percent).

Another reason for archiving is that many companies use their archived data to find out more information about their clients’ preferences, or which marketing strategies are more effective. This is known as data mining.

Business considerations have caused corporations to retain data for perhaps five to seven years. Government regulations have caused organizations to need to retain data for much longer periods. Examples include:

• The Sarbanes-Oxley Act (SOX)
• Health Insurance Portability and Accountability Act (HIPAA )
• BASEL II (a set of requirements for the banking industry).

There also are more than 150 state and federal laws dictating the length of time Electronically Stored Information (ESI) needs to be retained; many of these laws greatly extend the retention period for this data. These regulations, some of which apply only to specific industries, have increased data retention periods to, in some cases, more than 30 years. So regulatory compliance is another reason for archiving.

For all of these reasons, organizations must decide what to archive and when. What’s also needed is a way to ensure that records in the archive are deleted when they’re no longer needed for business or compliance reasons. There’s one important caveat to that last statement. In the event of litigation or the possibility of litigation, an organization must preserve all data relevant to the case. So the deletion policy must be changed. This is called “litigation hold. ” Should a company continue with its policy of deleting archive data, then this would be what’s called “spoliation. ” If any relevant documents are deleted, then the court case would probably be lost.

What are the alternatives to archiving? They are all probably worse solutions than archiving if the goal is reducing the data being stored in the database:

• Doing nothing—just letting the amount of data in the database increase over time—saves you from doing any work but as the database gets bigger, transaction processing and data become less efficient. Also, older data can be overwritten and information can be lost. If that occurs, it’s no longer authentic.