Database Archiving for the Future

by Trevor Eddolls
June 12, 2008

• Storing the archive data in UNLOAD files initially may seem a feasible, simple solution. However, the trouble comes when you must produce transaction records. The data can’t be restored to the originating database in case of corruption. Also, the database schema may have been changed since the UNLOAD, causing more problems. Restoring a single set of records can be time-consuming and labor-intensive. Think about the amount of effort required to restore records spanning several years.
• Moving the data to a parallel reference database combines the problems associated with the first two alternatives. In addition, you introduce the problem of being unable to prevent changes to the database. Using database security would seem to alleviate this, but security can be circumvented by “super user” IDs (e. g. , SYSADM).
• Deleting the data optimizes database performance, but the organization loses all benefits from retaining the data, including the benefit of regulatory compliance.

Moving the data to an appropriate archive is the only viable option, but just storing the data isn’t enough. SOX requires organizations to implement a robust data retention solution; Section 802 specifies penalties affecting companies that alter or delete their archive data or documents. This reinforces the preservation of records rule (240. 17a-4) in the Securities and Exchange Act (1934). Electronic storage media in use must store authentic records in a format that’s non-rewritable and non-erasable.

An archive also must make information easily available to authorized users. Today, the data required during the discovery process is typically electronic. Indeed, as Gartner has written, “Because more litigation is based on evidence that is stored and managed electronically, correct and swift production of that evidence is an important business process. ” An archive solution now must be able to quickly and easily work with any e-discovery tools either side involved in a court case might use.

The archive should store both original records and metadata— data that describes tables, columns, and relationships. During the life of a database, it’s quite likely that, for example, schema will change. As the database is archived, information about the different schema also must be archived. That way, records from the archive can be restored with the correct schema in use when the records were archived. Any queries against the archive data must deal with different versions of this metadata.

In December 2006, the U. S. Supreme Court approved amendments to the Federal Rules of Civil Procedure (FRCP) affecting the discovery of electronically stored information. Rule 34b states that, “A party who produces documents for inspection shall produce them … as they are kept in the usual course of business …” If the metadata has been stored, a company can comply with this regulation.