The U.S. National Archives this week awarded competitive contracts worth $20 million to create a system for permanently storing digital electronic records. As computer records increasingly replace paper, archivists face myriad challenges in preserving those "virtual" documents.
That old IBM punchcard-sorting machine is hopelessly obsolete now, but decades ago it was part of a state-of-the-art data processing center. Dealing with ever-changing computer technology is a daunting challenge for organizations, like the National Archives, which need to preserve and manage digital information for posterity.
The challenge facing rival contractors Lockheed Martin and Harris Corporation is to devise a system to guarantee that today's computer-generated records will be accessible years, decades - even centuries from now.
"Much of the information of the late 20th century and 21st century will be lost if we don't do something about solving the problems of saving electronic information over time," says Reynolds Cahoon, the National Archives' Chief Information Officer.
He told reporters that more and more of the Archives' acquisitions are coming in digital format. Those electronic records present archivists with several particular challenges.
"Electronic materials are by their very nature, vulnerable to loss and to change," he explains. "The technology that creates electronic records is constantly improving and, at the same time, is constantly becoming obsolete. There are over 16,000 formats of information that we know of today that need to be stored and managed. Finally, there's the volume factor."
The volume of records is staggering. The Archives, to take but one example, holds more than a billion digital images of official military personnel records.
The National Archives was created as a repository for mainly paper documents. Today, however, the government documents that the Archives acquires are increasingly electronic records, stored on tapes and disks, and created by programs and machines that quickly go obsolete. Have you tried to read a 20-centimeter computer disk lately?
The head of the National Archives, John Carlin, admits the project will be expensive, but he says there will be tremendous benefits associated with the Electronic Records Archives (ERA).
"You see, when we say that ERA will make electronic information available virtually any time, anywhere, and to anyone with internet access, we are not just talking about the information contained in Government records," he notes. "We will start with Government records but there is no end to where ERA can take us."
Bruce Miller focuses on some of these issues for computer giant IBM. "This has never been done on earth before. The technology challenges will be significant," he says.
He adds that some of the problems faced by the National Archives are unique to government, but others are shared by corporate records managers.
"This contract, assuming it's successfully delivered, will solve some of the bigger problems with technology obsolescence. That's a universal problem, albeit not as of great concern to business [as to government]. But if they can solve that by building new technology, I think that will be a benefit," he adds.
But the challenges go beyond the technical. University of Maryland professor Bruce Dearstyne of the College of Information Studies says a lot of the solution will involve policies and procedures.
"It starts with policy and ensuring that the people who are actually creating the records everyday, and that's thousands of people in the federal government, understand that they are creating records, what their responsibilities are, what the technical capabilities are of electronically filing them, preserving those that need to be preserved and disposing of those that are no longer needed," he explains.
One of the biggest challenges, says Professor Dearstyne, is e-mail. Electronic mail is generated in enormous volume, but only some messages need to be permanently saved. You want to keep the ones about policy decisions, but not the ones about meeting a colleague for lunch.
Development of the National Archives' electronic records program will move forward step-by-step, and is scheduled to be fully operational by 2011.