Guidelines for Ensuring the Long-Term Accessibility & Usability of Records Stored as Digital Images
Archives Technical Information Series #22

Introduction

Guidelines

Legal and Policy Considerations

Management Considerations

• System Planning, Acquisition, and Development
• System Management
• Migration

Technological Considerations

Appendix A

Appendix B

Appendix C

Introduction

Effective August 1, 1997 the Regulations of the Commissioner of Education were changed to reconcile and combine requirements for the retention and preservation of electronic records applicable to both State agencies and local governments. The consolidated requirements are now regulation 188.20 Retention and Preservation of Electronic Records. For all intents and purposes, the rest of the regulations affecting local government and State agency records management programs remain unchanged. The motivation for changing regulations affecting electronic records was to:

• increase the ability of local governments to take full advantage of imaging technology;
• reconcile regulations applied to State agencies and local governments.

Previous regulations applicable to local governments distinguished between images of records scheduled for less than ten years and those scheduled for ten years or more. In the case of the latter, local governments were required to retain for the life of the record either eye-readable (paper) or near eye-readable (microform) versions of the digitally imaged records. The new regulation do not distinguish between imaged and other electronic records or between records with short retention periods and those required to be retained for more than 10 years. They simply require both State and local officials to ensure that electronic records are not rendered unusable because of changing technology before their retention and preservation requirements are met. In the case of permanent or archival electronic records, an agency or local government, in consultation with the State Archives, must determine that the records will remain usable and accessible through conversion of the records to new system hardware and software and through the creation of adequate documentation... (full text of Section 188.20 is in Appendix A).

In substance, the changes in regulation only impact digital images of local government records that were originally on paper and which are retained for ten years or longer. The regulations do not specify how a local government or State agency is to accomplish the requirement and do not preclude them from transferring imaged records to other media or maintaining microfilm copies for long-term preservation purposes. The following sections outline guidelines for government officials who choose to maintain records with retention periods of ten years or more in imaged formats. The guidelines represent best practices in the information technology management field and reflect established State information policies and standards.

Guidelines

Ensuring the long term accessibility and usability of records stored as digital images is largely dependent on how digital imaging systems are designed, implemented, managed, and migrated. A common misperception is that imaged records will be available as long as the physical media used to store the images last. Preservation of media is only one element that ensures long-term accessibility to records. The key to maintaining usable imaged records for long periods is the ability to transport the records, access tools, and required system functionality between hardware platforms, software platforms, and storage media over time. The life of an imaging system is conservatively estimated at about three years, while records retention and access requirements often exceed this short lifecycle. You must apply appropriate policies, management procedures, and technology from the point at which a system is designed until it is redesigned or migrated, to ensure that long-term records are accessible for as long as they are needed. This means you must make a long-term commitment of resources to preserve the accessibility and usability of digital images.

Legal and Policy Consideration

Specific laws and regulations related to governmental functions may define how records are created, formatted, and maintained. These requirements, as well as legal minimum retention periods established by records retention schedules, should be identified and accounted for when contemplating imaging applications. Therefore, you are urged to consult with legal counsel and appropriate State or federal agencies when considering an imaging application.

Management Considerations

Managerial practices throughout an imaging systems life will have an important impact on your ability to access and use imaged records. These management practices are especially important for migrating imaged records to new technology environments, which will help ensure long-term retention. Below are the management considerations you need to address as systems are planned and developed, as well as those specifically related to system migration.

1. System Planning, Acquisition, and Development

Budget for change

Imaging hardware and software is relatively inexpensive to acquire. The true costs of imaging are centered on training, support, conversion of documents to digital formats, and the continuous system upgrades needed to stay current with the latest technology. Therefore, you should annually budget between 10 to 20% of the cost of the original system for maintenance, support, and upgrade.

Select a reliable vendor with a good track record

Vendor instability is a threat to the long-term viability of imaging systems. You must carefully assess the long-term viability of vendors and the systems they sell when acquiring imaging systems that depend heavily on support from the vendor or manufacturer. This is particularly an issue when you are dependent on the vendor for system support.

Consider using a consultant or system integrator

Independent consultants who are experienced in designing and installing imaging systems exist in most regions of the state. They may or may not also sell hardware and software products. You can contract with consultants to produce needs assessments and feasibility studies, specifications for systems, and requests for proposals (RFPs). Some consultants will conduct vendor conferences, respond to vendor questions, assist in vendor selection, and oversee vendors work. Systems integrators are individuals or firms that conduct needs assessments and design and implement systems using components from various manufacturers best suited to user requirements. They also provide ongoing training, support, and upgrades for the systems they design and install. Value-added resellers represent the products of one or only a few companies. They usually customize the products they represent, and also provide support services. It is impossible to generalize about the nature and quality of services provided by each type of vendor. However, it is safe to say that independent consultants and systems integrators are most likely to provide solutions which are based on standards and which are open rather than proprietary (see Technological Considerations below), while value-added resellers are more likely to represent proprietary systems.

2. System Management

The management of an imaging system is key to the long-term integrity and authenticity of imaged records and their acceptance in legal proceedings and audits. Not all systems merit the same degree of monitoring and control, which should be commensurate with the degree of risk and the benefits to be gained from more effective system management. You must pay special attention to any system that produces records needed in legal proceedings and audits, or one that potentially exposes your organization to a high degree of risk. Appendix B contains Technology Policy 96-10 - Legal Acceptance of Electronically Stored Documents, developed by the New York State Office for Technology (formerly the Governor's Task Force for Information Resource Management) for State agencies. Although this policy is not binding on local governments, it does outline the best practices for maintaining imaging applications to ensure that the records they produce or maintain will be accepted in legal proceedings. We recommend that local governments adhere to this policy when managing systems that capture long-term records.

3. Migration

A migration strategy is an essential component in ensuring long-term access to usable imaged records. Such a strategy should guide the movement of records from one generation of technology to another, as well as allow the recreation of access tools and necessary functionality for records use. As mentioned earlier, you will probably have to migrate most imaged records with a retention period of ten or more years at least twice during their lifetime. The possible approaches to migration are expanded by the use of open systems, standard-compliant technology (see below), wise budgeting that accounts for training and technology upgrades, selection of a dependable vendor, and sound management of the system. You should also:

Plan a technology strategy that will include a continuum of actions such as:

• ensuring the preservation of imaged records on existing media through careful attention to environmental storage
• maintaining the functionality of existing hardware and software through upgrades of equipment and source code
• transferring the images, index and other related data through successive versions of hardware and software
• migrating optical imaging systems to successive generations of technology, as yet undefined

Monitor technology developments and trends, modifying migration plans as needed.

Technological Considerations

We have discussed the importance of migrating applications and records to new technologies in order to retain imaged records over a long period. The technology choices made when systems are developed or upgraded will often determine the ease of and available options for such migrations. If possible, you should ensure that the new technology complies with the following guidelines:

Select an open system solution. An open system solution is one in which the hardware and software components are purchased from different vendors and integrated into a system by a consultant known as a systems integrator. The open systems approach provides a maximum amount of choice to the system developer and end user of the system. Software used in an open system is portable, which means that it can be moved to a variety of hardware. The software is also scalable, which means that a system can be sized to handle both small volumes of users and records and expanded to larger volumes. Open systems can therefore be scaled up with limited disruption to operations, including the maintenance of records.

Select standard-compliant system components. System components that are compliant with industry standards and best practices can be more easily upgraded and migrated. Appendix C contains Technology Policy 96-16A Electronic Document Management Systems - Standards established for State agencies by the Office for Technology. We recommend that local governments adhere to these standards when acquiring imaging and related technology.

Controls and system auditing tools should be available. When acquiring systems, you should ensure that they are capable of providing audit trails and system security. Effective audit trails can automatically detect who had access to the system, whether staff followed existing procedures, or whether fraud or unauthorized acts occurred or are suspected. Software is available for keystroke monitoring, time and date stamping, virus detection, and other controls that can be built into the design of systems.

Select appropriate storage media and environment. Information and images that are important to your process or government should be stored on a server (or a mainframe acting as a server) and backed up either on a different computer or on different media. In the past WORM (write once, read many times) technology has been recommended for offline storage of imaged records when long-term retention and legal admissibility is the primary consideration. However, other media may be suitable for such records. If CD-ROM is used as a storage media, it must comply with the ISO 9660 standard, which specifies how a CD-ROM disk stores information. Regardless of media selected, government agencies should:

• require the use of optical media with a pre-write shelf life of at least five years and a minimum post-write life of twenty years, based on accelerated aging test results that report on specific disk areas
• store optical media in areas with a stable room temperature (65 to 75¦ F) where relative humidity does not exceed 50% and does not fall below 30%
• never operate drive systems in environments with high levels of airborne particles
• periodically clean optical media to remove dust and other particulates

For More Information and Assistance

The State Archives provides records management services to state and local governments, including technical advice and assistance, publications, training and presentations, and consultations with government officials concerning records and information management issues. The Archives has regional offices throughout the state; each office has an expert records specialist who can visit government agencies and provide on-the-spot advice. For further information, contact your regional office, or:

Government Records Services
New York State Archives
State Education Department
9A47 Cultural Education Center
Albany, New York 12230
(518) 474-6926

APPENDIX A

REGULATIONS OF THE COMMISSIONER OF EDUCATION 188.20
RETENTION AND PRESERVATION OF ELECTRONIC RECORDS

188.20 Retention and preservation of electronic records.

(a) An agency or local government shall ensure that records retention requirements are incorporated into any plan and process for design, redesign, or substantial enhancement of an agency or local government information system that uses electronic data processing, electronic optical imaging, or other automated information technology, to maintain or store electronic records.

(b) An agency or local government shall ensure that electronic records are not rendered unusable because of changing technology before their retention and preservation requirements are met. In the case of permanent or archival electronic records, an agency or local government, in consultation with SARA, must determine that the records will remain usable and accessible through conversion of the records to new system hardware and software and through the creation of adequate documentation as defined in subdivision (c) of this section. If a state agency cannot accomplish such a conversion, it shall transfer the archival electronic records to the State Archives in a usable and accessible format.

(c) An agency or local government shall develop and maintain up-to-date documentation about all permanent or archival electronic records sufficient to:

1. specify all technical characteristics necessary for reading and processing the records;
2. identify all defined inputs and outputs from the system;
3. define the contents of the files and records;
4. determine restrictions on access and use;
5. understand the purposes and functions of the system;
6. describe update cycles or conditions and rules for adding information to the system, changing information in the system, or deleting information; and
7. ensure the ongoing retention of records by the agency or local government or the authorized transfer of records to an archives facility.

(d) A state agency shall prepare and store in a records center and local governments shall prepare and store in a secure off-site facility backup copies of permanent or archival electronic records in order to safeguard against loss.

(e) For magnetic computer media that contain permanent or archival electronic records, an agency or local government shall institute maintenance procedures to:

1. verify no more than 6 months prior to use that the magnetic media used to store permanent or archival electronic records is free of permanent errors;
   
2. rewind under constant tension all tapes and cartridges at least every 2 years;
   
3. annually test a 3 percent statistical sample of all volumes, or 10 volumes of each type, of magnetic media, whichever is larger, to identify any loss of data and to discover and correct the causes of data loss;
   
4. copy immediately onto new media any permanent or archival electronic records stored on media with 10 or more permanent errors per volume;
   
5. copy all permanent or archival electronic records onto new media before the media is 10 years old;
   
6. prepare external labels which provide a unique identifier for each volume, the name of the organizational unit responsible, and the permanent or archival electronic records title.

APPENDIX B

Governor's Task Force on Information Resource Management (Office of Technology website)
Technology Policy 96-10: Legal Acceptance of Electronically Stored Documents 
The purpose of Technology Policy is to outline the issues that must be addressed in developing or operating an imaging system for the legal admissibility of electronically stored documents.

APPENDIX C

Governor's Task Force on Information Resource Management (Office of Technology website)
Technology Policy 96-16A: Standards for Electronic Document Management Systems 
The purpose of these standards is to provide general guidance to agencies in the area of image enabled Electronic Document Management Systems (EDMS).