UPF User Survey

Complete results as of 2/98

Challenges for the long-term storage and preservation of media assets:
Challenges of storage media obsolescence and reliability got the most votes, but additional comments mentioned the challenge to formulate a strategy for migrating materials to a digital storage, dealing with budgetary constraints and, in some cases, the apathy of administrators. Establishing new strategies must factor in costs, and a recommended practice might look at the costs of migration to digital formats, perhaps looking at how a UPF might lower certain costs. This material must be presented in plain English. It is one thing to convince archivists of the need for a UPF; it may take more to give archivists the resources for convincing administrators and board of directors of this need.

What do you think are the greatest challenges for the long-term storage and preservation of media assets?
a. Finding a climate-controlled, stable storage environment to physically house a collection. 14
b. Chosing a storage technology that will not become obsolete in a few years. 39
c. Physical reliability of storage medium. 26
d. Meeting costs of state-of-the-art storage. 18
e. Accessing analog materials once they are in a digital format. 13

Designing an open system that you can exchange parts [...] to take advantage of new/better technologies as they become available. - Baker
[Physical reliability of storage medium] may be the "greatest" challenge, but [Chosing a storage technology that will not become obsolete in a few years] and [Meeting costs of state-of-the-art storage] are, I think, the ones that will make the most difference over the next century to the greatest amount of material. The solutions to [Finding a climate-controlled, stable storage environment to physically house a collection] and [Accessing analog materials once they are in a digital format] require education/action of individual institutions. - Carli
...developing strategies for migrating records to new technologies as old technologies become obsolete [...] that ensure the critical components of a records content, context, and structure are captured and moved forward through time. - Conrad
...data migration - moving it from today's format to tomorrow's. - Eiteljorg
...hardware to play the formats is the crucial topic. - Gaustad
The challenge is in preparing for technology obsolescence. The library and archive community are beginning to talk about encapsulation of behaviors with "documents" (a document can be a memo or a film) such that the software that created it can be emulated. - Graham
...the existing image data bank in analog form may be "traditional" but I do not have to worry about changing softwares every few months that make it impossible for me to retrieve what records/ images I had made last year.
... What happens if the caretaker is removed? - Hehman
...format obsolescence, and the difficulty of managing the migration of data to new environments (especially if establishing preservation standards for the conversion of material from one format to another is required). - Hirtle
Format migration is one solution, but it requires careful planning and is expensive, requires also a flexible storage area design. - Lee
The act of migrating from an old medium or format to a new one for which it was not originally designed. It is precisely that moment of transition when information is in danger of being lost. - Lowe
...convincing administration of the need for long-term storage and preservation of media assets. - McLemore
It is a fact that the rate of physical and chemical deterioration is linked to environmental conditions. If the "native" format can be stabilized through proper storage environment, then (at least in theory) it can be accessed well into the future for the purposes of duplication and format migration. Also, since doing the actual piece by piece reformatting is a costly, time consuming proposition, a good environment is economical since its has a stabilizing effect on all materials in the collection. - Messier
...software systems that manage and access the hardware storage systems [...] are unpredictable, inflexible, buggy, and they change every few years, causing us [...] to be unable to read data even though the hardware storage was stable. -
Ogle
...a bigger problem is ensuring continual funding in keeping / maintaining the environment in the storage facility stable. - Phan
Migrating the data as new technologies develop. - Sinclair
The big word here is money. Trained staff and adequate space and time to do the work .... - Skarstad
...getting people to care. - Vanderlinden
...cost of data migration as technology changes. If the technology wave-period can be extended, as UPF seems to offer, the stability of storage media becomes significant, along with the conditions required to make it last the distance. - Webb
If paradigm shift occurs (switch to digital), backward compatability.- Weiss
...convinc[ing] the companies to produce a product for us to purchase and the equipment to do the transfers. - Wilson
The greatest challenge is continuing to provide material to users IN THE FORMAT THEY FIND CONVENIENT AND USEFUL. This implies format conversion. The logistical challenge is performing such conversions in a cost-effective manner. - Wright

Concerning digital and non-digital storage types or formats:
The vast majority of those who answered our survey said they had at least some knowledge of data storage formats. Due to the technical nature of some of our questions, the survey may have scared off many who do not have this level of knowledge.

a. I am familiar with a fairly wide range of digital and analog storage types or formats. 24
b. I have knowledge about some types or storage types. 20
c. I have a minimum knowledge of types or storage types. 5
d. I have absolutely no knowledge of types or storage types. 2

Concerning metadata
In contrast to the storage format question, metadata as a concept seemed to be understood at least on some level by almost all of those who responded. The term has been somewhat popularized by various search engines or services on the World Wide Web, but many archivists know the concept through finding aids initiatives.

Metadata is a term that can bewitch, bother and bewilder us, but it's really quite easy to explain: it's simply data about data. An example of metadata are a library's catalogue records: information that refers or directs one to particular objects. Users of the World Wide Web may be familiar with the HTML tag, META. Among its many uses, the META tag enables Web designers to insert indexing materials or "keywords" that can be processed by certain search engines such as Hot Bot and Alta Vista. In addition, Web designers can use the META tag to insert a their own descriptions of their sites, which is displayed when site information is retrieved by some of these search engines. Based on functions, metadata may be categorized into four areas:
Format: information to decode the Essence
Descriptive: cataloguing information, information used for search & retrieval
Association: information that synchronizes or interleaves the Content Components
Composition: instructions for the various components to work together in unison

a. I have a good understanding of metadata and have followed one or more metadata initiatives or projects. 20
b. I have some understanding of metadata and metadata initiatives or have followed one or more metadata initiatives or projects. 17
c. I have minimal understanding of metadata, but have not followed any metadata initiatives or projects. 10
d. Until this survey, I had absolutely no knowledge of metadata. 2

Concerning knowledge of SMPTE:
Considering that we were appealing to archivists in this survey, it should not be too surprising that so few people knew about the Society of Motion Picture Engineers. Until now, SMPTE has not sought input from the archival or library science communities. One of our missions is to make these groups aware of this technical standards organization.

The Society of Motion Picture and Television Engineers (SMPTE), a standards-making organization, has recently (Sept. 23, 1997) established a Study Group to "document the requirements of data formats for the preservation of electronically generated media and related information. Extensive input from the archival community will be gathered through the use of surveys and meetings."

a. I am an active member of SMPTE. 3
b. I am aware of the aims and functions of SMPTE. 8
c. I have heard or read something about SMPTE, but don't know much about the organization or its relevance to a universal preservation format or framework. 21
d. Until reading this survey, I have never heard of SMPTE and so cannot determine how it might relate to a universal preservation format or framework. 17

Concerning wrappers or "bento containers":
Interestingly, only two people thought they had a good understanding of wrappers. This may be attributed to the elusive nature of the concept. More likely, archivists and librarians understand the concept without quite connecting it to the new terminology.

Bento is a Japanese term that refers to a lunchbox which contains many individual compartments. The Bento storage specification is like this lunchbox in that it holds several pieces and formats of a compound document within a single container. Though the phrase "Bento container" refers to a technical specification developed and last revised by Apple Computer, Inc in 1993, we may use the term in a generic or metaphorical sense to mean a container for packing up and moving both the essence of a media format (a movie, a musical composition, etc.) and the information about this essence (the metadata) as a single unit.
Concerning your knowledge of wrappers or "bento containers":

a. I have a good understanding and have followed one or more related initiatives or projects. 2
b. I have some understanding of the concept. 13
c. I have a bare minimum knowledge of wrappers or containers. 14
d. I have absolutely no knowledge of wrappers or containers. 21

Ultimate goals concerning digital preservation
This question evoked a commentary similar to the earlier "challenges" question. Some of the same issues were raised, particularly budgets. The majority stated that they intended to let public demand determine which analog materials would be converted into digital. Also mentioned throughout the commentary was the determination to continue all analog preservation efforts, even while dipping toes into the digital waters. Implicit throughout many of these commentaries was a "wait and see" approach to digital archiving. Establishing a standard for a universal preservation format through SMPTE and other engineering groups would be like placing a life guard before these troubling waters.

Which of the following statements best reflects your thinking about the ultimate goals of institutions concerned with the digital preservation of media collections?
a. We will save the non-digital originals as long as possible, converting them to digital when the originals are in measurable danger. 1
b. We will convert non-digital materials into digital formats for access purposes (such as Internet and user requests). 28
c. Only materials in measurable danger will be converted to a digital format. 0
d. Non-digital collections will continue to be maintained through traditional storage methods, and only selected materials will be digitized for preservation purposes. 13
e. There are no plans to digitally preserve our collection at present. 5

Funding [...] is of great concern to my institution. - Allen
We will convert archives to a digital format when we can raise the money to move those collections. Others will migrate to digital if we think public access is important for them. - Baker
Our current access/preservation methods are based on microfilm usage, and we now plan to film any items (practically all our collections of interest are on paper) that we digitize for access purposes. We use RLG (Research Libraries Group) standards for microfilm metadata. - Carli
Given the present problems with migration and technological obsolescence, digital "preservation" is probably not the best idea. - Conrad
We will save the non-digital originals as long as possible, with plans to reformat them onto other analog formats as funds permit. For those items where digital copies are needed (for wide access/preservation/internet availability), the item will be reformatted onto analog and digital formats *simultaneously*." - Dale
...sav[e] the non-digital originals as long as possible, but [...] digitizing for preservation and access should be done *before* there's any clear sign of danger/deterioration. While ideally one would like to save everything forever, the cost of conversion and ongoing maintenance of digital versions probably means that appraisal strategies will have to determine what gets digitized and saved... Access and user demand as well as preservation needs should be factors in deciding what to digitize. - Hadley
...our thrust is to identify images which are used often [...] and/or images which best represent a critical component and get it in a medium for quick and easy retrieval and use company-wide. - Hamal
We will use digital preservation for some things when they are in danger... Other things we may convert to digital form to improve access, but when we do so, we should convert them at a preservation level... (see the remarks by Anne Kenney and Steve Chapman on "Full Informational Capture" in DLIB magazine). Hybrid solutions (digital conversion, but an analog preservation copy on traditional media) may still be necessary. - Hirtle
We convert non-digital 35 mm neg. stock to digital so we can use it. We keep the original always. - Lee
Materials should be stored in multiple formats, both analog and digital to minimize risks of catastrophic unrecoverable loss. - Lindner
...I advocate the hybrid approach to the library collection, with preservation reformatting options that include paper, microfilm, electro-magnetic analog, and digital options. - Lowe
...I'd digitize the lot and preserve the originals in the best possible environment with as little disturbance as possible. - Lucas
...office files are likely to be handled differently from library collections, and within the latter, rare books and manuscripts are likely to be handled differently from newspapers. - Maher
For this question, I think it is useful to talk about two categories of artifacts: 1. Machine readable, electronic artifacts (video, computer files, art etc) and 2. "Material" artifacts (e.g. books, photographs, paintings, etc.). For category 1 objects, digitization will emerge as the only viable way to preserve this class of artifacts. For category 2, digitization should almost never be used as a preservation tool. For artifacts in category 2, digitization would result in facsimiles that might be useful for the purposes of access but would be nearly useless for the purposes of preservation. - Messier
...digitize as needed for projects and to offer selections to colleagues, but to wait until standards are in place to do a major digitizing project. - Pollock
We are developing guidelines for digitisation... It is important to encourage people to digitise their collections before they deteriorate [and] present many other problems. - Rymer
I [...] would like to explore available [formats] and convert deteriorating materials first, then move on to "all" materials for access purposes... - Schwan
...everything should be digitised as a preservation and a distribution strategy, but [...] original materials should be kept in cold storage. I feel we should get rid of film only when (a) very high resolution, non compressed images can be stored in computers or (b) material was originally shot in digital format (film prints might be made for distribution). - Turner
...provide [...] functional access to all [our] materials, chosing from [...] storage and transmission techniques as the needs and resources of those we serve allows. - Vanderlinden
[Because] staff and money is a great concern, [...] we will have to be selective as to which non-digital materials are transferred... - Vetter
1. For our audio collections we will digitise everything that is worth keeping because it is in danger of obsolescence as market support for analogue technology shrinks...
2. For multi-media, [...] we will [...] preserve [...] only selected materials for ongoing access. That preservation decision will be based on collection development values...
3. We have a program of digitising to improve access to our rare and valuable Australian non digital collections. While access is the prime goal of digitisation in these programs, there is a preservation connection, partly to protect the originals from wear and tear caused by use, and partly to discover more about the technology in order to preserve the digital copy.
4. Finally, we are involved in programs to digitise material with preservation as the main goal. At this stage they are experimental in that the technology is still too immature to be able to regard digitisation as an appropriate preservation format... Distinctions between preservation and access are less definite in the digital environment. - Webb
This decision will be most affected by costs of conversion. - Wilson
Our plans vary slightly with media type. For photographs, we will mainly ... [save the non-digital originals as long as possible, converting them to digital when the originals are in measurable danger].. For video and audio, we need to provide material in a useful format, hence [We will convert non-digital materials into digital formats for access purposes (such as Internet and user requests)]. For documents and for commercial, recorded music we [have no plans to digitally preserve our collection at present]. - Wright

Migration
The majority of respondents saw the need to maintain a dual storage strategy, though we must add that the "no opinion" choose received the second highest number of votes, which -- considering the thoughtfulness of the commentaries -- may be seen more as further evidence of a "wait and see" strategy than as indifference.

Which statement best characterizes your thinking about institutions converting from analog to digital formats for long-term storage?
a. Institutions should actively investigate digital formats with a plan to eventually convert all collections to a digital storage format. 6
b. Institutions should plan to maintain a dual set of storage for materials. 29
c. Institutions should wait for a digital storage format that addresses the issues of tape fragility and rapid and irretrievable deterioration before making any new investments. 7
d. No opinion at this time. 13

Eventually the conversion will need to be made because of a lack of playback devices for the old analog medium... - Baker
...we currently plan to fold preservation microfilming into any digitization program, until we are satisfied with longevity standards (in terms of technology and expense) for digital copies. - Carli
Collections should not be "preserved" solely in digital form until the problems of technological obsolescence and migration strategies are addressed. The fragility of the media and the obsolescence of storage technology can be addressed today. Finding a format for storing images, audio, video that can be easily and completely migrated is the key issue. - Conrad
...in the best of all worlds, institutions prefer to aim for an analog copy for long-term preservation and a digital copy for easy and readily available access. - Dale
Digital storage for our materials makes sense when either the paper media are inadequate (CAD models and complex databases) or when the physical media are deteriorating. Digital storage may also be appropriate when access is needed for more than a small number of scholars. - Eiteljorg
It will never be possible to convert everything, which means a triage process must be in place. - Graham
I think it very much depends upon what type of collection is involved and the usage it's to receive. - Hamal
...administrators [mistakenly] only see value in the newest technology and do not see any merit in keeping [...] analog materials for future access when the digital environment fails....- Hehman
Holders of copyright must be much more flexible if any progress is to be made in this area. - Hughes
I think it is premature to consider conversion of non-digital information to digital format for the purpose of long term preservation, due to the unresolved problems of managing systems of migration and refreshment, and the cost of doing so. I'm not sure the physical fragility of digital media is a particular obstacle, if we assume the physical media will be discarded with migration. - Jones
I'd like to see digitization happen as rapidly as possible for access and preservation both, but of course preserve originals in the best possible environment. - Lucas
...I'm inclined not to make any NEW investments in techology to handle things we want to preserve "forever", but I WOULD consider investing in a technology which provided long enough shelf-life and media availability to justify the costs, for use with things like office files. - Maher
With technology in such rapid flux, I do not think enough information is available about the shelf life [...] or future retrieval capabilities of current digital storage formats to commit to any particular plan at this time... Much of the media I work with has intrinsic value in its original format. - McLemore
institutions should make it their priority to both 1. Preserve original artifacts through a stable environment and 2. Begin the process of coming to terms with what it means to preserve electronic records. - Messier
There is no way a digital storage format can ever replace a physical collection. It is important to maintain both side by side to allow scholars [and others] access... - Rymer
Standardization needs to be complete before long-term investments are made by anyone. - Schwan
The original image is the best image - a maxim passed down to me in archives school. - Skarstad
...everything should be converted eventually, but that it's probably wise to wait for a permanent support rather than lock into a cycle of copying every few years. - Turner
I think institutions need to be concerned about preserving original formats as well as creating preservation formats. - Vetter
"Institutions should actively investigate digital formats with a plan to eventually convert all collections to a digital storage format" [...] so long as "all collections" means all the material worth keeping accessible within the technology-dependant collections. We have no plans to digitise all of our collections. Need to remember that investigation is still very active, and will influence whatever conversion plans we develop. "Institutions should plan to maintain a dual set of storage for materials" is also true for some collections for some time. For example, we are maintaining both analogue and digital versions of our sound recordings while the analogue set can be accessed, because we want that level of redundancy in case one medium fails - that was our approach with analogue where we used at least two different kinds of tape to spread the risk. With our multi-media collections our preservation path is migration but we are not confident that we can migrate publications successfully, so we will probably need to maintain dual storage arrangements for the "originals" which may be in good condition but not accessible with current technology, and for reformatted versions which have lost some of the functionality and look and feel of the original but at least provide some access using current technology. This approach would be based on an assumption that technology won't always drive us away from accessibility - that there will be recurrent cycles of innovation and incompatability, followed by tools to bring the inaccessible back into view. "Institutions should wait for a digital storage format that addresses the issues of tape fragility and rapid and irretrievable deterioration before making any new investments" is true to some extent, but in many cases we just can't wait: the problem reaches crisis point before the ideal solution is available. - Webb
Broadcasting is already digital, and much new material coming into the archive is digital. Transmission is also rapidly moving to digital. So the 'conversion' is a process being thrust upon us. - Wright

Compression for long-term storage:
Most people answered this question with a "Well, it depends..." Commentaries linked this issue with an institution's budget and with the types of media to be archived.

SMPTE defines compression as "the process of reducing the number of bits required to represent information by removing redundancy." There are basically two kinds of data compression: lossy and lossless. Lossy data compression results in some loss of data and is best suited to graphics and sound, where some loss of data is acceptable. Lossy compression algorithms can often be modified to allow a relative degree of data compression. The more you compress, the less accurate your decompressed data. Considering that compression of data files is potentially a trade-off between file size and a loss of digital information, what degree of compression do you feel is acceptable for long-term physical storage?
a. No compression. 6
b. Only lossless compression is acceptable. 21
c. Moderate lossy compression is acceptable, dependent upon function or purpose of long-term storage and ratio of file space saved. 8
d. Standard should allow for both non-compression and an infinite variety of compression formats. 26
e. No opinion at this point. 1

There is so much variety of function in the data to be stored that it seems best to begin with as high a degree of flexibility as possible in terms of standard-setting. It's pointless, I think, for any one organization (or generation, for that matter) to spend a great deal of time determining precise standards where judgment depends so much on so many variables. - Carli
...lossless compression is the most agreeable, but our member institutions also realize that the function and purpose of creating the copy should drive the creation. As with digital image creation, if the function is to totally replace the original, no compression or lossless compression is the *only* option. But, if the copy is only meant as an access/temporary surrogate copy or to transmit files over the internet, a derivative digital copy with lossy compression could be acceptable. - Dale
...a moderate lossless compression with plenty of error correction capability. - Gaustad
...Just as with artifacts, we will make both general and ad hoc decisions; we need tools to do both. - Graham
For critical records, all original data must be preserved, but the standard should recognize various categories of preservation importance. - Hadley
The issue of compression [...] is highly dependent upon the type of collection and [its] use... We typically do not compress files for storage, but there are some types of information where compression is logical and sensible. - Hamal
Lossless compression is only acceptable if the algorithms for recovering the full information are so established or simple that there is no danger that the compressed information will be loss. Also, the media must be such that an uncorrectable error will not corrupt the remainder of the file (as can happen with a JPEG-compressed image). Lossless compression is the only acceptable solution for preservation [...] because we do not know what kinds of analysis or manipulation we may care to use on the files in the future. An automatic indexing program, for example, might work on a lossless file, but not on a file which has been compressed so that humans cannot detect the loss. These concerns must be balanced against the medium of the original. It is probably better to compress a VHS videotape using MPEG than it is to allow the original to continue to deteriorate. It is just not ideal. - Hirtle
Given the probable improvement of future technologies, and the possible loss of original information before more accurate reproduction is possible, I'd rather use lossless. But economic or other practical considerations tend to intervene, and archivists often have to settle for less than the best. Perhaps a hierarchy of desirables could express the standard. - Lucas
If we are talking about compression then I guess we are talking about migrating the original content from its "native" file format to a compressed format. For many records, this transition away from the native format will be required for the purposes of preservation. If this is the case, then I think compression is acceptable since the link between the preserved information and its original, "native" format has already been broken. Therefore, since the preservation copy is already a "facsimile" there is little lost in leveraging compression techniques. Methods of compression and guidelines for use should be developed. - Messier
Compression depends on the object. Lossy compression is very acceptable for thumbnail images used for browsing, but perhaps not for the larger resolutions that will be sent to the printer. - Ogle
I favor lossless because I cannot predict whether there will be some future cause of loss. - Skarstad
Standards should take into consideration the type of media, for example some loss may be acceptable for audio but not for video. - Vetter
...by allowing choices of compression, the curators and archivists will presumably have storage cost choices. This will allow some items to be preserved with various levels of compression depending upon the "value" of the item. For example, some home videos, collected to document family life, could be converted with quite a lot of compression if we were collecting them for the cultural information about home video production. - Wilson
...Generally speaking we favour lossless compression or no compression for our digital conversion files: we have invested in high resolution capture and want to be able to take advantage of technological advances that will make it feasible to transmit and view high resolution images or recordings. But we have accepted that lossy compression has a place for some material that needs to be kept but where the quality is not a critical preservation factor. - Webb
Some material arrives in the archive already compressed (eg feeds via satellite into newsgathering). The digital transmission will be compressed. The broadcast archive needs to MANAGE compressed materials (cf EC-sponsored project Atlantic), and ensure they are labelled and handled so that the compression does not lead to avoidable deleterious effects. - Wright

Metadata Initiatives
The Dublin Core Workshop and Berkeley EAD (Encoded Archival Description) Projects were the two metadata initiatives that received the most recognition votes, though it is quite possible that the Text Encoding Initiative (TEI) and Recordkeeping Functional Requirements Project (U of Pittsburgh), would have received more votes if we had included them from the very beginning.

Which of the following Metadata projects or initiatives are you familiar with?
a. Warwick Framework 12
b. Dublin Core Workshop 25
c. Berkeley Distributed Video-on-Demand System 3
d. Berkeley EAD (Encoded Archival Description) Project 24
e. Digital Object Identifier System 10
f. MIT Headers and Descriptors Research 1
g. Computer Interchange of Museum Information (CIMI) 10
h. Text Encoding Initiative (TEI) 14+
i. *Recordkeeping Functional Requirements Project (U of
Pittsburgh) 1+
*We did not add this option until late in the survey.
j. None of the above. 8

Also the work of Duranti at the University of British Columbia. - Andolsen
Pittsburgh and Philadelphia Projects
Text Encoding Initiative
Draft Functional Recordkeeping projects at the National Archives. - Conrad
See also commentary.
Your discussion of metadata [...] ignores the work identifying the need for structural, administrative, and descriptive metadata. In addition, we have to remember that one person's metadata may become another person's data. - Hirtle

Cataloging media collections
Overwhelmingly, the institutions represented in our survey use some form of commercial computer application to catalog their media collections. Popular programs like FileMaker and Access were championed, as were specialized packages, such as Endeavor, Cuadra, Cairs, and Tinlib. Of course, some form of MARC cataloging software was mentioned throughout the commentary. It was interesting that, except for MAVIS, no one mentioned software known as "asset management products" that were designed to "to effectively catalogue, index, access, search and retrieve video clips, digital libraries." (See: http://www.cmis.csiro.au/DMIS/VideoTalk/software.html for more information about this type of application.)

How is the cataloging and/or bibliographic material for your media collections now being generated or maintained?
a. in-house via popular commercial software packages, such as Filemaker or Access 23
b. through specialized database management systems, such as Willoughby or Cinebase 17
c. through printed catalogs maintained by staff and distributed to the public 6
d. through out-sourcing or periodic hiring of a temporary staff 0
e. This question is not relevant to my position or institution. 6

Other systems mentioned in the survey:
Nesbit's Media Library System

We converted our paper card catalogues to electronic access over the last 20 years, and have just engaged a new library database management firm, Endeavor... - Carli
WWW access to underlying data in Cuadra Star and Oracle databases. Metadata is stored in the system we use for validating electronic records and is available for answering public queries. - Conrad
Our members catalog into both RLIN and OCLC, the other bibliographic utility. Metadata is also placed in the files (e.g., TIFF headers, SGML, or HTML) themselves. - Dale
We roll our own with postgresql. (http://www.postgresql.org/) - Drelinger
We use MAVIS, Merged AudiVisual Information System, developed by Wizard in co-operation wit National Film & Sound Archive in Australia. - Gaustad
Inhouse through online catalogs (in our case, use of OCLC and RLIN to develop cataloging for our SIRSI Unicorn system). - Graham
Probably through a MARC cataloging software, with finding guides generated in paper form for researcher use (until full retroactive conversion of all Special Collections finding aids is complete). - Hadley
Access is my Bible, but a group I'm associated with also utilizes strange little programs, a kind of high-level mix and match potpourri (because we are a strange little group!) - Hamal
All Library acquired media formats are cataloged inhouse via OCLC in Ohio using MARC standards according to AACR2 and Library of Congress policies. - Hebert
Bibliographic data about all media is included in our online public access catalog according to MARC descriptive standards. - Hughes
In general, I'd say we try to make MARC records work in our NOTIS-based online catalog. Beyond that, our Digital Library collections, such as Making of America, are searchable on a more enhanced level due in great part to the wonders of SGML. - Lowe
My particular project is due to generate both a Filemaker database and an EAD-standard finding aid. - Lucas
Also: through custom-designed databases and application-specific metadata (for example: 1. botanical datasets and 2. image feature information - color,texture) - Ogle
Through electronic records in local and national bibliographic utilities (OCLC). - Olley
We also still have a card catalog with the majority of our collection.- Pollock
For online publications we also use ABN for cataloguing, but we are also developing a metadata archive as part of our PANDORA (Preserving and Accessing Networked Documentary Resources in Australia) Project. This has linkages with ABN but incorporates much more detailed information required for management rather than access. - Webb
These are treated in accordance with standard bibliographic treatment, with full MARC records entered into the Library's online catalog. - Wilding
MARC cataloging on OCLC and distributed via our statewide on line catalog called PALS. - Wilson
All physical-based items (ie sound recordings and multi-media discs, etc) are catalogued via our Australian Bibliographic Network (ABN) which is the national cataloguing system. The records are also included on the Library's own local catalogue. There are other specialised databases we use to record other metadata including preservation management data. Our largest catalogue (2 GB of text) is a bespoke system based on Informix 4GL. We also use commercially-available library packages (Cuadra, Cairs, Tinlib). For documents we have our own registry system written using DataEase. - Wright

Using descriptive information or metadata
This was one of the survey's "dah" questions. Asked how metadata might be used, respondents said in effect that the use of metadata was only as limited as the imagination, though granting "easy accessibility to the material" by way of search and retrieval got the most votes. Copyright figured in a couple of the commentaries, including the need for identifying sources and "terms for usage," and for preserving data integrity.

How would you like to see institutional media collections use descriptive information or metadata?
a. to handle content in a more efficient manner (for example, information about a media object might be entered just once and then distributed to and used by a variety of applications) 32
b. to grant easy accessibility to the material (for example, through a sophisticated search and retrieval application) 40
c. to tag material with original source or copyright information 27
d. to help in the automated creation of Finding Aids 29
e. to assist in linking references from items in your collection to related items in other collections 29
f. do not know yet 7

To allow easy browsing to cut down on the the need to review entire documents and/or productions. - Baker
I do in fact think that source/copyright tagging is of crucial importance in terms of data integrity. - Carli
...bit of a blue sky question here. - Graham
We don't need copyright information as much as identifying the source and knowing if we have approval for public release, or if it's from a stock house, what terms for usage have been set. - Hamal
I would hope to link the visual resources collection to our larger state wide library database system. The faculty are being pushed to teach as many classes as possible by using the Web. - Hehman
There are many layers of metadata that are desirable and necessary for managing all aspects of the digital file, and we need to work out every single one of them. - Turner
For once, "all of the above" seems the right answer! Although we are exploring all of the suggested options, we also expect to find other ways of making use of the descriptive metadata as we move to increasing automation of archiving and retreival, and increasing integration of internal and external systems. - Webb

Media collections on the World Wide Web
Those who knew how they would use the World Wide Web were evenly divided among the limited choices for this question. Commentaries suggested additional possibilities. And though the question used the word "envisioned," it is clear that several institutions are already implementing Web options. A recommended practice might impact on how archivists use the World Wide Web. For example, a common browser might access materials within an archive stored in UPF, either through an intermediary HTML page or through a browser plug-in.

One of the issues involved with container initiatives is how might a specific technical specification impact access to media collections through the World Wide Web. Already one may observe a trend in the software industry in which middleware or intranet products are being adapted for more general use. The latest versions of popular database software are also providing built-in HTML tools and translators. Archival media files and/or information about those files might be available through both database applications and web browsers.
How do you envision using the World Wide Web in respect to your media collections?
a. Finding Aids or bibliographic information only. 28
b. Downloadable "thumbprint" Video. 23
c. Ordering Information. 25
d. Multimedia on demand. 26
e. Do not know. 7

We plan to use an intranet and/or the internet via a browser as The method for searching the index and retrieving audio (both linear audio and reduced bandwidth versions for browsing) from our digital archive. - Baker
As a research library, we are responsible for using whatever method is most appropriate for giving our patrons access to the material they need in a timely fashion. - Carli
Downloadable data in multiple formats -- not just video. - Conrad
The Web is a wonderful way to help researchers find out about collection materials, and can feature materials as exhibits, but it is not a substitute for the reading room. - Hadley
Our image library is on the intranet and to get it there we must go from Access through Cold Fusion to make it Web Happy. We are looking into products (like Cumulus) to find a one-stepper which will still meet all of our database needs and successfully support our web site. - Hamal
...making a robust, viable UPF would be a major contribution and that making UPF "web smart" might not be worth the trouble. In other words: once you get a UPF up and running, let www software engineers figure out how to best leverage the format. - Messier
The Web for our organization will probably be used as a catalog for our picture collection, providing access to low resolution versions of our images. - Pollock
It's a question of time. As the infrastructure becomes able to handle it (the "information highway" as opposed to the Web) it will be possible to have more and more layers available on line, and I feel it is desirable to have them all. - Turner
We are currently constrained by legal, ethical and technology issues that make it hard to predict how we will use WWW for both our sound and multi-media collections. For example, the oral history collection contains a majority of material for which networked transmission has not been approved - this is a significant ethical issue even when we overcome the technological difficulties of passing huge sound files through the network. Likewise, for multi-media publications we face copyright and licencing constraints which may or may not be addressed with technology-based security devices. Of course, for our archive of online publications, the Web is the means of providing access, both to the bibliographic information and to the publication itself. - Webb
I would like to "deliver" some video images so that text alone does not describe the item. But I do intend to put the finding aids on the Web as well. - Wilson
We are part of the EC-sponsored Euromedia project, developing automated indexing and retrieval tools to support online, realtime video delivery in 'preview' quality. We expect to have LIMITED amounts of video online; we already have approx 70k online stills, delivered via intranet to registered users of the system. - Wright

Distributing and Storing Media
This first of two "blue sky" questions was perhaps too narrowly focused to generate feedback from most of our respondents, especially those concerned primarily with electronic records. Nonetheless, a few of the commentaries saw the application of metadata streaming for still image collections. Many other commentaries mentioned the problem of cost versus value in establishing detailed metadata streaming.

How should material that describes an archives collection be distributed and/or stored?
a. Continuously with content : for example, at any given point, one might freeze a video frame and access information about:
+ the video as a complete entity (example: copyright material, original format and running time...)
+ the specific video clip or sequence that includes this frame
(example: number of frames in this version versus number of frames in original, which numbered frame within the sequence)
+ the individual frame's descriptive information (what objects are contained in the image) 19
b. As header information only: information describing "essence" stored as a Table of Contents at the beginning of the video. 13
c. As a separate document: information about the "essence" would load and be stored as a document, accessible via retrieval or "parsing" software. 18
d. Do not know. 13

I am concerned mostly with audio and so I have no opinion on some of these moving image issues. I believe most metadata about audio can be captured in descriptive data entry fields such as: key words, participants... - Baker
[Describing continuously with content] ... might be a realistic undertaking within the budget of a major motion picture or for a uber-publisher, but I doubt individual artists (who can publish their own work on the net), small businesses, and other under-funded or under-staffed groups would be able to afford the time or money to catalog their work with this level of granularity. - Dreilinger
Frame-by-frame description seems extreme for most institutions. But, like all archival description, it should be possible to describe videos similarly at all levels--from a collection of 500 tapes down to a single frame.It's nice to have description attached to the material, but it should also be able to stand alone. Are these mutually exclusive options? -Hadley
Each of these are viable alternatives. Implementation would be dependent on institutions technical resources and abilitiies. - Hedges
"a" through "c" seem nearly impossible for most materials since someone has to oversee the input and value of the massive amount of descriptive information these methods would call for. - Messier
I really like [Continuously with content]. This would also be great for still images. FlashPix suggests this but we are not there yet. - Ogle
I would have video and descriptive information in the same file, as well as stills. - Schwan
We are exploring the use of both header and separated metadata, and think we want both, with a preference for linked separate metadata files. Again, we are very involved in investigating this... [Continuously with content] looks good, and we are after a similar level of retrievability of content and metadata for our sound collection, but it looks like a luxury at this stage. - Webb
For Euromedia, which also uses browser technology and is fully object-oriented, we have a hierarchical storyboard as a metadata descriptor, with keyframes, and with text associated at four levels: frame, shot, sequence, programme. For security, this information could be stored with the digitised data as well as separately. At the object level this distinction is invisible. - Wright

Metadata Streaming
How deep should descriptive cataloging go? If one picture is worth a thousands words, how many of those words do you include in your metadata? The UPF is not designed to answer these questions specifically, but it would establish a foundation that would help enable archivists and other information professionals to resolve metadata questions for their own institutions. One area with which the UPF might deal explicitly is the placement of copyright information, and the identification of "original works" received the highest number of votes. The UPF might also grapple with "versioning": assigning unique identifiers to the first generation object or compound document, then referencing them with unique identifiers with subsequent generations.

Metadata streaming, a kind of sequentially flowing descriptive data, relates to the EBU/SMPTE concept of a "new class of enablers of systems using bit streams for program material exchange." There are many practical applications for this metadata streaming, including a "standardized descriptive data to help in locating material through various database entries." Would you favor a standard by which:
a. each frame of a digitized film or video identifies a relationship to its shot, sequence, and entity. 28
b. actions within a film or video sequence are described. 27
c. objects and placement of objects within an image or frame are catalogued. 29
d. "original works" are identified through standard fields of information. This standard may stem from an established practice, but it must encompass the full gamut of media sources. Just as published works are assigned an ISBN, standard identifiers could be assigned to materials that are digitized for preservation. Distinct identifiers could also be assigned for distinct versions of this material. 33

...integrity and intellectual associations are more important than image-related associations. ...I feel strongly that ["orginal works" are identified through standard fields of information] is paramount for any standard. - Carli
...will we describe the boats in the sea around the Statue of Liberty, considering that one day someone might be interested in this type of sequences? or do we decide that describing the Statue of liberty wil 'implicitly' indicate to a future user that there might be boats around it ? - Despres-Lonnet
The metadata should be able to be applied at this level automatically from a top level record. Enter the title in a base record and when the data is streamed out the title is added to each individual frame. Tools to make this information easily replicated will be crucial. - Hedges
Artists and art students are very aware of originals and "restored" / colorized versions since that is what they sometimes learn to do in their craft. But the industry should not make it so difficult that artists and students cannot be allowed their creativity - They are your future. Think of Steven Spielberg. - Hehman
I think these sound good, but not all will take off at the same time. Some efforts will be the realm of libraries, while others will be left to more risk-taking ventures for entrepreneurial publishers. - Lowe
Yes! Yes! (Oh, excuse me . . . ) It's just that it seems so futuristic compared to not being able to afford to make a reference copy . . . - Lucas
The Standards of Practice and the Code of ethics for the American Institute for the Preservation of Artistic and Historic Works might be interesting in this context. - Messier
All of these features would be so desirable it isn't funny! - Turner
Metadata will be tricky and big. - Weiss
Version control [...] is a very important part of the projects we are running here -- without it we don't believe we have effective preservation because we would lose the provenance of the copies! This is an issue for users but also for the management of the archive. ...we [...] see [this level of retrievability] as a relatively low priority for our kind of collection. However, we are involved in a project looking at adapting that kind of moving image-based software to audio. [Describing actions within a film or video sequence] ...would be very useful, but it is much harder to characterise significant reference points such as movements, colours or shapes in oral history sound recordings. ...to what extent are these preservation tools, worthy though they may be? Is there a danger of access tools distracting us from the preservation challenges? - Webb
We train cataloguers to have a principled, consistent approach, but also they have to vary this approach depending upon the nature and value of the programme (entity). But [Each frame identifies a relationship] and ["orginal works" are identified through standard fields] are both essential to the integrity of the labelling. - Wright

Closing Thoughts
Closing commentaries reassured us that a standard is imperative to any digital migration initiative. Areas discussed in this section that were not singled out in the survey questions include the need of a preservation format to include a common lexicon, a recommended practice or guideline, and even a call for standardizing digital equipment. Education was also stressed throughout the commentaries; archivist organizations should consider establishing online tutorials for its members. Whatever one feels about the specifics of the Universal Preservation Format, it cannot be denied that many archivists are reluctant to invest their time and money into migration projects that are perceived to be short-term or intermediary solutions. Because our project focuses on such crucial issues as platform independence and embedded metadata, we feel we have the start of a workable solution that will last well into the next century.

We realize that concepts touched upon in this survey may be new to you. Here is an opportunity to convey your thoughts or concerns on the development of a digital preservation standard. Based on your understanding from reading this survey and supplementary materials found on our web site, how might this project for a universal preservation format orframework benefit you or your organization? And what activities would you like to see this project undertake that perhaps have not been addressed?

I am in the position of gathering information to support the conversion of materials in an obsolete format to a useable and, at the same time, archival format. We have very limited funds so I am always looking for information that will lead me to the safest choice for our archival show tapes and field tapes. I am glad there is a project that is looking at the idea of a universal format since format wars between manufacturers have proven a nightmare for archivists in past years. This project makes me feel that there is help for those of us with no idea of where to turn as the digital age takes over. Thanks! - Allen
A cohesive standard would allow us to work intelligently with our clients to develop a range of preservation options and approaches that would provide a large measure of assurance that the digital material would be available and could be transferred to newer media with less effort than the current situation requires. - Andolsen
The benefits I see are standards that describe how different parts of an archive solution should interact.The media [that is] used and how they get there will change over time and so by defining the interaction rather than file formats you open things up to rapid development. - Baker
Most important would be to settle on the standard for picture and sound and then a medium, such as DVD as the recommended media. - Bailey
We are in the process of developing metadata requirements for digital items in our repository. A system identifing the archival or "best" version along with related and derivative digital items would be useful. - Bramel
...I preceive a time in the not too distant future that our young will no longer go to school as they do now. Digital data will be employed via virtual reality technique and the use of drugs to educate our children much better and faster than we ever imagined possible. This same technology will also make juvenile crime, prisons and jails obsolete , or almost forgotten. What and how we store digital info now will play an important part in what happens then. - Burris
We are developing a digital library, including both static media such as images and dynamic media such as streamed audio and video, using the IBM Digital Library version 2. This system will be used for three primary purposes: a) provide infrastructure for on-demand and long-term centralized multimedia storage; b) provide system for library special collections to be converted and stored in digital format; c) static and multimedia courseware. - Calarco
...my professional interest in digitization relates mainly to paper-based materials. The idea of the potential complexity and cost of preservation of other media is staggering to me. My husband works in the film department of the George Eastman House, so I have some idea of the technologies and costs involved in analogue preservation; my sense is that budget limitations will be a key limiting factor in digital preservation as well, and that the development of standards will have to take that into account, as well as the fact that technology is now unfolding so fast that whole new paradigms can be expected to come into play in even the near future, much as Web access has done in the recent past. I actually expect, in fact, that the SMPTE initiative's most important contribution may well be to create such a new paradigm, simply by concentrating on the particular needs of archival storage and access. Best of luck! - Carli
I think your project may be too narrowly focused. The survey shows a lack of understanding of preservation practices/ initiatives for archival electronic records that are already underway. Developing a format for long term preservation of graphics, video, and audio is a critical issue. But the work should build upon work that has already been done in the archival world with text and numeric data. - Conrad
I think my comments to the questions above express my concerns and the concerns of our members. While RLG does not own any collections, the collections of our approximate 160 members represent some of the greatest collections that exist. I would be very interested to work with SMPTE on this (or have the opportunity to attend meetings where possible) because the Research Libraries Group is also working with and on behalf of our members to help establish best practices for preserving and providing access to materials through digital means. The website mentioned above gives specific information about the working groups already formed, their charges and progress reports. As well, please see the Digital Initiatives portion of our web site at for further work and involvement in the evolving metadata standards. - Dale
I want to learn more about UPF itself, rather than the issues that led to this survey (yes, I'm backing out and exploring your web site next) if results or a summary of this survey are available, please share: sed@netcom.com or sean@interactivate.com good luck! - Dreilinger
I'm following your work because of my activities in the library community. What you're doing looks good; keep up the work. - Graham
I have a lot more to learn and consider! Glad you're working on this; I'm a firm believer in the usefulness of standards. - Hadley
If you find a great way to catalogue videos...and get clips onto the web...I have not yet investigated any of those avenues. It would also be nice if the TIF were standardized (or is that too much wishful thinking?) - Hamal
Will follow the progress of the project and comment at a later time. Thanks for the opportunity. - Hedges
I need to know what are the possibilities for future preservation of the images which I now provide for the teaching of art at my institution and how I can work to preserving those images in a digital world so that others can learn from the visual arts. - Hehman
Being a preservation administrator, my main concern is how long this information will last in a digital format. Recognizing that discs now have a finite life of only 30 years, digitization is not a viable preservation option at this time. If, by "universal format" you mean a standardization of digitization equipment and process, I think it would be useful. - Herrick
Question 3 under the wrapper format makes it clear that you are looking at video preservation (as one would expect from SMPTE). Don't you think that calling this a "universal preservation format" and a "digital preservation standard" is a little pushy? Do you really envision your standard working for audio, bitmapped and vector images, graphics, text, etc., etc., etc.? - Hirtle
This needs to be an international effort. - Hughes
I would like to see more opportunity for input about the capture and playback equipment itself. I am also curious if the format would still have NTSC / PAL standards or if it could/ would be truly "universal." I am also interested in the length of the tapes and what institutions would need or prefer. - Lee
In my opinion one of the major aspects of the quest for a Universal Preservation Format is the perceived (and in my opinion misdirected) need for a single format to replace the wide array of video/audio/data and multimedia formats in the marketplace. I do not feel that this is realistic or viable now or in the forseeable future. Indeed - in UNIX parlance - I see this effort as YAFF (Yet Another Format Format). Since this new format would have to be migrated to/from existing formats, would have to be used along side of existing formats for production purposes, and also be necessary to output (and input) to different media types, I see it as being yet another format that will necessarily live with formats in the video, audio, and data world... and as such I consider it largely redundant. Perhaps when we have network and local bandwidth that is so fast and storage that is so inexpensive that the network itself becomes the "virtual archive" because it is just as fast as and as cheap as local storage ... this idea would be more appealing to me. Indeed, with RAID mirroring type methodology to ensure reliable data reconstruction from different data storage node sites on very fast data networks - one could have a "virtual archive" and a distributed mediaLESS way of storing information.... that would be the time to seriously consider these issues - but I am afraid that these advances are more than a few years away - and as such I fear that this effort will spawn YAFF that will have to be supported - along with all the others thereby just making matters worse. One aspect that I do find appealing, however, is the storage of other data associated with the a/v records in the same way as the a/v records. For instance shot lists, EDL, and storyboards could all be stored along with the different tagged versions of the a/v material. Good luck on the project. - Lindner
Just as traditional libraries have been doing for centuries, digital libraries will have to contain the world (even worlds). To do so digitally means to model the world's structure with conceptual maps and other frameworks. That task, awesome though it may be, is not as terrible as handling exceptions to that overarching framework. I believe it was Pope who said: "The exception tests the rule." We see that over and over in libraries, and new media are not new in that regard. - Lowe
Education. Especially to library schools like mine: San Jose State University School of Library and Information Science. Workshops. Information about costs, groups, opportunities, training. How do we get from vinegar syndrome in a metal can to metadata streaming? - Lucas
First of all, we would like to preserve our material. Secondly, if we had our material in a digital form, we could provide easy acces to our users and other people in Greece to our Collection. Additionally, we could make links with other collections and provide a more wholesome aid to the researchers. - Mamalinga
I think a standard needs to be developed that will easily accomodate changes in technology, and I think managers and administrators need to be sold on it, not just archivists and records managers. - McLemore
As an art conservator, I am concerned that any preservation effort for electronic media attempts to document and preserve an artifact's integrity and original artistic intent. While I understand that for many electronic artifacts, preservation of the "native" format will be impractical for the purposes of preservation, I am also convinced that there is value in preserving the intrinsic characteristic of the original to whatever extent is possible. For instance there are certain attributes of 2" video and that are part of the material record of a piece and will remain significant even if the piece (again for preservation purposes) is long disembodied from the original 2" tape. Therefore, the material changes that occur as visual information is preserved should be documented. This documentation (including any visual enhancements, artifacts or glitches that occur as part of a preservation program) should become part of a permanent record always associated with the piece. Therefore, a video scholar looking at a 1970's video in a 100 years will not have to ask the question "Am I looking at visual content actually dating from the 70's or am I seeing the cumulative result of manipulations made over time to preserve the piece?" - Messier
A concern that I don't see here is the need to automate these cataloguing activities. It just is not feasible in many cases to manually enter all the info needed for each object in a large collection. Unless a new standard can be reasonably implemented, it is hard to see how it can be entirely useful. (I don't have the answer, but it is a concern of mine.) Automatic identification of image feature information is an active research area in my own project - content analysis using computer vision techniques. I would suggest that any standards for images - still or otherwise- should be flexible enough to allow this kind of info to be stored as it becomes available. - Ogle
Our organization is committed to preserving the history of chemistry. To do that we must preserve not only books and photos, but personal papers, video, film, sound recordings and objects. As a museum and a library, it would probably be enough to preserve and provide access to materials where possible and digitize only the most fragile items. But in our time it is possible to do more. The oldest book we have may be 200 years old. Are we going to be able to do better than that with the pictures and documents and in your case, film and audio, we are producing today? How much longer can we preserve items that are hundreds of years old? We Archivists, historians and librarians are in it for the long haul and we need standards that are going to be sustainable for centuries. - Pollock
Any "universilizing" and standardizing of methods for conversion and storage of data and media is welcome if for only one reason: It allows for more time and energy to be devoted to the creative process, which includes the research process necessary to stimulate new ideas. - Reilly
I am currently researching digital preservation in order to draw up a list of guidelines for our users. This would include a section on best practice. Consequently ANY information on preservation is useful to the Visual Arts Data Service, especially since we will be archiving datasets. We will need to establish our own set of rules and regulations for digital preservation and ensure that these will not become obsolete over time. - Rymer
What type of computer should i buy to be ready for any computer-based archiving technology that is about to come out? Should I keep digitizing clips using jpg and other apple applications? Thinking about a mac 9300 when we get some money. - Schwan
First, we need to talk about definitions. "Archival" means capable of lasting 200 years. I am a preservation librarian. - Sinclair
I feel like I jumped in on the middle of a discussion. My main concerns for the collection are accessibility and preservation. I see a real need to transfer or copy the collection onto a usable format so the films can be used in a classroom setting. I also want to make sure that the original, master film survives. I would like to have special curatorial training so I can make more informed decisions about the collection. - Skarstad
It's clear that a universal preservation format would solve most of the important problems in the area of digital preservation that archivists are facing today. There's a shift in preservation theory because of the technologies involved in digital-format documents, and much of it involves copying and migrating data because the physical supports are not stable enough for long-term storage. A universal physical support that is stable would relieve the first problem, and a complex and thorough enough metadata format would relieve the migration problem--in other words, documents would become independent of hardware and software. This can only benefit everyone who works with archival materials. It's as if we're presently in the midst of a huge set of problems that seemingly have no solution, and some kind of universal preservation format would finally signal the arrival at some kind of plateau at which we could stop worrying so much and get on with our work. I'll be watching this work with much interest. - Turner
I see this project for a "Universal Preservation Format or Framework" as being an opportunity to create a focus-point, a standard to which the caretakers of all media may resort. Thus, I believe it should be capable of handling all media. I will bookmark this site and try to become more aware of your efforts. - Vanderlinden
It would be useful to create a common language (if possible) that included terminology familiar to archivists/preservationists and the terminology familiar to computer/web/digital professionals. It is still difficult for various professionals to talk to each other. - Vetter
Not clear on the benefits for us. That reflects two things: - not sure how well the approach extends beyond moving image collections. I'm sure that if it works it will be applicable to some of material we are dealing with, but we haven't thought the idea through sufficiently to see immediately how it will work for us. - some concerns about its workability. Seems to be a good way of reconciling a diversity of standards at acquisition so that a limited number of formats have to be dealt with for preservation and access, but it requires an input of standard data formats that can be easily converted and it only expands the migration period, rather than eliminating the technological obsolescence problem. Still, the preservation of digital collections is almost certain to be made up of a patchwork of interim solutions that buy us a bit (or byte) more time. Perhaps I would be more comfortable if it was described as a "Universal Archiving Framework". We need to learn a lot more about UPF before making any sort of definitive judgments for this institution. Thank you for the opportunity to comment. - Webb
The problem that I don't understand how it will be addressed is one of copyright and ownership of material once digitized. in this regard, would authentication stamps be available to determine if digitized data had been doctored? - Weiss
Since we exist in a world where interdependence is critical, perhaps the achievement of a standard is more important than all of the specifics within the standard. Whatever happens we will have an increasing need for standards so that we can ensure preservation and access for the future. - Wilding
I can tell from some questions in this survey that my knowledge of digital standards and new developments is lacking. I would like AMIA to develop coursework, for conferences or on line, to help others like me learn more about the concepts and ideas you introduced in your survey and supplementary documents. - Wilson
Until we can move from a 'hold-it-in-your-hand' (physical) format onto a fully 'down-the-wire'method of working, we will be faced with format conversions of the physical items -- meaning there won't be a preservation format, only a succession of physical formats incorporating (hopefully) more and more of the metadata and structure required by the ultimate 'down-the-wire' format. - Wright

Followup Questions

Should the UPF require that the source code used to read data be encapsulated in the storage format? Or should a Recommended Practice allow the option of storing the Rosetta stone as a separate file that might be executable code but would be required to operate on a set of specified platforms? Should the Rosetta stone simply be an ascii file of pre-compiled code and the compiler requirements stored separately or contained within a larger application? Should the UPF recommend a list of specific file formats that would be defined by the Rosetta stone?

Related to the Rosetta stone is the issue of platform independence. In respect to a digital archives, how important is platform independence to the concept of a digital preservation format? Is this a "wait and see" issue, or should the UPF make a definitive statement on the requirement of platform-independence?

How should unique identifers be used for archives? Should there be a central registration body for these identifications? At what level should material be tagged? Should the unique identifiers be allocated to one storage area?

What is the solution for organizations with limited human resources to maintain a wide range of skills? What are the implications of technical expertise upon a universal preservation format, or upon any techincal standard adopted for digital materials? What would be the process for "updating" the Rosetta stone? Another way of asking this question: Should a Recommended Practice advocate technological "Ease-of-Use"?

As we presented our initiative to groups concerned with preserving analog media, we grew aware of a crippling gap between the language used by archivists and the language adopted by digital communities. Words casually chosen by digital initiatives often have deep analog roots. As a result, we wish to preface any Recommended Practice with a glossary of terms that might help bridge the analog-digital domains. Where do we find acceptable source definitions for such a cross-domain dictionary?

One of the points we try to emphasize to archivist groups is that the UPF would help carry on the traditional practices of archivists and librarians by providing a robust framework for digital materials. For example, by incorporating such concepts as unique identifiers and by "gluing" certains kinds of cataloging information to the stored media, we hoped to perpetuate the two principles that are the foundation of standard archival practice: Provenance and Original Order. We also realize that many variations of these practices have evolved, and that no two archives will follow the same guidelines. Are archiving practices standardized enough to be used as a model or metaphor for designing digital archiving practices? Or is it permissible to explore entirely new methods?

Is the available technology adequate to address the problem of digital storage?
Storage technology evolves at a dizzying pace. Recently, we looked at PaperDisk from Cobblestone Software, technology that prints digital information on plain paper through a common laser or inkjet printer, then reads it back into a computer through a standard flatbed or hand-held scanner. We have also heard about using DVD as an archival storage medium. More exciting for storage capacity and permanance is the Pancake Disc developed by Los Alamos National Laboratory. Norsam Technologies, which sells this product, claims that Pancake Disks will last thousands of years without any special storage requirements. Norsam is issuing two products on the Pancake this, one for digital storage and one for analog storage. The analog product, called HD-Rosetta, is an intriguing variation on microfiche technology. You access the stored records as a series of images through a high-powered microscope. The Company's HD-ROSETTA writer can record approximately 90,000 8 * x 11 analog images (in an "xy" coordinate format which it proposes to market primarily as a historical preservation and archival data storage product) on a two-inch Pancake Disc. The other product, called the HD-ROM, developed in a joint venture with IBM, can store 650 gigabytes, which is 1,000 times the capacity of today's CD-ROMs, and may eventually hold up to 12 Terabytes or 12 thousand gigabytes. Norsam's technology uses charged particle beams, rather than laser beams, to write data onto disks.

When the problem of physical storage is resolved, does compression cease to be an issue for archival storage? If a physical storage media can be virtually limitless, then why not store everything in uncompressed format? Or are there other issues to consider, such as speed of retrieval?

Can the Universal Preservation Format in practical terms be envisioned as a "file interchange format" as well as a "storage format"? A "blue sky" use of the UPF would be that "select sets" within a software application, such as an asset management program or even a video editing application, might be saved or exported in UPF in the same way text files are saved or exported in RTF or ASCII. Records in this format could be saved to, say, a zip drive or tape or burned into CD-ROM, and then imported into another program that understands UPF.