6C.
Technical Infrastructure:
FILE MANAGEMENT
Key Concepts
introduction
keeping
track
image databases
storage
storage types
storage needs
STORAGE
Typically, the component of the technical infrastructure that gets the most attention is the capture device, because it interacts directly with the tangible object being digitized and has the greatest influence on the quality and fidelity of the resulting image. Much less thought goes to the storage medium on which the captured bits will reside. This is unfortunate, since poor choices in storage technology can be detrimental to every step of digitization and can lead to production slowdowns, inefficient delivery, needlessly high short- and long-term costs, and the corruption and loss of data.
The reluctance to focus on storage technology is understandable. Storage devices perform a routine and utilitarian function within the digitization chain and are easy to take for granted. Additionally, mass storage is one of the most competitive and rapidly advancing computing technologies. As a result, it can be quite daunting even for the technically savvy to keep up with the ever-changing storage landscape, let alone understand some of its more complex aspects.
Except for relatively small installations, decisions about storage technology will probably be made in close consultation with systems staff. For that consultation relationship to be an effective partnership, knowledge of the basic terminology and concepts lays the foundation for asking the right questions.
General criteria for evaluation include:
Rapid changes in storage technology have altered the impact of these criteria on digitization planning. In the early 1990s, storage was expensive, slow and of relatively limited capacity. Projects creating multiple gigabytes of image files experimented with various new (and often proprietary) optical disk technologies in order to find affordable means to safeguard their new digital treasures, often sacrificing speed and reliability in the process.
Today, the spinning magnetic disk drive is the undisputed king of storage. For all but the most ambitious projects, the production phase of digitization will be well-served by everyday, inexpensive parallel ATA drives, now commonly available in capacities of 120 GB per drive and interface transfer speeds up to 133 MB/second. Neither speed nor capacity is likely to create performance bottlenecks.
Today, the storage challenge is more likely to arise in the delivery stage, from efforts to consolidate disparate digital collections into a large digital library, sometimes containing terabytes of data (a terabyte is 1000 gigabytes). Efficient management, delivery and maintenance of such collections is not a trivial task, and the premium pricing of large storage arrays with high reliability, excellent performance and integrated backup facilities can still strain budgets. Smaller collections that are in great demand may also require higher performance storage systems.
Within the range of available storage technologies, it is generally safest to choose one that is at or near its peak of popularity and acceptance. Technologies too close to the leading edge may never achieve widespread support from manufacturers or users, leaving early adopters with orphaned, unsupported hardware or media. Technologies too close to the trailing edge may suffer from declining product support and diminished upgrade paths. Also, don't buy substantially more storage than you think you'll need within the next couple of years. Under-utilized storage is not cost effective given the rapidly declining price and relatively short life expectancy. Most storage systems today are designed to accommodate incremental growth.
© 2000-2003 Cornell University Library/Research Department
