1st International ICST Conference on Scalable Information Systems

Research Article

Indexing and searching tera-scale Grid-Based Digital Libraries

  • @INPROCEEDINGS{10.1145/1146847.1146850,
        author={Robert Sanderson and Ray R.  Larson},
        title={Indexing and searching tera-scale Grid-Based Digital Libraries},
        proceedings={1st International ICST Conference on Scalable Information Systems},
        publisher={ACM},
        proceedings_a={INFOSCALE},
        year={2006},
        month={6},
        keywords={},
        doi={10.1145/1146847.1146850}
    }
    
  • Robert Sanderson
    Ray R. Larson
    Year: 2006
    Indexing and searching tera-scale Grid-Based Digital Libraries
    INFOSCALE
    ACM
    DOI: 10.1145/1146847.1146850
Robert Sanderson1,*, Ray R. Larson2,3,*
  • 1: Department of Computer Science, University of Liverpool, Liverpool, L69 3BX, U.K.
  • 2: School of Information Management and Systems, University of California, Berkeley
  • 3: Berkeley, California, USA
*Contact email: azaroth@liverpool.ac.uk, ray@sims.berkeley.edu

Abstract

The University of California, Berkeley and the University of Liverpool in conjunction with the San Diego Supercomputer Center are developing a framework for Grid-Based Digital Library systems and Information Retrieval Services (Cheshire3) that operates in both single-processor and distributed computing environments. In this paper we discuss some results of testing Grid-based parallel approaches in indexing and retrieval for a variety of information resources, ranging from small test collections like the TREC and INEX collections, to medium-scale metadata collections like Medline and a test version of University of California Online Union Catalog, MELVYL (with 15 million and 16.5 million records respectively) ranging up to large-scale collections like the US National Records and Archives Administration (NARA) Preservation Prototype. This paper examines our approaches to indexing and retrieving from these collections and the architecture of the system that supports them.