NDLTD Union Catalog Project

About

NDLTD is comprised of many individual member institutions and consortia, each of which has a process in place for archiving and distribution of ETDs. The Union Catalog Project is an attempt to make these individual collections appear as one seamless digital library of ETDs to students and researchers seeking out theses and dissertations.

While ETDs are owned and maintained by the institutions at which they were produced or archived, it is possible to give searchers the appearance of a single collection by gathering all the metadata (title, author, etc.) into a central search engine. Then, when a potentially relevant document is found, the user will be redirected to the institution that contains the actual document.

This approach of making metadata available to aid in discovery of resources is supported by the Open Archives Initiative (OAI), which has developed a protocol for exchanging such metadata. Using the OAI's Protocol for Metadata Harvesting, individual sites can make their metadata accessible to providers of search and discovery services, while still maintaining complete control over the resources.

While membership in NDLTD does not require making metadata publicly available, it is highly recommended. NDLTD is currently developing global resource discovery services  to promote the visibility of ETDs at individual member sites. The easiest and recommended way to contribute to these and future endeavours is by supporting the OAI protocol at individual institutions!

More information about this project can be found in the following articles on NDLTD in the September 2001 issue of D-Lib Magazine:

• Networked Digital Library of Theses and Dissertations: Bridging the Gaps for Global Access - Part 1: Mission and Progress
• Networked Digital Library of Theses and Dissertations: Bridging the Gaps for Global Access - Part 2: Services and Research

How to Make a Collection Accessible

1. The person(s) in charge of the ETD collection needs to decide to publicly export the metadata, and settle on a set of rights and conditions under which this can be done. For most institutions this is usually a statement of the form "Metadata may be used for non-commercial purposes only" or "Metadata may be used for commercial and non-commercial purposes" (This statement is added onto all exported metadata).
2. A technical person (programmer) may need to build support for the OAI Protocol for Metadata Harvesting into the archive. This protocol is a web-based service that can be implemented using CGI scripts, Java servlets, PHP scripts, ActiveX controls, or any other web server back-end processing mechanism. The following list recommends steps to follow to achieve OAI-compliance for various technologies upon which ETD collections could be based:
• ETD-db software from Virginia Tech

There is a drop-in package available to easily add on OAI support for all versions of the ETD-db software from v1.0 to v1.7 (and possibly beyond). Instructions and downloadable files can be found here.

• EPrints software from University of Southampton

All versions of the EPrints software contain support for the OAI protocol. Check the website and documentation for more information.

• DSpace software from MIT/HP

All versions of the DSpace software contain support for the OAI protocol. Check the website and documentation for more information.

• Custom-built or other database-driven software

The OAI website has tools in various programming languages that may be used as templates or starting points in implementing the OAI protocol. Many of these templates require only modifications to access the database and format the data appropriately. Any custom solution will require reading through and understanding the OAI Protocol for Metadata Harvesting. The most common implementation issues that arise include:

• How to provide unique identifiers for each record.
• How to access records by the last modification date.
• How to convert/map the existing metadata into Dublin Core as required by the OAI protocol.
• Whether to divide the records into sets and, if so, how.

Further assistance can be sought from members of the OAI implementation mailing list. See the OAI website for details on how to join.

• Website (one or more HTML pages)

The OAI protocol is primarily devised to share metadata contained in databases, but it is possible to use data from files, if those files have a standard format. As before, the tools on the OAI website are useful as a starting point for any implementation.

Another option is to create a database or set of XML files corresponding to the ETD collection and then use standard OAI software tools.

3. Implement support for the ETD-MS metadata format. This is a metadata format that is specific to ETDs and allows for the exporting of fields specific to ETDs which are not supported by other metadata formats.
   (The ETD-db OAI extensions already have this support built-in).
4. Test your OAI interface with the Repository Explorer.
5. Register your OAI interface with the NDLTD Union Catalog Project by emailing your OAI baseURL to hussein@cs.uct.ac.za along with the metadata formats and sets (if ETDs are located in sets within a larger repository).
6. After your OAI interface is confirmed to be operational, your metadata will be harvested on a periodic basis into the OAI-based Union Archive maintained by NDLTD (OAI baseURL:  http://union.ndltd.org/OAI-PMH/) which provides a central merged metadata archive for service providers such as the NDLTD Union Catalog.
   Note: since harvesting occurs on a periodic basis (usually daily), changes will not show up immediately.

How to Provide Services

While it is not anticipated that many institutions will want to provide global resource discovery services, all NDLTD members are welcome to use the data in the Union Archive to supplement their own services or provide new and enhanced services to the NDLTD community.

To obtain the data, a service provider must use an OAI harvester (see OAI website) to periodically gather data from the Union Archive located at the baseURL: http://alcme.oclc.org/ndltd/servlet/OAIHandler

Contact

If you have comments, questions, or suggestions, contact hussein@cs.uct.ac.za.