WAIS & GOPHER JANUARY 14, 1993 OVERVIEW Wide Area Information Server (WAIS) is a suite of programs and protocol for creating and searching full text databases. The University of Minnesota's Unix Gopher (UMN) server, gopherd, can easily access WAIS databases thus integrating these materials into gopher-space. This document outlines the necessary steps to have your gopherd server use WAIS databases. It is concerned with only the most typical of uses. This is a working document and may contain errors and omission. If you have any comments, please send them to Andrew_Gilmartin@Brown.Edu. The latest version of this document is available via anonymous ftp at ftp.brown.edu in the /pub/gopher directory. WAIS AND GOPHER AVAILABILITY The WAIS software is available via anonymous ftp at think.com in the /wais directory. The file wais-8-b5.tar.Z is the Unix source code for the full text indexer, database server, and a couple of Unix clients. In addition, there are a number of machine specific clients available there. The file WAIStation-0-63.sit.hqx is the Macintosh client. To have gopherd use WAIS databases, however, you only need the Unix source code. The Gopher software is available via anonymous ftp at boombox.micro.umn.edu in the /pub/gopher/Unix directory. The file gopher gopher1.03.tar.Z contains the latest version of gopherd. You need at least version 1.0 to use these instructions. INSTALLING WAIS AND GOPHERD Both WAIS and Gopher Unix software are rather complex. It is best to test and install each separately before attempting to join them. If you are not familiar with Unix software installation procedures, I recommend that you work with someone who is--the process is messy. Install and test the WAIS software first. While the WAIS installation is not documented well, the software is very portable. Follow the wais INSTALLATION instructions very carefully. You do not need to make the X Windows Systems or Emacs client; to prevent this remove from the Makefile's default rule the x and tags dependencies. I recommend that you install and test gopherd before trying to link in the WAIS code. Once gopherd is working, you will have to remake it for WAIS. To incorporate WAIS into gopherd, carefully follow the "Full Text Indexing Setup for WAIS" installations in the file gopher/doc/INSTALL. PROTOCOL LINKS Gopher can access existing WAIS database servers via the WAIS protocol. To do this all you need do is to place a WAIS database source description file (.src) into your Gopher data directory. To test your gopherd, copy the following to the file aesop.src (:source :version 3 :ip-address "18.85.0.233" :ip-name "bloat.media.mit.edu" :tcp-port 8000 :database-name "Aesop-Fables" :maintainer "jh@bloat.media.mit.edu" ) Place this file in the root of the gopher directory (while testing). If you want to give the file a more descriptive name create the file aesop.src Name=Search Aesop's Fables (Testing) and place this in the root data directory's .cap directory. Delete the root data directory's .cache file and you are ready to test. Using any Gopher client that supports searching, connect to the root of your Gopher server. You should see a search menu item named "Search Aesop's Fables (Testing)". If you don't see this check that you followed each step above ù Created the WAIS database source description file aesop.src at the root of the gopher data directory ù Created the Gopher .cap file aesop.src in the root gopher directory's .cap directory. ù Deleted the root gopher directory's .cache file. To test the search, look for "hare" (a common Aesop character). After a short while, your Gopher client should display a list of 10 items, among which is "The-Lion-and-the- Hare...". Success! A list of publicly accessible WAIS databases is kept at quake.think.com. The list, wais-sources.tar.Z, is available via anonymous ftp in the /wais directory. You can use this to build your own list of available WAIS databases. Alternatively, you can use the shell script waisload to do it for your. Waisload is available via anonymous ftp at ftp.brown.edu in the /pub/gopher directory. DISK LINKS Having tested gopherd with a protocol link you are ready to test a disk link. A disk link enables you to access WAIS databases available within gopherd's file system. For example, you might want to create a full text index of all the documents within your Gopher data directory. To test disk links you will construct a simple wais database of several of your files. Create the following directory structure at the root of your gopher data directory ./myfiles/ ./myfile/.waisindex/ Place several simple text files in ./myfiles. The WAIS database's index and auxiliary files will be placed in ./myfiles/.waisindex. To create the database issue the command waisindex -d ./myfiles/.waisindex/index ./myfiles/* This will build a WAIS database of all the files in ./myfiles and place the database's index and auxiliary files in ./myfiles/.waisindex. To have gopherd recognize the WAIS database files for searching, create the following .link file in ./myfiles Name=Search My Files (Testing) Type=7 Path=7/myfiles/.waisindex/index Host=+ Port=+ Using any Gopher client that supports searching, connect to the root of your Gopher server. You should see a folder item "myfiles". Enter this folder and you should see a menu listing all your files plus the "Search My Files (Testing)" search item. If you don't see this make sure that you carefully followed the steps ù Created ./myfiles and ./myfiles/.wasindex directories ù Copied several files to ./myfiles ù Built the WAIS database ù Have a .link file in ./myfiles To test the search, look for a word--other then connectives- -you know is within one of your files. After a short while, your Gopher client should display a list of files that contain the search word. Success! WAIS INDEX TO WHOLE CWIS To build an index for the whole content of your CWIS do the following. mkdir ./.waisindex find ./ \ \( \ ! -name .\?\* \ ! -name core \ ! -name adm \ ! -name bin \ ! -name dev \ ! -name etc \ ! -name usr \ -print \ \) \ -o -prune \ | waisindex -d ./.waisindex/index -stdin Note that this uses GNU find, the standard Unix find does not have the -prune option.