I added the X_db, SQL_db and MySQL_db classes, and the hierarchy of classes becomes:
X_db | `--- Database | `------ DB2_db | `------ SQL_db | `------ MySQL_db
The only MySQL specific code is in the MySQL class, which uses the API of mysql (library libmysqlclient.a). The SQL code doesn't serialize the information as the DB2 code does, instead it uses specialized tables for all information (URLS, URLINFO, WORDS, LINKS and so). The Perl scripts for creating the SQL tables in the database are found in htdig3/contrib/sqldb. So I had to change all Database::Put/Get() calls everywhere to specialized calls. That's what I've put in the X_db class. To keep the DB2 functionality unchangeD, I added a config variable 'db_type' which I used where the Put/Get methods where called (if we use SQL then the new functions will be called).
There is a patch for 3.1.5 available with the changes to the 3.1.5 version of ht://Dig. It contains a README on how to apply the patch and use htdig with MySQL. You will need a running MySQL server and the Perl modules for MySQL.
Sat Dec 25 10:04:32 1999 Zoran Constantinescu
* htlib/Database.h: Class Database is inherited from X_db too. * htlib/X_db.h: Generic functions for SQL. Added to class Database to make as small changes to the code as possible. * htlib/Database.cc: Choose database type from config file! Changed getDatabaseInstance to use config["db_type"]. * htlib/SQL_db.cc,h: Added code for generic SQL databases. * htlib/MySQL_db.cc,h: Added code for MySQL databases. * htlib/HashWord.h: Experimental code for buffering the words from database in memory (for database speed-ul). Need to be completed. * htcommon/DocumentRef.h: Added new ReferenceStates and the server where the document is (docServerID) and its type (docServerType). * htcommon/DocumentDB.cc,h: Added the external 'config' variable. The Add() function is changed to call SQL_db::AddDocInfo(). Added operator(int) to get documents from database by ID. Modified code for SQL in CreateSearchDB() and URLs() (do nothing if SQL!?). * htcommon/defaults.cc: added default values for config parameters: db_type - database type: (DB2), GDBM, MySQL (! any SQL type database should contain the "SQL" letters!), SQL_debug - debug level for logging SQL messages (default 0), SQL_log_file - log file for SQL messages, SQL_dbase - connection information for SQL servs "user:pswd@host:dbase" * htcommon/WordList.cc: Flush() changed to write link informations to the SQL database (if SQL). * htfuzzy/Fuzzy.h: Added new getWords(...+wordIDs) function which returns also a list of the wordsIDs (if we are SQL, we use IDs much more). * htfuzzy/Synonym.cc,h: Added code for the new getWords() function. * htfuzzy/Endings.cc,h: Added code for the new getWords() function. * htfuzzy/EndingsDB.cc: Changed code to use SQL databases if necessary. The createRoot() function uses SQL. * htlib/Makefile.in: Added SQL_db.o MySQL_db.o to the list of objects. * htdig/htdig.cc: If SQL, no need to add the list to retriever. Added options '-x start_url' and '-z limit_urls_to'. * htdig/Document.cc,h: Added server_type (what kind of web server). * htdig/Retriever.cc,h: Added SQL support for links. Each link found in documents will be added to the database. * htsearch/htsearch.cc: Added SQL support. And add the search to the database. * htsearch/WeightWord.h: Added word_id; * htsearch/ResultMatch.h: Added id and functions setDocID(), getDocID(). * htsearch/Display.cc: Added SQL support. To be checked!