New things

I added the X_db, SQL_db and MySQL_db classes, and the hierarchy of classes becomes:

    `--- Database
            `------ DB2_db
            `------ SQL_db
                      `------ MySQL_db

The only MySQL specific code is in the MySQL class, which uses the API of mysql (library libmysqlclient.a). The SQL code doesn't serialize the information as the DB2 code does, instead it uses specialized tables for all information (URLS, URLINFO, WORDS, LINKS and so). The Perl scripts for creating the SQL tables in the database are found in htdig3/contrib/sqldb. So I had to change all Database::Put/Get() calls everywhere to specialized calls. That's what I've put in the X_db class. To keep the DB2 functionality unchangeD, I added a config variable 'db_type' which I used where the Put/Get methods where called (if we use SQL then the new functions will be called).

There is a patch for 3.1.5 available with the changes to the 3.1.5 version of ht://Dig. It contains a README on how to apply the patch and use htdig with MySQL. You will need a running MySQL server and the Perl modules for MySQL.


Sat Dec 25 10:04:32 1999  Zoran Constantinescu 
  * htlib/Database.h: Class Database is inherited from X_db too.

  * htlib/X_db.h: Generic functions for SQL. Added to class Database
  to make as small changes to the code as possible.

  * htlib/ Choose database type from config file! Changed 
  getDatabaseInstance to use config["db_type"].

  * htlib/,h: Added code for generic SQL databases.

  * htlib/,h: Added code for MySQL databases.

  * htlib/HashWord.h: Experimental code for buffering the words from
  database in memory (for database speed-ul). Need to be completed.

  * htcommon/DocumentRef.h: Added new ReferenceStates and the server
  where the document is (docServerID) and its type (docServerType).

  * htcommon/,h: Added the external 'config' variable. 
        The Add() function is changed to call SQL_db::AddDocInfo(). Added 
  operator[](int) to get documents from database by ID. Modified 
  code for SQL in CreateSearchDB() and URLs() (do nothing if SQL!?). 

  * htcommon/ added default values for config parameters: 
  db_type - database type: (DB2), GDBM, MySQL (! any SQL type database
  should contain the "SQL" letters!),
  SQL_debug - debug level for logging SQL messages (default 0),
  SQL_log_file - log file for SQL messages,
  SQL_dbase - connection information for SQL servs "user:pswd@host:dbase"

  * htcommon/ Flush() changed to write link informations 
  to the SQL database (if SQL).

  * htfuzzy/Fuzzy.h: Added new getWords(...+wordIDs) function which 
  returns also a list of the wordsIDs (if we are SQL, we use IDs 
  much more).

  * htfuzzy/,h: Added code for the new getWords() function.

  * htfuzzy/,h: Added code for the new getWords() function.
  * htfuzzy/ Changed code to use SQL databases if necessary.
  The createRoot() function uses SQL.

  * htlib/ Added SQL_db.o MySQL_db.o to the list of

  * htdig/ If SQL, no need to add the list to retriever. 
  Added options '-x start_url' and '-z limit_urls_to'.

  * htdig/,h: Added server_type (what kind of web server).

  * htdig/,h: Added SQL support for links. Each link
  found in documents will be added to the database.

  * htsearch/ Added SQL support. And add the search to 
  the database.

  * htsearch/WeightWord.h: Added word_id;

  * htsearch/ResultMatch.h: Added id and functions setDocID(), getDocID().

  * htsearch/ Added SQL support. To be checked!