Why is a SQL database required?
conceptSearching stores its probabilistic index in a proprietary database. However, the conceptIndexer uses a SQL database to manage the queue of documents to be indexed. The SQL database contains all information necessary to perform indexing, such as the individual filenames and URLs, access criteria, re-indexing frequency, inclusions and exclusions, etc. The SQL database may also be used to store any application specific meta-data.