Xapian::PostingSource is an API class which you can subclass to feed data to Xapian's matcher. This feature can be made use of in a number of ways - for example:
As a filter - a subclass could return a stream of document ids to filter a query against.
As a weight boost - a subclass could return every document, but with a varying weight so that certain documents receive a weight boost. This could be used to prefer documents based on some external factor, such as age, price, proximity to a physical location, link analysis score, etc.
As an alternative way of ranking documents - if the weighting scheme is set to Xapian::BoolWeight, then the ranking will be entirely by the weight returned by Xapian::PostingSource.
When first constructed, a PostingSource is not tied to a particular database. Before Xapian can get any postings (or statistics) from the source, it needs to be supplied with a database. This is performed by the init() method, which is passed a single parameter holding the database to use. This method will always be called before asking for any information about the postings in the list:
virtual void init(const Xapian::Database & db) = 0;
Three methods return statistics independent of the iteration position. These are upper and lower bounds for the number of documents which can be returned, and an estimate of this number:
virtual Xapian::doccount get_termfreq_min() const = 0; virtual Xapian::doccount get_termfreq_max() const = 0; virtual Xapian::doccount get_termfreq_est() const = 0;
These methods are pure-virtual in the base class, so you have to define them when deriving your subclass.
It must always be true that:
get_termfreq_min() <= get_termfreq_est() <= get_termfreq_max()
PostingSources must always return documents in increasing document ID order.
After construction, a PostingSource points to a position before the first document id - so before a docid can be read, the position must be advanced.
Two methods return weight related information - get_weight() returns the weight for the current document, while get_maxweight() returns an upper bound on what get_weight() can return from now on. The weights must always be >= 0:
virtual Xapian::weight get_maxweight() const; virtual Xapian::weight get_weight() const;
These methods have default implementations which always return 0, for convenience when deriving "weight-less" subclasses.
The at_end() method checks if the current iteration position is past the last entry:
virtual bool at_end() const = 0;
The get_docid() method returns the document id at the current iteration position:
virtual Xapian::docid get_docid() const = 0;
There are three methods which advance the current position. All of these take a Xapian::Weight parameter min_wt, which indicates the minimum weight contribution which the matcher is interested in. The matcher still checks the weight of documents so it's OK to ignore this parameter completely, or to use it to discard only some documents. But it can be useful for optimising in some cases.
The simplest of these three methods is next(), which simply advances the iteration position to the next document (possibly skipping documents with weight contribution < min_wt):
virtual void next(Xapian::weight min_wt) = 0;
Then there's skip_to(). This advances the iteration position to the next document with document id >= that specified (possibly also skipping documents with weight contribution < min_wt):
virtual void skip_to(Xapian::docid did, Xapian::weight min_wt);
A default implementation of skip_to() is provided which just calls next() repeatedly. This works but skip_to() can often be implemented much more efficiently.
The final method of this group is check(). In some cases, it's fairly cheap to check if a given document matches, but the requirement that skip_to() must leave the iteration position on the next document is rather costly to implement (for example, it might require linear scanning of document ids). To avoid this where possible, the check() method allows the matcher to just check if a given document matches:
virtual bool check(Xapian::docid did, Xapian::weight min_wt);
The return value is true if the method leaves the iteration position valid, and false if it doesn't. In the latter case, next() will advance to the first matching position after document id did, and skip_to() will act as it would if the iteration position was the first matching position after did.
The default implementation of check() is just a thin wrapper around skip_to() which returns true - you should use this if skip_to() incurs only a small extra cost.
There's also a method to return a string describing this object:
virtual std::string get_description() const;
The default implementation returns a generic answer. This default is provided to avoid forcing you to provide an implementation if you don't really care what get_description() gives for your sub-class.
FIXME: Provide some!
In order to work with searches across multiple databases, or in remote databases, some additional methods need to be implemented on Xapian::PostingSources. The first of these is clone(), which is used for multi database searches. This method should just return a newly allocated instance of the same posting source class, initialised in the same way as the source that clone() was called on. The returned source will be deallocated by the caller (using "delete" - so you should allocate it with "new").
If you don't care about supporting searches across multiple databases, you can simply return NULL from this method. In fact, the default implementation does this, so you can just leave the default implementation in place. If clone() returns NULL, an attempt to perform a search with multiple databases will raise an exception:
virtual PostingSource * clone() const;
To work with searches across remote databases, you need to implement a few more methods. Firstly, you need to implement the name() method. This simply returns the (fully namespaced) name of your posting source:
virtual std::string name() const;
Next, you need to implement the serialise and unserialise methods. The serialise() method converts all the settings of the PostingSource to a string, and the unserialise() method converts one of these strings back into a PostingSource. Note that the serialised string doesn't need to include any information about the current iteration position of the PostingSource:
virtual std::string serialise() const; virtual PostingSource * unserialise(const std::string &s) const;
Finally, you need to make a remote server which knows about your PostingSource. Currently, the only way to do this is to hack the source slightly, and compile your own. To do this, you need to edit xapian-core/bin/xapian-tcpsrv.cc and find the register_user_weighting_schemes() function. At the end of this function, add the lines:
SerialisationContext context; context.register_postingsource(MyPostingSource()); server.set_context(context);
Where MyPostingSource is your posting source.