[KinoSearch] Caching QueryFilter objects

[Marvin Humphrey]

On Oct 11, 2006, at 3:05 PM, Chris Nandor wrote:

> I assume I need to cache a QueryFilter object in order to benefit  
> from its
> caching?  Or does it magically fetch a cache when I recreate the  
> object
> with the same parameters?  I am assuming not.

In the current implementation, the filter is stored as a BitVector  
object inside the QueryFilter object.  The BitVector is lazily  
generated, waiting until the search is executed to appear rather than  
during QueryFilter object construction.

When you execute a second search using the same QueryFilter, the  
existing BitVector gets used.

> And can I add multiple filters somehow?  It looks like I can just  
> add a
> bunch of terms to a BooleanQuery and then pass that to  
> QueryFilter.  And
> then to cache that, I need to smartly cache that object by the several
> criteria that went into creating it.

That will work fine, though with a large index and several filters,  
RAM usage might start to become a concern.  Each filter consumes a  
byte for every eight documents in the index (plus a small amount for  
object overhead).

> And of course, these filters will expire after some time anyway.  I
> somewhat idly wonder how worth it it is to bother.

It's the difference between executing the query (which generally  
means disk i/o) and reading from a BitVector.  Doesn't take too many  
searches to make that worthwhile!

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/