[KinoSearch] serializing safely

Hans Dieter Pearcey hdp at pobox.com
Thu Jun 14 20:03:04 PDT 2007


On Thu, Jun 14, 2007 at 07:32:36PM -0700, Marvin Humphrey wrote:
> I don't think you should rule out pre-loading.  KinoSearch is heavily  
> optimized for the use case of running many queries against a single  
> view of an index.
> 
> The costs for warming the a Searcher vary linearly with index size,  
> and get significantly higher if you perform sorting or range  
> operations.  To put things in perspective, for very large indexes  
> (larger than you're likely to see for any one individual's email), it  
> can conceivably take several seconds to warm up a Searcher, then a  
> fraction of a second to process the query.

That's good to know, since this is for mailing lists (and thus much larger than
an average user's email) and I plan to both sort and use RangeFilter(s).

It may still be worthwhile to preload, depending on ram usage for searchers and
actual startup times.  I'll have to test with real data to find out, though.

hdp.



More information about the KinoSearch mailing list