[KinoSearch] Field-specific terms vs. query filter?

[Marvin Humphrey]

Howdy, Larry...

On Nov 30, 2007, at 6:47 PM, Larry Leszczynski wrote:

> I'm working with an index that has a "category_id" field, and I  
> need to filter search results based on specific categories which  
> the user shouldn't be able to see.  Right now I'm taking the user's  
> original query and appending a lot of booleans with field-specific  
> terms, e.g. a search for "foo" turns into something like "(foo) AND  
> NOT (category_id:1) AND NOT (category_id:7) AND NOT ...".
>
> Is there any advantage to building up a Boolean query for just the  
> category_id parts, and using that as a query filter instead?  I  
> won't necessarily be able to cache/reuse the query filter.

There shouldn't be much difference if any between the filtering  
approach and the complex-query approach.  You have to build up the  
same data either way; the QueryFilter just executes the "excluded"  
parts of the search first and stuffs the result into a BitVector,  
then applies that to the "required" part.

If you were able to keep the QueryFilter objects around, then on  
subsequent searches, you wouldn't have to re-run the excluded parts  
of the search, but oh well.  If the box you're running this on has  
sufficient memory, the kernel cache will probably help you out behind  
the scenes.

> Is there a better way in general to do this?

Though there are some approaches that we could take for better  
supporting this kind of search, they all involve caching something.   
For large datasets, KS is really designed to be used with a  
persistent search object.  For small datasets, it doesn't matter.   
Where the threshold between "large" and "small" lies depends on a lot  
of variables.

> Related question:  Assuming the category_id field is indexed but  
> not analyzed, is a field-specific term going to do an exact match?   
> I.e. will "category_id:1" match just "1", or will it also match "10"?

It will do an exact match.

Cheers,

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/



_______________________________________________
KinoSearch mailing list
KinoSearch at rectangular.com
http://www.rectangular.com/mailman/listinfo/kinosearch