[KinoSearch] Couldn't get lock

[Marvin Humphrey]

On Jun 18, 2007, at 7:12 AM, Jesse Erlbaum wrote:

> We've been rolling out a Kinosearch-based site search engine, and  
> periodically get the following error:
>
>     Couldn't get lock using '/tmp/kinosearch_lockdir/ 
> kinosearch-677654d74548827b32ec1abc555b92a7-commit.lock' at /path/ 
> to/lib/i686-linux/KinoSearch/Store/Lock.pm line 28
>
> When I look in /tmp I see nothing.  The directory  
> "kinosearch_lockdir" does not exist.

Presumably it exists sometimes -- otherwise, you'd never get a  
Searcher.  KS never removes that directory itself.

> What's going on here?  Is there a way to set the lock directory?

There's a hack.

   use KinoSearch::Store::FSInvIndex;
   $KinoSearch::Store::FSInvIndex::LOCK_DIR = '/path/to/wherever';
   mkdir('path/to/wherever') or die $!;

Do that before you create the Searcher.

This is for 0.1x.  In the 0.20 branch, the locking mechanism has  
changed.  The lock dir is now the index dir itself, and Searchers  
don't use locking by default.  However, since your index is on an NFS  
volume, you're going to need it.

> Is there a way to tell Kino to create the directory if it is missing?

That's happening at Class load time.  Snoop the BEGIN block in  
KinoSearch::Store::FSInvIndex.

> This error causes the Search engine to periodically crash with an  
> Internal Server Error.
>
> FYI, I don't know if it is important, but we build the index on one  
> machine and the front-end search runs on another.  (The index  
> directory is NFS mounted, and UNIX user ids are in sync.  The "/ 
> tmp" directory is *not* shared between machines.)

NFS is going to require special care, but for other reasons.  I  
recommend that you upgrade to the 0.20 branch, and that you supply a  
LockFactory to both InvIndexer and IndexReader.  Please check out the  
0.20 branch docs for those two classes here:

   http://www.rectangular.com/kinosearch/docs/devel/

Note that 0.20 and 0.15 are not compatible either in terms of API or  
file format.

Please write back to let us know how things go.

HTH,

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/