[KinoSearch] Parallel indexing runs

henka at cityweb.co.za henka at cityweb.co.za
Thu Jul 20 12:40:13 PDT 2006



> Does InvIndexer's add_invindex method address your needs?  Write to
> several separate indexes, then merge them into one main index.

my $invindexer = KinoSearch::InvIndexer->new(
        invindex => $invindex,
        analyzer => $analyzer,
    );
    $invindexer->add_invindexes( $another_invindex, $yet_another_invindex );

$invindexer->finish;
----------

OK - this might be the ticket.
Correct me if I'm wrong:

1.  run multiple indexers simultaneously, each updating "temp" indexes
somewhere.
2.  as they run/complete, have a process monitor them and
add_invindexes(a,b,c) to the main index.

A few questions, if I may:

1.  What happens to the "temp" indexes?  Must I trash them after
add_invindexes(), or does it do it?
2.  Presumably I can only run one add_invindexes() at a time (locks the
main index, right)?
3.  Related to (1), I presume the temp index data is physically transfered
to the main index?
4.  Resource limits aside, I presume there's no limitation to the number
of parallel indexers I can run (each to it's own temp index of course)?
5.  there was one more question that has slipped my mind.  dammit.




_______________________________________________
KinoSearch mailing list
KinoSearch at rectangular.com
http://www.rectangular.com/mailman/listinfo/kinosearch




More information about the kinosearch mailing list