[KinoSearch] Problem merging index

[Marvin Humphrey]

On Fri, Aug 14, 2009 at 06:02:13PM -0700, Dan wrote:

> I was hitting this bug also using 0.30_04... one of the fixes in 0.30_05
> resolved this in my case.

> On Fri, Aug 14, 2009 at 2:08 AM, Mark Holland <mark at thinkfoo.com> wrote:
> > I'm building several indexes in parallel for a dataset of 30m records, then
> > merging into one. Individually, each index builds fine, however when I try
> > to merge I always get the same error on the same batch:
> >
> > Merge (12000000-13000000) to main index...
> > Out of bounds: -2147050923 >= 999408 at core/KinoSearch/Util/I32Array.c:40
> > kino_I32Arr_get
> >          at databuild/search/merge_kino_batches line 33

This is almost certainly the Lexicon bug fixed in 0.30_05.  If that's the
case, you will need to rebuild your indexes to resolve the problem.  The "out
of bounds" error is the result of an internal sanity check failing; the
message is a bit cryptic because what happened wasn't an expected error
condition.

> > Am I hitting some sort of limit here or is there something wrong with my
> > indexed data?

I doubt you are hitting any of the logical limits (2**31 terms in a field,
2**31 documents in a collection, 2**31 revisions to an index, etc).

Marvin Humphrey