[KinoSearch] API for subclassing Scorer (was adding a proximity scorer)

[Marvin Humphrey]

On Jun 16, 2007, at 12:05 AM, Nathan Kurz wrote:

>> Ideally, our discussion will result in an improvement upon that
>> scheme that will allow you to write your ORScorer subclass without
>> touching BoilerPlater.  Something like this:
>>
>>    package MyORScorer;
>>    use base qw( KinoSearch::Search::ORScorer );
>>
>>    __PACKAGE__->register_c_method( tally => 'my_tally' );
>>
>>    use Inline => C << 'END_C';
>>
>>    kino_Tally*
>>    my_tally(kino_OrScorer *self) {
>>        /* ... */
>>    }
>>
>>    END_C
> That seems like a great goal.  For now I'm happy writing C.

OK, check.  But I want to make it easier for you to maintain your  
Scorer subclass, and I want to make it easier for other people to  
write them.

> Perhaps
> more useful for most people would be the ability to override a
> BoilerPlated C method with a Perl function, with it automatically
> wrapped in just enough C to push the args.

Yes, I absolutely agree we should do that.

In this particular case, adding Perl's function call overhead to  
Scorer_Tally() would be a disaster for search-time performance,  
because it's inner loop code.  But that's not true everywhere, and  
for rapid prototyping taking the performance hit would be acceptable.

> You aren't already doing this anywhere, are you?

No, but the prime candidate would be Similarity->length_norm.

> Personally, though, I'd probably rather see a greater split between
> the Perl and the C. I love them both individually, but I'd be more
> comfortable with a standard C library (libidf?) with a Perl wrapper
> and a clearly defined boundary.

This is clearly the direction that KS is headed.

   ...  ...

Let's design the ideal API for subclassing Scorer, then work  
backwards to implement it and see how close we can get.

   * It should be possible to implement a Scorer class entirely in
     Perl and have KS use it.  (Schema and FieldSpec sort of work
     this way.)
   * It should be possible to override individual methods used by
     a Scorer implemented in C with wrapped Perl subroutines.
   * It should be possible to override individual methods used by
     a Scorer implemented in C with C functions, as in the code
     block at the top of this post.  (This is fairly easy.)
   * It should be possible to add additional Perl member variables
     to a Scorer implemented in C.
   * It should be possible to add additional C member variables to
     a Scorer implemented in C.
   * It _must_ be possible to upgrade KS without encountering binary
     compatibility problems such as reordered vtables or object
     structs.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/