Framework for intersections/arrangements with a field

[SamHames]

Common operations like time series trends are going to require an operator that takes a query and intersects it against an entire field, with varying operations and measurements. This is particularly going to be important for time where simple counts and similarity will go a long way.

The best long term API/functionality is very uncertain, but right now something simple like an iterator over the values in a field, combined with indexing-time binning might be enough to do some useful things with prechosen granularity for aggregations. This would allow something like one of the following to construct an hourly count of something:

from hyperreal import index
i = index.Index('test_index.db')

# reuse __getitem__ as an iterator when provided with only a string as the field name
[(value, query.intersection_cardinality(docs)) for value, docs in i['text']]
# Add an additional method --> this is probably clearer that this is an iterator of outputs
[(value, query.intersection_cardinality(docs)) for value, docs in i.iterate_field_values('text')]
# Use the slice notation, with the use of the None value as a sentinel for "from the beginning/to the end"
[(field, value, query.intersection_cardinality(docs)) for (field, value), docs in i[(field, None):(field, None)]

This will probably also feed into future work on fancier drilldowns, and approaches to how we integrate/display non-model features within the context of the model clusters.