gnes.indexer.chunk.helper module¶

class gnes.indexer.chunk.helper.DictKeyIndexer(*args, **kwargs)[source]¶

Bases: gnes.indexer.base.BaseChunkIndexerHelper

add(keys, weights, *args, **kwargs)[source]¶

adding new chunks and their vector representations

Parameters:
  • keys (List[Tuple[int, int]]) – list of (doc_id, offset) tuple
  • vectors – vector representations
  • weights (List[float]) – weight of the chunks
Return type:

int
query(keys, *args, **kwargs)[source]¶
Return type:List[Tuple[int, int, float]]
train(*args, **kwargs)¶

Train the model, need to be overrided

class gnes.indexer.chunk.helper.ListKeyIndexer(*args, **kwargs)[source]¶

Bases: gnes.indexer.base.BaseChunkIndexerHelper

add(keys, weights, *args, **kwargs)[source]¶

adding new chunks and their vector representations

Parameters:
  • keys (List[Tuple[int, int]]) – list of (doc_id, offset) tuple
  • vectors – vector representations
  • weights (List[float]) – weight of the chunks
Return type:

int
query(keys, *args, **kwargs)[source]¶
Return type:List[Tuple[int, int, float]]
train(*args, **kwargs)¶

Train the model, need to be overrided

class gnes.indexer.chunk.helper.ListNumpyKeyIndexer(*args, **kwargs)[source]¶

Bases: gnes.indexer.chunk.helper.ListKeyIndexer

add(*args, **kwargs)[source]¶

adding new chunks and their vector representations

Parameters:
  • keys – list of (doc_id, offset) tuple
  • vectors – vector representations
  • weights – weight of the chunks
Return type:

int
query(keys, *args, **kwargs)[source]¶
Return type:List[Tuple[int, int, float]]
train(*args, **kwargs)¶

Train the model, need to be overrided

class gnes.indexer.chunk.helper.NumpyKeyIndexer(buffer_size=10000, col_size=3, *args, **kwargs)[source]¶

Bases: gnes.indexer.base.BaseChunkIndexerHelper

add(keys, weights, *args, **kwargs)[source]¶

adding new chunks and their vector representations

Parameters:
  • keys (List[Tuple[int, int]]) – list of (doc_id, offset) tuple
  • vectors – vector representations
  • weights (List[float]) – weight of the chunks
Return type:

int
capacity¶
query(keys, *args, **kwargs)[source]¶
Return type:List[Tuple[int, int, float]]
train(*args, **kwargs)¶

Train the model, need to be overrided