gnes.indexer.chunk.faiss module

class gnes.indexer.chunk.faiss.FaissIndexer(num_dim, index_key, data_path, *args, **kwargs)[source]

Bases: gnes.indexer.base.BaseChunkIndexer

Initialize an FaissIndexer

Parameters:
  • num_dim (int) – when set to -1, then num_dim is auto decided on first .add()
  • data_path (str) – index data file managed by the faiss indexer
add(keys, vectors, weights, *args, **kwargs)[source]

adding new chunks and their vector representations

Parameters:
  • keys (List[Tuple[int, Any]]) – list of (doc_id, offset) tuple
  • vectors (ndarray) – vector representations
  • weights (List[float]) – weight of the chunks
post_init()[source]

Declare class attributes/members that can not be serialized in standard way

query(keys, top_k, *args, **kwargs)[source]
Return type:List[List[Tuple]]
train(*args, **kwargs)

Train the model, need to be overrided