gnes.indexer.doc.filesys module

class gnes.indexer.doc.filesys.DirectoryIndexer(data_path, keep_na_doc=True, file_suffix='gif', *args, **kwargs)[source]

Bases: gnes.indexer.base.BaseDocIndexer

add(keys, docs, *args, **kwargs)[source]

write GIFs of each document into disk folder structure: /data_path/doc_id/0.gif, 1.gif…

Parameters:
  • keys (List[int]) – list of doc id
  • docs (List[Document]) – list of docs
query(keys, *args, **kwargs)[source]

Find the doc according to the keys

Parameters:keys (List[int]) – list of doc id
Return type:List[Document]
Returns:list of documents whose chunks field contain all the GIFs of this doc(one GIF per chunk)
train(*args, **kwargs)

Train the model, need to be overrided