gnes.indexer.doc.filesys module

class gnes.indexer.doc.filesys.DirectoryIndexer(data_path, keep_na_doc=True, file_suffix='gif', *args, **kwargs)

Bases: gnes.indexer.base.BaseDocIndexer

add(keys, docs, *args, **kwargs)

write GIFs of each document into disk folder structure: /data_path/doc_id/0.gif, 1.gif…

Parameters:

• keys (List[int]) – list of doc id
• docs (List[Document]) – list of docs

query(keys, *args, **kwargs)

Find the doc according to the keys

Parameters:keys (List[int]) – list of doc id Return type:List[Document] Returns:list of documents whose chunks field contain all the GIFs of this doc(one GIF per chunk)

train(*args, **kwargs)

Train the model, need to be overrided