gnes.preprocessor.base moduleΒΆ

class gnes.preprocessor.base.BaseAudioPreprocessor(uniform_doc_weight=True, *args, **kwargs)[source]ΒΆ

Bases: gnes.preprocessor.base.BasePreprocessor

doc_type = 4ΒΆ
train(*args, **kwargs)ΒΆ

Train the model, need to be overrided

class gnes.preprocessor.base.BaseImagePreprocessor(uniform_doc_weight=True, *args, **kwargs)[source]ΒΆ

Bases: gnes.preprocessor.base.BasePreprocessor

doc_type = 2ΒΆ
train(*args, **kwargs)ΒΆ

Train the model, need to be overrided

class gnes.preprocessor.base.BasePreprocessor(uniform_doc_weight=True, *args, **kwargs)[source]ΒΆ

Bases: gnes.base.TrainableBase

apply(doc)[source]ΒΆ
Return type:None
doc_type = 0ΒΆ
train(*args, **kwargs)ΒΆ

Train the model, need to be overrided

class gnes.preprocessor.base.BaseTextPreprocessor(uniform_doc_weight=True, *args, **kwargs)[source]ΒΆ

Bases: gnes.preprocessor.base.BasePreprocessor

doc_type = 1ΒΆ
train(*args, **kwargs)ΒΆ

Train the model, need to be overrided

class gnes.preprocessor.base.BaseVideoPreprocessor(uniform_doc_weight=True, *args, **kwargs)[source]ΒΆ

Bases: gnes.preprocessor.base.BasePreprocessor

doc_type = 3ΒΆ
train(*args, **kwargs)ΒΆ

Train the model, need to be overrided

class gnes.preprocessor.base.PipelinePreprocessor(*args, **kwargs)[source]ΒΆ

Bases: gnes.base.CompositionalTrainableBase

apply(doc)[source]ΒΆ
Return type:None
train(data, *args, **kwargs)[source]ΒΆ

Train the model, need to be overrided

class gnes.preprocessor.base.RawChunkPreprocessor(uniform_doc_weight=True, *args, **kwargs)[source]ΒΆ

Bases: gnes.preprocessor.base.BasePreprocessor

apply(doc)[source]ΒΆ
Return type:None
train(*args, **kwargs)ΒΆ

Train the model, need to be overrided

class gnes.preprocessor.base.UnaryPreprocessor(doc_type, *args, **kwargs)[source]ΒΆ

Bases: gnes.preprocessor.base.BasePreprocessor

apply(doc)[source]ΒΆ
is_trained = TrueΒΆ
raw_to_chunk(chunk, raw_bytes)[source]ΒΆ
train(*args, **kwargs)ΒΆ

Train the model, need to be overrided