gnes.encoder.video.yt8m_feature_extractor module

class gnes.encoder.video.yt8m_feature_extractor.YouTube8MFeatureExtractor(model_dir, pca_dir, select_layer='PreLogits', ignore_audio_feature=True, *args, **kwargs)

Bases: gnes.encoder.base.BaseVideoEncoder

Extracts YouTube8M features for RGB frames.

First time constructing this class will create directory yt8m inside your home directory, and will download inception model (85 MB) and YouTube8M PCA matrix (15 MB). If you want to use another directory, then pass it to argument model_dir of constructor.

If the model_dir exist and contains the necessary files, then files will be re-used without download.

Usage Example:

from PIL import Image import numpy

# Instantiate extractor. Slow if called first time on your machine, as it # needs to download 100 MB. extractor = YouTube8MFeatureExtractor()

image_file = os.path.join(extractor._model_dir, ‘cropped_panda.jpg’)

im = numpy.array(Image.open(image_file)) features = extractor.extract_rgb_frame_features(im)

** Note: OpenCV reverses the order of channels (i.e. orders channels as BGR instead of RGB). If you are using OpenCV, then you must do:

im = im[:, :, ::-1] # Reverses order on last (i.e. channel) dimension.

then call extractor.extract_rgb_frame_features(im)

batch_size = 64

encode(data, *args, **kwargs)

Return type:List[ndarray]

post_init()

Declare class attributes/members that can not be serialized in standard way

train(*args, **kwargs)

Train the model, need to be overrided