matchzoo.preprocessors.bert_preprocessor

Bert Preprocessor.

Module Contents

Classes

BertPreprocessor

Baisc preprocessor helper.

class matchzoo.preprocessors.bert_preprocessor.BertPreprocessor(mode: str = 'bert-base-uncased')

Bases: matchzoo.engine.base_preprocessor.BasePreprocessor

Baisc preprocessor helper.

Parameters

mode – String, supported mode can be referred https://huggingface.co/pytorch-transformers/pretrained_models.html.

fit(self, data_pack: DataPack, verbose: int = 1)

Tokenizer is all BertPreprocessor’s need.

transform(self, data_pack: DataPack, verbose: int = 1) → DataPack

Apply transformation on data.

Parameters

• data_pack – Inputs to be preprocessed.

• verbose – Verbosity.

Returns

Transformed data as DataPack object.