matchzoo.dataloader.callbacks.ngram
¶
Module Contents¶
Functions¶
|
Generate the word to ngram vector mapping. |
-
class
matchzoo.dataloader.callbacks.ngram.
Ngram
(preprocessor: mz.preprocessors.BasicPreprocessor, mode: str = 'index')¶ Bases:
matchzoo.engine.base_callback.BaseCallback
Generate the character n-gram for data.
- Parameters
preprocessor – The fitted
BasePreprocessor
object, which contains the n-gram units information.mode – It can be one of ‘index’, ‘onehot’, ‘sum’ or ‘aggregate’.
Example
>>> import matchzoo as mz >>> from matchzoo.dataloader.callbacks import Ngram >>> data = mz.datasets.toy.load_data() >>> preprocessor = mz.preprocessors.BasicPreprocessor(ngram_size=3) >>> data = preprocessor.fit_transform(data) >>> callback = Ngram(preprocessor=preprocessor, mode='index') >>> dataset = mz.dataloader.Dataset( ... data, callbacks=[callback]) >>> _ = dataset[0]
-
on_batch_unpacked
(self, x, y)¶ Insert ngram_left and ngram_right to x.
-
matchzoo.dataloader.callbacks.ngram.
_build_word_ngram_map
(ngram_process_unit: mz.preprocessors.units.NgramLetter, ngram_vocab_unit: mz.preprocessors.units.Vocabulary, index_term: dict, mode: str = 'index') → dict¶ Generate the word to ngram vector mapping.
- Parameters
ngram_process_unit – The fitted
NgramLetter
object.ngram_vocab_unit – The fitted
Vocabulary
object.index_term – The index to term mapping dict.
mode – It be one of ‘index’, ‘onehot’, ‘sum’ or ‘aggregate’.
- Returns
the word to ngram vector mapping.