matchzoo.auto.tuner

Package Contents

Classes

Tuner

Model hyper-parameters tuner.

Functions

tune

class matchzoo.auto.tuner.Tuner(params: mz.ParamTable, optimizer: str = 'adam', trainloader: mz.dataloader.DataLoader = None, validloader: mz.dataloader.DataLoader = None, embedding: np.ndarray = None, fit_kwargs: dict = None, metric: typing.Union[str, BaseMetric] = None, mode: str = 'maximize', num_runs: int = 10, verbose=1)

Bases: object

Model hyper-parameters tuner.

model.params.hyper_space reprensents the model’s hyper-parameters search space, which is the cross-product of individual hyper parameter’s hyper space. When a Tuner builds a model, for each hyper parameter in model.params, if the hyper-parameter has a hyper-space, then a sample will be taken in the space. However, if the hyper-parameter does not have a hyper-space, then the default value of the hyper-parameter will be used.

See tutorials/model_tuning.ipynb for a detailed walkthrough on usage.

Parameters

  • params – A completed parameter table to tune. Usually model.params of the desired model to tune. params.completed() should be True.

  • optimizer – Str or Optimizer class. Optimizer for optimizing model.

  • trainloader – Training data to use. Should be a DataLoader.

  • validloader – Testing data to use. Should be a DataLoader.

  • embedding – Embedding used by model.

  • fit_kwargs – Extra keyword arguments to pass to fit. (default: dict(epochs=10, verbose=0))

  • metric – Metric to tune upon. Must be one of the metrics in model.params[‘task’].metrics. (default: the first metric in params.[‘task’].metrics.

  • mode – Either maximize the metric or minimize the metric. (default: ‘maximize’)

  • num_runs – Number of runs. Each run takes a sample in params.hyper_space and build a model based on the sample. (default: 10)

  • verbose – Verbosity. (default: 1)

tune(self)

Start tuning.

Notice that tune does not affect the tuner’s inner state, so each new call to tune starts fresh. In other words, hyperspaces are suggestive only within the same tune call.

_fmin(self, trials)
_run(self, sample)
_create_full_params(self, sample)
_fix_loss_sign(self, loss)
classmethod _log_result(cls, result)
property params(self)

params getter.

property trainloader(self)

trainloader getter.

property validloader(self)

validloader getter.

property fit_kwargs(self)

fit_kwargs getter.

property metric(self)

metric getter.

property mode(self)

mode getter.

property num_runs(self)

num_runs getter.

property verbose(self)

verbose getter.

classmethod _validate_params(cls, params)
classmethod _validate_optimizer(cls, optimizer)
classmethod _validate_dataloader(cls, data)
classmethod _validate_kwargs(cls, kwargs)
classmethod _validate_mode(cls, mode)
classmethod _validate_metric(cls, params, metric)
classmethod _validate_num_runs(cls, num_runs)
matchzoo.auto.tuner.tune(params: mz.ParamTable, optimizer: str = 'adam', trainloader: mz.dataloader.DataLoader = None, validloader: mz.dataloader.DataLoader = None, embedding: np.ndarray = None, fit_kwargs: dict = None, metric: typing.Union[str, BaseMetric] = None, mode: str = 'maximize', num_runs: int = 10, verbose=1)

Tune model hyper-parameters.

A simple shorthand for using matchzoo.auto.Tuner.

model.params.hyper_space reprensents the model’s hyper-parameters search space, which is the cross-product of individual hyper parameter’s hyper space. When a Tuner builds a model, for each hyper parameter in model.params, if the hyper-parameter has a hyper-space, then a sample will be taken in the space. However, if the hyper-parameter does not have a hyper-space, then the default value of the hyper-parameter will be used.

See tutorials/model_tuning.ipynb for a detailed walkthrough on usage.

Parameters

  • params – A completed parameter table to tune. Usually model.params of the desired model to tune. params.completed() should be True.

  • optimizer – Str or Optimizer class. Optimizer for optimizing model.

  • trainloader – Training data to use. Should be a DataLoader.

  • validloader – Testing data to use. Should be a DataLoader.

  • embedding – Embedding used by model.

  • fit_kwargs – Extra keyword arguments to pass to fit. (default: dict(epochs=10, verbose=0))

  • metric – Metric to tune upon. Must be one of the metrics in model.params[‘task’].metrics. (default: the first metric in params.[‘task’].metrics.

  • mode – Either maximize the metric or minimize the metric. (default: ‘maximize’)

  • num_runs – Number of runs. Each run takes a sample in params.hyper_space and build a model based on the sample. (default: 10)

  • callbacks – A list of callbacks to handle. Handled sequentially at every callback point.

  • verbose – Verbosity. (default: 1)

Example

>>> import matchzoo as mz
>>> import numpy as np
>>> train = mz.datasets.toy.load_data('train')
>>> valid = mz.datasets.toy.load_data('dev')
>>> prpr = mz.models.DenseBaseline.get_default_preprocessor()
>>> train = prpr.fit_transform(train, verbose=0)
>>> valid = prpr.transform(valid, verbose=0)
>>> trainset = mz.dataloader.Dataset(train)
>>> validset = mz.dataloader.Dataset(valid)
>>> padding = mz.models.DenseBaseline.get_default_padding_callback()
>>> trainloader = mz.dataloader.DataLoader(trainset, callback=padding)
>>> validloader = mz.dataloader.DataLoader(validset, callback=padding)
>>> model = mz.models.DenseBaseline()
>>> model.params['task'] = mz.tasks.Ranking()
>>> optimizer = 'adam'
>>> embedding = np.random.uniform(-0.2, 0.2,
...     (prpr.context['vocab_size'], 100))
>>> tuner = mz.auto.Tuner(
...     params=model.params,
...     optimizer=optimizer,
...     trainloader=trainloader,
...     validloader=validloader,
...     embedding=embedding,
...     num_runs=1,
...     verbose=0
... )
>>> results = tuner.tune()
>>> sorted(results['best'].keys())
['#', 'params', 'sample', 'score']