matchzoo.modules

Package Contents

class matchzoo.modules.Attention(input_size: int = 100)

Bases: torch.nn.Module

Attention module.

Parameters:
  • input_size – Size of input.
  • mask – An integer to mask the invalid values. Defaults to 0.

Examples

>>> import torch
>>> attention = Attention(input_size=10)
>>> x = torch.randn(4, 5, 10)
>>> x.shape
torch.Size([4, 5, 10])
>>> x_mask = torch.BoolTensor(4, 5)
>>> attention(x, x_mask).shape
torch.Size([4, 5])
forward(self, x, x_mask)

Perform attention on the input.

class matchzoo.modules.BidirectionalAttention

Bases: torch.nn.Module

Computing the soft attention between two sequence.

forward(self, v1, v1_mask, v2, v2_mask)

Forward.

class matchzoo.modules.MatchModule(hidden_size, dropout_rate=0)

Bases: torch.nn.Module

Computing the match representation for Match LSTM.

Parameters:
  • hidden_size – Size of hidden vectors.
  • dropout_rate – Dropout rate of the projection layer. Defaults to 0.

Examples

>>> import torch
>>> attention = MatchModule(hidden_size=10)
>>> v1 = torch.randn(4, 5, 10)
>>> v1.shape
torch.Size([4, 5, 10])
>>> v2 = torch.randn(4, 5, 10)
>>> v2_mask = torch.ones(4, 5).to(dtype=torch.uint8)
>>> attention(v1, v2, v2_mask).shape
torch.Size([4, 5, 20])
forward(self, v1, v2, v2_mask)

Computing attention vectors and projection vectors.

class matchzoo.modules.RNNDropout

Bases: torch.nn.Dropout

Dropout for RNN.

forward(self, sequences_batch)

Masking whole hidden vector for tokens.

class matchzoo.modules.StackedBRNN(input_size, hidden_size, num_layers, dropout_rate=0, dropout_output=False, rnn_type=nn.LSTM, concat_layers=False)

Bases: torch.nn.Module

Stacked Bi-directional RNNs.

Differs from standard PyTorch library in that it has the option to save and concat the hidden states between layers. (i.e. the output hidden size for each sequence input is num_layers * hidden_size).

Examples

>>> import torch
>>> rnn = StackedBRNN(
...     input_size=10,
...     hidden_size=10,
...     num_layers=2,
...     dropout_rate=0.2,
...     dropout_output=True,
...     concat_layers=False
... )
>>> x = torch.randn(2, 5, 10)
>>> x.size()
torch.Size([2, 5, 10])
>>> x_mask = (torch.ones(2, 5) == 1)
>>> rnn(x, x_mask).shape
torch.Size([2, 5, 20])
forward(self, x, x_mask)

Encode either padded or non-padded sequences.

_forward_unpadded(self, x, x_mask)

Faster encoding that ignores any padding.

class matchzoo.modules.GaussianKernel(mu: float = 1.0, sigma: float = 1.0)

Bases: torch.nn.Module

Gaussian kernel module.

Parameters:
  • mu – Float, mean of the kernel.
  • sigma – Float, sigma of the kernel.

Examples

>>> import torch
>>> kernel = GaussianKernel()
>>> x = torch.randn(4, 5, 10)
>>> x.shape
torch.Size([4, 5, 10])
>>> kernel(x).shape
torch.Size([4, 5, 10])
forward(self, x)

Forward.

class matchzoo.modules.Matching(normalize: bool = False, matching_type: str = 'dot')

Bases: torch.nn.Module

Module that computes a matching matrix between samples in two tensors.

Parameters:
  • normalize – Whether to L2-normalize samples along the dot product axis before taking the dot product. If set to True, then the output of the dot product is the cosine proximity between the two samples.
  • matching_type – the similarity function for matching

Examples

>>> import torch
>>> matching = Matching(matching_type='dot', normalize=True)
>>> x = torch.randn(2, 3, 2)
>>> y = torch.randn(2, 4, 2)
>>> matching(x, y).shape
torch.Size([2, 3, 4])
classmethod _validate_matching_type(cls, matching_type: str = 'dot')
forward(self, x, y)

Perform attention on the input.

class matchzoo.modules.BertModule(mode: str = 'bert-base-uncased')

Bases: torch.nn.Module

Bert module.

BERT (from Google) released with the paper BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding by Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova.

Parameters:mode – String, supported mode can be referred https://huggingface.co/pytorch-transformers/pretrained_models.html.
forward(self, x, y)

Forward.

class matchzoo.modules.CharacterEmbedding(char_embedding_input_dim: int = 100, char_embedding_output_dim: int = 8, char_conv_filters: int = 100, char_conv_kernel_size: int = 5)

Bases: torch.nn.Module

Character embedding module.

Parameters:
  • char_embedding_input_dim – The input dimension of character embedding layer.
  • char_embedding_output_dim – The output dimension of character embedding layer.
  • char_conv_filters – The filter size of character convolution layer.
  • char_conv_kernel_size – The kernel size of character convolution layer.

Examples

>>> import torch
>>> character_embedding = CharacterEmbedding()
>>> x = torch.ones(10, 32, 16, dtype=torch.long)
>>> x.shape
torch.Size([10, 32, 16])
>>> character_embedding(x).shape
torch.Size([10, 32, 100])
forward(self, x)

Forward.

class matchzoo.modules.SemanticComposite(in_features, dropout_rate: float = 0.0)

Bases: torch.nn.Module

SemanticComposite module.

Apply a self-attention layer and a semantic composite fuse gate to compute the encoding result of one tensor.

Parameters:
  • in_features – Feature size of input.
  • dropout_rate – The dropout rate.

Examples

>>> import torch
>>> module = SemanticComposite(in_features=10)
>>> x = torch.randn(4, 5, 10)
>>> x.shape
torch.Size([4, 5, 10])
>>> module(x).shape
torch.Size([4, 5, 10])
forward(self, x)

Forward.

class matchzoo.modules.DenseNet(in_channels, nb_dense_blocks: int = 3, layers_per_dense_block: int = 3, growth_rate: int = 10, transition_scale_down_ratio: float = 0.5, conv_kernel_size: tuple = (2, 2), pool_kernel_size: tuple = (2, 2))

Bases: torch.nn.Module

DenseNet module.

Parameters:
  • in_channels – Feature size of input.
  • nb_dense_blocks – The number of blocks in densenet.
  • layers_per_dense_block – The number of convolution layers in dense block.
  • growth_rate – The filter size of each convolution layer in dense block.
  • transition_scale_down_ratio – The channel scale down ratio of the convolution layer in transition block.
  • conv_kernel_size – The kernel size of convolution layer in dense block.
  • pool_kernel_size – The kernel size of pooling layer in transition block.
out_channels :int

out_channels getter.

forward(self, x)

Forward.

classmethod _make_transition_block(cls, in_channels: int, transition_scale_down_ratio: float, pool_kernel_size: tuple)
class matchzoo.modules.MatchingTensor(matching_dim: int, channels: int = 4, normalize: bool = True, init_diag: bool = True)

Bases: torch.nn.Module

Module that captures the basic interactions between two tensors.

Parameters:
  • matching_dims – Word dimension of two interaction texts.
  • channels – Number of word interaction tensor channels.
  • normalize – Whether to L2-normalize samples along the dot product axis before taking the dot product. If set to True, then the output of the dot product is the cosine proximity between the two samples.
  • init_diag – Whether to initialize the diagonal elements of the matrix.

Examples

>>> import matchzoo as mz
>>> matching_dim = 5
>>> matching_tensor = mz.modules.MatchingTensor(
...    matching_dim,
...    channels=4,
...    normalize=True,
...    init_diag=True
... )
forward(self, x, y)

The computation logic of MatchingTensor.

Parameters:inputs – two input tensors.
class matchzoo.modules.SpatialGRU(channels: int = 4, units: int = 10, activation: typing.Union[str, typing.Type[nn.Module], nn.Module] = 'tanh', recurrent_activation: typing.Union[str, typing.Type[nn.Module], nn.Module] = 'sigmoid', direction: str = 'lt')

Bases: torch.nn.Module

Spatial GRU Module.

Parameters:
  • channels – Number of word interaction tensor channels.
  • units – Number of SpatialGRU units.
  • activation – Activation function to use, one of: - String: name of an activation - Torch Modele subclass - Torch Module instance Default: hyperbolic tangent (tanh).
  • recurrent_activation –

    Activation function to use for the recurrent step, one of:

    • String: name of an activation
    • Torch Modele subclass
    • Torch Module instance

    Default: sigmoid activation (sigmoid).

  • direction – Scanning direction. lt (i.e., left top) indicates the scanning from left top to right bottom, and rb (i.e., right bottom) indicates the scanning from right bottom to left top.

Examples

>>> import matchzoo as mz
>>> channels, units= 4, 10
>>> spatial_gru = mz.modules.SpatialGRU(channels, units)
reset_parameters(self)

Initialize parameters.

softmax_by_row(self, z: torch.tensor)

Conduct softmax on each dimension across the four gates.

calculate_recurrent_unit(self, inputs: torch.tensor, states: list, i: int, j: int)

Calculate recurrent unit.

Parameters:
  • inputs – A tensor which contains interaction between left text and right text.
  • states – An array of tensors which stores the hidden state of every step.
  • i – Recurrent row index.
  • j – Recurrent column index.
forward(self, inputs)

Perform SpatialGRU on word interation matrix.

Parameters:inputs – input tensors.