matchzoo.modules
¶
Submodules¶
matchzoo.modules.attention
matchzoo.modules.bert_module
matchzoo.modules.character_embedding
matchzoo.modules.dense_net
matchzoo.modules.dropout
matchzoo.modules.gaussian_kernel
matchzoo.modules.matching
matchzoo.modules.matching_tensor
matchzoo.modules.semantic_composite
matchzoo.modules.spatial_gru
matchzoo.modules.stacked_brnn
Package Contents¶
Classes¶
Attention module. |
|
Computing the soft attention between two sequence. |
|
Computing the match representation for Match LSTM. |
|
Dropout for RNN. |
|
Stacked Bi-directional RNNs. |
|
Gaussian kernel module. |
|
Module that computes a matching matrix between samples in two tensors. |
|
Bert module. |
|
Character embedding module. |
|
SemanticComposite module. |
|
DenseNet module. |
|
Module that captures the basic interactions between two tensors. |
|
Spatial GRU Module. |
-
class
matchzoo.modules.
Attention
(input_size: int = 100)¶ Bases:
torch.nn.Module
Attention module.
- Parameters
input_size – Size of input.
mask – An integer to mask the invalid values. Defaults to 0.
Examples
>>> import torch >>> attention = Attention(input_size=10) >>> x = torch.randn(4, 5, 10) >>> x.shape torch.Size([4, 5, 10]) >>> x_mask = torch.BoolTensor(4, 5) >>> attention(x, x_mask).shape torch.Size([4, 5])
-
forward
(self, x, x_mask)¶ Perform attention on the input.
-
class
matchzoo.modules.
BidirectionalAttention
¶ Bases:
torch.nn.Module
Computing the soft attention between two sequence.
-
forward
(self, v1, v1_mask, v2, v2_mask)¶ Forward.
-
-
class
matchzoo.modules.
MatchModule
(hidden_size, dropout_rate=0)¶ Bases:
torch.nn.Module
Computing the match representation for Match LSTM.
- Parameters
hidden_size – Size of hidden vectors.
dropout_rate – Dropout rate of the projection layer. Defaults to 0.
Examples
>>> import torch >>> attention = MatchModule(hidden_size=10) >>> v1 = torch.randn(4, 5, 10) >>> v1.shape torch.Size([4, 5, 10]) >>> v2 = torch.randn(4, 5, 10) >>> v2_mask = torch.ones(4, 5).to(dtype=torch.uint8) >>> attention(v1, v2, v2_mask).shape torch.Size([4, 5, 20])
-
forward
(self, v1, v2, v2_mask)¶ Computing attention vectors and projection vectors.
-
class
matchzoo.modules.
RNNDropout
¶ Bases:
torch.nn.Dropout
Dropout for RNN.
-
forward
(self, sequences_batch)¶ Masking whole hidden vector for tokens.
-
-
class
matchzoo.modules.
StackedBRNN
(input_size, hidden_size, num_layers, dropout_rate=0, dropout_output=False, rnn_type=nn.LSTM, concat_layers=False)¶ Bases:
torch.nn.Module
Stacked Bi-directional RNNs.
Differs from standard PyTorch library in that it has the option to save and concat the hidden states between layers. (i.e. the output hidden size for each sequence input is num_layers * hidden_size).
Examples
>>> import torch >>> rnn = StackedBRNN( ... input_size=10, ... hidden_size=10, ... num_layers=2, ... dropout_rate=0.2, ... dropout_output=True, ... concat_layers=False ... ) >>> x = torch.randn(2, 5, 10) >>> x.size() torch.Size([2, 5, 10]) >>> x_mask = (torch.ones(2, 5) == 1) >>> rnn(x, x_mask).shape torch.Size([2, 5, 20])
-
forward
(self, x, x_mask)¶ Encode either padded or non-padded sequences.
-
_forward_unpadded
(self, x, x_mask)¶ Faster encoding that ignores any padding.
-
-
class
matchzoo.modules.
GaussianKernel
(mu: float = 1.0, sigma: float = 1.0)¶ Bases:
torch.nn.Module
Gaussian kernel module.
- Parameters
mu – Float, mean of the kernel.
sigma – Float, sigma of the kernel.
Examples
>>> import torch >>> kernel = GaussianKernel() >>> x = torch.randn(4, 5, 10) >>> x.shape torch.Size([4, 5, 10]) >>> kernel(x).shape torch.Size([4, 5, 10])
-
forward
(self, x)¶ Forward.
-
class
matchzoo.modules.
Matching
(normalize: bool = False, matching_type: str = 'dot')¶ Bases:
torch.nn.Module
Module that computes a matching matrix between samples in two tensors.
- Parameters
normalize – Whether to L2-normalize samples along the dot product axis before taking the dot product. If set to True, then the output of the dot product is the cosine proximity between the two samples.
matching_type – the similarity function for matching
Examples
>>> import torch >>> matching = Matching(matching_type='dot', normalize=True) >>> x = torch.randn(2, 3, 2) >>> y = torch.randn(2, 4, 2) >>> matching(x, y).shape torch.Size([2, 3, 4])
-
classmethod
_validate_matching_type
(cls, matching_type: str = 'dot')¶
-
forward
(self, x, y)¶ Perform attention on the input.
-
class
matchzoo.modules.
BertModule
(mode: str = 'bert-base-uncased')¶ Bases:
torch.nn.Module
Bert module.
BERT (from Google) released with the paper BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding by Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova.
- Parameters
mode – String, supported mode can be referred https://huggingface.co/pytorch-transformers/pretrained_models.html.
-
forward
(self, x, y)¶ Forward.
-
class
matchzoo.modules.
CharacterEmbedding
(char_embedding_input_dim: int = 100, char_embedding_output_dim: int = 8, char_conv_filters: int = 100, char_conv_kernel_size: int = 5)¶ Bases:
torch.nn.Module
Character embedding module.
- Parameters
char_embedding_input_dim – The input dimension of character embedding layer.
char_embedding_output_dim – The output dimension of character embedding layer.
char_conv_filters – The filter size of character convolution layer.
char_conv_kernel_size – The kernel size of character convolution layer.
Examples
>>> import torch >>> character_embedding = CharacterEmbedding() >>> x = torch.ones(10, 32, 16, dtype=torch.long) >>> x.shape torch.Size([10, 32, 16]) >>> character_embedding(x).shape torch.Size([10, 32, 100])
-
forward
(self, x)¶ Forward.
-
class
matchzoo.modules.
SemanticComposite
(in_features, dropout_rate: float = 0.0)¶ Bases:
torch.nn.Module
SemanticComposite module.
Apply a self-attention layer and a semantic composite fuse gate to compute the encoding result of one tensor.
- Parameters
in_features – Feature size of input.
dropout_rate – The dropout rate.
Examples
>>> import torch >>> module = SemanticComposite(in_features=10) >>> x = torch.randn(4, 5, 10) >>> x.shape torch.Size([4, 5, 10]) >>> module(x).shape torch.Size([4, 5, 10])
-
forward
(self, x)¶ Forward.
-
class
matchzoo.modules.
DenseNet
(in_channels, nb_dense_blocks: int = 3, layers_per_dense_block: int = 3, growth_rate: int = 10, transition_scale_down_ratio: float = 0.5, conv_kernel_size: tuple = 2, 2, pool_kernel_size: tuple = 2, 2)¶ Bases:
torch.nn.Module
DenseNet module.
- Parameters
in_channels – Feature size of input.
nb_dense_blocks – The number of blocks in densenet.
layers_per_dense_block – The number of convolution layers in dense block.
growth_rate – The filter size of each convolution layer in dense block.
transition_scale_down_ratio – The channel scale down ratio of the convolution layer in transition block.
conv_kernel_size – The kernel size of convolution layer in dense block.
pool_kernel_size – The kernel size of pooling layer in transition block.
-
property
out_channels
(self) → int¶ out_channels getter.
-
forward
(self, x)¶ Forward.
-
classmethod
_make_transition_block
(cls, in_channels: int, transition_scale_down_ratio: float, pool_kernel_size: tuple) → nn.Module¶
-
class
matchzoo.modules.
MatchingTensor
(matching_dim: int, channels: int = 4, normalize: bool = True, init_diag: bool = True)¶ Bases:
torch.nn.Module
Module that captures the basic interactions between two tensors.
- Parameters
matching_dims – Word dimension of two interaction texts.
channels – Number of word interaction tensor channels.
normalize – Whether to L2-normalize samples along the dot product axis before taking the dot product. If set to True, then the output of the dot product is the cosine proximity between the two samples.
init_diag – Whether to initialize the diagonal elements of the matrix.
Examples
>>> import matchzoo as mz >>> matching_dim = 5 >>> matching_tensor = mz.modules.MatchingTensor( ... matching_dim, ... channels=4, ... normalize=True, ... init_diag=True ... )
-
forward
(self, x, y)¶ The computation logic of MatchingTensor.
- Parameters
inputs – two input tensors.
-
class
matchzoo.modules.
SpatialGRU
(channels: int = 4, units: int = 10, activation: typing.Union[str, typing.Type[nn.Module], nn.Module] = 'tanh', recurrent_activation: typing.Union[str, typing.Type[nn.Module], nn.Module] = 'sigmoid', direction: str = 'lt')¶ Bases:
torch.nn.Module
Spatial GRU Module.
- Parameters
channels – Number of word interaction tensor channels.
units – Number of SpatialGRU units.
activation – Activation function to use, one of: - String: name of an activation - Torch Modele subclass - Torch Module instance Default: hyperbolic tangent (tanh).
recurrent_activation –
Activation function to use for the recurrent step, one of:
String: name of an activation
Torch Modele subclass
Torch Module instance
Default: sigmoid activation (sigmoid).
direction – Scanning direction. lt (i.e., left top) indicates the scanning from left top to right bottom, and rb (i.e., right bottom) indicates the scanning from right bottom to left top.
Examples
>>> import matchzoo as mz >>> channels, units= 4, 10 >>> spatial_gru = mz.modules.SpatialGRU(channels, units)
-
reset_parameters
(self)¶ Initialize parameters.
-
softmax_by_row
(self, z: torch.tensor) → tuple¶ Conduct softmax on each dimension across the four gates.
-
calculate_recurrent_unit
(self, inputs: torch.tensor, states: list, i: int, j: int)¶ Calculate recurrent unit.
- Parameters
inputs – A tensor which contains interaction between left text and right text.
states – An array of tensors which stores the hidden state of every step.
i – Recurrent row index.
j – Recurrent column index.
-
forward
(self, inputs)¶ Perform SpatialGRU on word interation matrix.
- Parameters
inputs – input tensors.