Considering the output of a BLSTM in pytorch, what's the order of the elements?

Question

I am currently using pytorch to implement a BLSTM-based neural network. I understand that the output of the BLSTM is two times the hidden size. However, I am currently unable to find out whether this is ordered as [forward_state_0, backward_state_n, forward_state_1, backward_state_n-1,..., forward_state_n, backward_state_0] or as [forward_state_0, forward_state_1,..., forward_state_n, backward_state_n, backward_stat_n-1,...,backward_state_0] or something else. I'd like to feed a pairwise maximum of the outputs to the next layer so the most import thing for me is which are the corresponding forward and backward states.

Answer 1

Your answer is in the documentation of the code you linked in your comment:

For the unpacked case, the directions can be separated using output.view(seq_len, batch, num_directions, hidden_size), with forward and backward being direction 0 and 1 respectively. Similarly, the directions can be separated in the packed case.

So you just need yo separate the directions as specified above and then take the element-wise maximum with torch.max.