top | item 19503063 (no title) jimfleming | 7 years ago Highway connections are actually used heavily in NLP in combination with LSTMs. discuss order hn newest slashcom|7 years ago And replaced by residual connections in transformers, which are absolutely dominating LSTMs now. stochastic_monk|7 years ago Transformer-XL uses recurrence, and most NLP SOTA is still with LSTMs. I’m not sure I’d expect attention mechanisms to fully replace recurrence.
slashcom|7 years ago And replaced by residual connections in transformers, which are absolutely dominating LSTMs now. stochastic_monk|7 years ago Transformer-XL uses recurrence, and most NLP SOTA is still with LSTMs. I’m not sure I’d expect attention mechanisms to fully replace recurrence.
stochastic_monk|7 years ago Transformer-XL uses recurrence, and most NLP SOTA is still with LSTMs. I’m not sure I’d expect attention mechanisms to fully replace recurrence.
slashcom|7 years ago
stochastic_monk|7 years ago