Passer au contenu principal

Vocabulary of natural language processing

Choisissez le vocabulaire dans lequel chercher

  • Aide
aligner
annoter
télécharger

Concept information

... > NLP methods and tools > artificial intelligence > machine learning > machine learning model > neural networks model > deep neural network > residual connection

Terme préférentiel

residual connection  

Définition

  • A technique used to facilitate the optimization of deep neural networks by connecting the output of one layer to the input of a later layer, skipping one or more layers in between, which enables better flow of information through the network. (Liu et al., Rethinking Skip Connection with Layer Normalization, 2020)

Concept générique

Synonyme(s)

  • skip connection

Exemple

  • All available embeddings for a token are concatenated and processed with two bidirectional LSTM layers with residual connections. (Straka & Straková, 2019)
  • Each sub-layer uses residual connections around each of the sub-layers followed by layer normalization. (Prabhumoye, Black & Salakhutdinov, 2020)
  • It has been shown that residual connections help propagate positional information across layers. (Onabola, Ma, Yang, Akera, Abdulrahman, Xue, Liu & Bengio, 2021)
  • The ELMo encoder employs CNN and highway networks over characters whose output is given to a multiple-layer biLSTM with residual connections. (Bai & Zhao, 2018)

Traductions

URI

http://data.loterre.fr/ark:/67375/8LP-C1M2Z3B5-P

Télécharger ce concept :

RDF/XML TURTLE JSON-LD Dernière modification le 10/06/2024