Skip to main content

Vocabulary of natural language processing

Search from vocabulary

  • Help
align
annotate
download

Concept information

... > NLP methods and tools > artificial intelligence > machine learning > machine learning model > neural networks model > neuron > layer > attention mechanism > multi-head attention

Preferred term

multi-head attention  

Definition

  • An attention mechanism used in NLP tasks such as machine translation and summarization that involves dividing the input sequence into multiple heads and combining their outputs to produce a single output, allowing the attention mechanism to focus on different aspects and relationships in the input sequence. (Multi Head Attention, on medium.com, 2023)

Broader concept

Example

  • Further analyses show that our multi-head attention is able to attend information from various aspects and boost classification or generation in diverse scenarios. (Wang, Li, Lyu & King, 2020)
  • Moreover we propose a novel Multi-Modality Multi-Head Attention to capture the dense interactions between texts and images where image wordings explicit in optical characters and implicit in image attributes are further exploited to bridge their semantic gap. (Wang, Li, Lyu & King, 2020)
  • Then we apply multi-head attention between the output of those attention layers expecting that the source context helps the decoder to recognize those common words which should be remained in post-edited sentence. (Shin & Lee, 2018)

In other languages

URI

http://data.loterre.fr/ark:/67375/8LP-G68PZTML-J

Download this concept:

RDF/XML TURTLE JSON-LD Last modified 5/21/24