Passer au contenu principal

Vocabulary of natural language processing

Choisissez le vocabulaire dans lequel chercher

  • Aide
aligner
annoter
télécharger

Concept information

NLP methods and tools > NLP software > sentencepiece

Terme préférentiel

sentencepiece  

Concept générique

Contexte(s) définitoire(s)

  • SentencePiece is a powerful and flexible method for unsupervised tokenization and subword segmentation and provides an implementation of the BPE algorithm. (Lamar & Kaya, 2023)

Exemple

  • SentencePiece allows the user to specify special characters that cannot be crossed when constructing subword tokens both during training of the tokenizer and during tokenization of a sentence. (Soulos, Rao, Smith, Rosen, Celikyilmaz, McCoy, Jiang, Haley, Fernandez, Palangi, Gao & Smolensky, 2021)

Traductions

URI

http://data.loterre.fr/ark:/67375/8LP-RLF76QH0-3

Télécharger ce concept :

RDF/XML TURTLE JSON-LD