Composition of weighted finite transducers in MapReduce - Université du Littoral Côte d'Opale
Article Dans Une Revue International Journal of Big Data Intelligence Année : 2021

Composition of weighted finite transducers in MapReduce

Résumé

Weighted finite-state transducers have been shown to be a general and efficient representation in many applications such as text and speech processing, computational biology, and machine learning. The composition of weighted finite-state transducers constitutes a fundamental and common operation between these applications. The NP-hardness of the composition computation problem presents a challenge that leads us to devise efficient algorithms on a large scale when considering more than two transducers. This paper describes a parallel computation of weighted finite transducers composition in MapReduce framework. To the best of our knowledge, this paper is the first to tackle this task using MapReduce methods. First, we analyze the communication cost of this problem using Afrati et al. model. Then, we propose three MapReduce methods based respectively on input alphabet mapping, state mapping, and hybrid mapping. Finally, intensive experiments on a wide range of weighted finite-state transducers are conducted to compare the proposed methods and show their efficiency for large-scale data.
Fichier principal
Vignette du fichier
bmc_article.pdf (378.18 Ko) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte
Licence

Dates et versions

hal-03507182 , version 1 (18-04-2023)

Licence

Identifiants

Citer

Bilal Elghadyry, Faissal Ouardi, Sébastien Verel. Composition of weighted finite transducers in MapReduce. International Journal of Big Data Intelligence, 2021, 8 (22), ⟨10.1186/s40537-020-00397-4⟩. ⟨hal-03507182⟩
41 Consultations
41 Téléchargements

Altmetric

Partager

More