Task Grouping for Multilingual Text Recognition

Jing Huang; Kevin J. Liang; Rama Kovvuri; Tal Hassner

doi:10.1007/978-3-031-25069-9_20

Task Grouping for Multilingual Text Recognition

Jing Huang, Kevin J. Liang, Rama Kovvuri, Tal Hassner

Computer Science discipline

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Most existing OCR methods focus on alphanumeric characters due to the popularity of English and numbers, as well as their corresponding datasets. On extending the characters to more languages, recent methods have shown that training different scripts with different recognition heads can greatly improve the end-to-end recognition accuracy compared to combining characters from all languages in the same recognition head. However, we postulate that similarities between some languages could allow sharing of model parameters and benefit from joint training. Determining language groupings, however, is not immediately obvious. To this end, we propose an automatic method for multilingual text recognition with a task grouping and assignment module using Gumbel-Softmax, introducing a task grouping loss and weighted recognition loss to allow for simultaneous training of the models and grouping modules. Experiments on MLT19 lend evidence to our hypothesis that there is a middle ground between combining every task together and separating every task that achieves a better configuration of task grouping/separation.

Original language	English
Title of host publication	Computer Vision – ECCV 2022 Workshops, Proceedings
Editors	Leonid Karlinsky, Tomer Michaeli, Ko Nishino
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	297-313
Number of pages	17
ISBN (Print)	9783031250682
DOIs	https://doi.org/10.1007/978-3-031-25069-9_20
State	Published - 2023
Event	17th European Conference on Computer Vision, ECCV 2022 - Tel Aviv, Israel Duration: 23 Oct 2022 → 27 Oct 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13804 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	17th European Conference on Computer Vision, ECCV 2022
Country/Territory	Israel
City	Tel Aviv
Period	23/10/22 → 27/10/22

Bibliographical note

Publisher Copyright:
© 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Keywords

Multilingual text recognition
OCR
Task grouping

Access to Document

10.1007/978-3-031-25069-9_20

Cite this

Huang, J., Liang, K. J., Kovvuri, R., & Hassner, T. (2023). Task Grouping for Multilingual Text Recognition. In L. Karlinsky, T. Michaeli, & K. Nishino (Eds.), Computer Vision – ECCV 2022 Workshops, Proceedings (pp. 297-313). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13804 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-25069-9_20

Huang, Jing ; Liang, Kevin J. ; Kovvuri, Rama et al. / Task Grouping for Multilingual Text Recognition. Computer Vision – ECCV 2022 Workshops, Proceedings. editor / Leonid Karlinsky ; Tomer Michaeli ; Ko Nishino. Springer Science and Business Media Deutschland GmbH, 2023. pp. 297-313 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{30dd98d0a38f4f108aa155cdf6a111e1,

title = "Task Grouping for Multilingual Text Recognition",

abstract = "Most existing OCR methods focus on alphanumeric characters due to the popularity of English and numbers, as well as their corresponding datasets. On extending the characters to more languages, recent methods have shown that training different scripts with different recognition heads can greatly improve the end-to-end recognition accuracy compared to combining characters from all languages in the same recognition head. However, we postulate that similarities between some languages could allow sharing of model parameters and benefit from joint training. Determining language groupings, however, is not immediately obvious. To this end, we propose an automatic method for multilingual text recognition with a task grouping and assignment module using Gumbel-Softmax, introducing a task grouping loss and weighted recognition loss to allow for simultaneous training of the models and grouping modules. Experiments on MLT19 lend evidence to our hypothesis that there is a middle ground between combining every task together and separating every task that achieves a better configuration of task grouping/separation.",

keywords = "Multilingual text recognition, OCR, Task grouping",

author = "Jing Huang and Liang, {Kevin J.} and Rama Kovvuri and Tal Hassner",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 17th European Conference on Computer Vision, ECCV 2022 ; Conference date: 23-10-2022 Through 27-10-2022",

year = "2023",

doi = "10.1007/978-3-031-25069-9_20",

language = "אנגלית",

isbn = "9783031250682",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "297--313",

editor = "Leonid Karlinsky and Tomer Michaeli and Ko Nishino",

booktitle = "Computer Vision – ECCV 2022 Workshops, Proceedings",

address = "גרמניה",

}

Huang, J, Liang, KJ, Kovvuri, R & Hassner, T 2023, Task Grouping for Multilingual Text Recognition. in L Karlinsky, T Michaeli & K Nishino (eds), Computer Vision – ECCV 2022 Workshops, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13804 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 297-313, 17th European Conference on Computer Vision, ECCV 2022, Tel Aviv, Israel, 23/10/22. https://doi.org/10.1007/978-3-031-25069-9_20

Task Grouping for Multilingual Text Recognition. / Huang, Jing; Liang, Kevin J.; Kovvuri, Rama et al.
Computer Vision – ECCV 2022 Workshops, Proceedings. ed. / Leonid Karlinsky; Tomer Michaeli; Ko Nishino. Springer Science and Business Media Deutschland GmbH, 2023. p. 297-313 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13804 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Task Grouping for Multilingual Text Recognition

AU - Huang, Jing

AU - Liang, Kevin J.

AU - Kovvuri, Rama

AU - Hassner, Tal

PY - 2023

Y1 - 2023

N2 - Most existing OCR methods focus on alphanumeric characters due to the popularity of English and numbers, as well as their corresponding datasets. On extending the characters to more languages, recent methods have shown that training different scripts with different recognition heads can greatly improve the end-to-end recognition accuracy compared to combining characters from all languages in the same recognition head. However, we postulate that similarities between some languages could allow sharing of model parameters and benefit from joint training. Determining language groupings, however, is not immediately obvious. To this end, we propose an automatic method for multilingual text recognition with a task grouping and assignment module using Gumbel-Softmax, introducing a task grouping loss and weighted recognition loss to allow for simultaneous training of the models and grouping modules. Experiments on MLT19 lend evidence to our hypothesis that there is a middle ground between combining every task together and separating every task that achieves a better configuration of task grouping/separation.

AB - Most existing OCR methods focus on alphanumeric characters due to the popularity of English and numbers, as well as their corresponding datasets. On extending the characters to more languages, recent methods have shown that training different scripts with different recognition heads can greatly improve the end-to-end recognition accuracy compared to combining characters from all languages in the same recognition head. However, we postulate that similarities between some languages could allow sharing of model parameters and benefit from joint training. Determining language groupings, however, is not immediately obvious. To this end, we propose an automatic method for multilingual text recognition with a task grouping and assignment module using Gumbel-Softmax, introducing a task grouping loss and weighted recognition loss to allow for simultaneous training of the models and grouping modules. Experiments on MLT19 lend evidence to our hypothesis that there is a middle ground between combining every task together and separating every task that achieves a better configuration of task grouping/separation.

KW - Multilingual text recognition

KW - OCR

KW - Task grouping

UR - http://www.scopus.com/inward/record.url?scp=85150837210&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-25069-9_20

DO - 10.1007/978-3-031-25069-9_20

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85150837210

SN - 9783031250682

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 297

EP - 313

BT - Computer Vision – ECCV 2022 Workshops, Proceedings

A2 - Karlinsky, Leonid

A2 - Michaeli, Tomer

A2 - Nishino, Ko

PB - Springer Science and Business Media Deutschland GmbH

T2 - 17th European Conference on Computer Vision, ECCV 2022

Y2 - 23 October 2022 through 27 October 2022

ER -

Huang J, Liang KJ, Kovvuri R, Hassner T. Task Grouping for Multilingual Text Recognition. In Karlinsky L, Michaeli T, Nishino K, editors, Computer Vision – ECCV 2022 Workshops, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 297-313. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-25069-9_20