Unsupervised Style Transfer of Modern Hebrew using Generative Language Modeling and Zero-Shot Prompting

Pavel Kaganovich; Ophir Munz-Manor; Elishai Ezra Tsur

doi:10.1109/BigData59044.2023.10386846

Unsupervised Style Transfer of Modern Hebrew using Generative Language Modeling and Zero-Shot Prompting

Pavel Kaganovich, Ophir Munz-Manor, Elishai Ezra Tsur

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Style transfer is one of the most intriguing hallmarks of natural language processing. It involves the semantic preserving conversion of artistic 'style'. Style transfer of the Hebrew language is an exceptionally challenging task due to the language's intricate morphology, inflectional structure, and orthography, which have undergone significant transformations throughout its history. In this work, we present the first generative language model for unsupervised textual style transfer for modern Hebrew, which rewrites sentences in a target style in the absence of parallel style corpora. We create a pseudo-parallel corpus through back translation, fine-tunes a pre-trained Hebrew language model, and leverages zero-shot learning. Our results demonstrate the first significant results in Hebrew style transfer in terms of transfer accuracy, semantic similarity, and fluency.

Original language	English
Title of host publication	Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023
Editors	Jingrui He, Themis Palpanas, Xiaohua Hu, Alfredo Cuzzocrea, Dejing Dou, Dominik Slezak, Wei Wang, Aleksandra Gruca, Jerry Chun-Wei Lin, Rakesh Agrawal
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	4646-4653
Number of pages	8
ISBN (Electronic)	9798350324457
DOIs	https://doi.org/10.1109/BigData59044.2023.10386846
State	Published - 2023
Event	2023 IEEE International Conference on Big Data, BigData 2023 - Sorrento, Italy Duration: 15 Dec 2023 → 18 Dec 2023

Publication series

Name	Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023

Conference

Conference	2023 IEEE International Conference on Big Data, BigData 2023
Country/Territory	Italy
City	Sorrento
Period	15/12/23 → 18/12/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

Computational Literary Studies
Hebrew Language
Language Model
Machine Learning
Modern Hebrew Literature
Natural Language Processing
Style Transfer

Access to Document

10.1109/BigData59044.2023.10386846

Cite this

Kaganovich, P., Munz-Manor, O., & Tsur, E. E. (2023). Unsupervised Style Transfer of Modern Hebrew using Generative Language Modeling and Zero-Shot Prompting. In J. He, T. Palpanas, X. Hu, A. Cuzzocrea, D. Dou, D. Slezak, W. Wang, A. Gruca, J. C.-W. Lin, & R. Agrawal (Eds.), Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023 (pp. 4646-4653). (Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData59044.2023.10386846

Kaganovich, Pavel ; Munz-Manor, Ophir ; Tsur, Elishai Ezra. / Unsupervised Style Transfer of Modern Hebrew using Generative Language Modeling and Zero-Shot Prompting. Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023. editor / Jingrui He ; Themis Palpanas ; Xiaohua Hu ; Alfredo Cuzzocrea ; Dejing Dou ; Dominik Slezak ; Wei Wang ; Aleksandra Gruca ; Jerry Chun-Wei Lin ; Rakesh Agrawal. Institute of Electrical and Electronics Engineers Inc., 2023. pp. 4646-4653 (Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023).

@inproceedings{05c32a35653548cfb5b94912a9858dec,

title = "Unsupervised Style Transfer of Modern Hebrew using Generative Language Modeling and Zero-Shot Prompting",

abstract = "Style transfer is one of the most intriguing hallmarks of natural language processing. It involves the semantic preserving conversion of artistic 'style'. Style transfer of the Hebrew language is an exceptionally challenging task due to the language's intricate morphology, inflectional structure, and orthography, which have undergone significant transformations throughout its history. In this work, we present the first generative language model for unsupervised textual style transfer for modern Hebrew, which rewrites sentences in a target style in the absence of parallel style corpora. We create a pseudo-parallel corpus through back translation, fine-tunes a pre-trained Hebrew language model, and leverages zero-shot learning. Our results demonstrate the first significant results in Hebrew style transfer in terms of transfer accuracy, semantic similarity, and fluency.",

keywords = "Computational Literary Studies, Hebrew Language, Language Model, Machine Learning, Modern Hebrew Literature, Natural Language Processing, Style Transfer",

author = "Pavel Kaganovich and Ophir Munz-Manor and Tsur, {Elishai Ezra}",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 2023 IEEE International Conference on Big Data, BigData 2023 ; Conference date: 15-12-2023 Through 18-12-2023",

year = "2023",

doi = "10.1109/BigData59044.2023.10386846",

language = "אנגלית",

series = "Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "4646--4653",

editor = "Jingrui He and Themis Palpanas and Xiaohua Hu and Alfredo Cuzzocrea and Dejing Dou and Dominik Slezak and Wei Wang and Aleksandra Gruca and Lin, {Jerry Chun-Wei} and Rakesh Agrawal",

booktitle = "Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023",

address = "ארצות הברית",

}

Kaganovich, P, Munz-Manor, O & Tsur, EE 2023, Unsupervised Style Transfer of Modern Hebrew using Generative Language Modeling and Zero-Shot Prompting. in J He, T Palpanas, X Hu, A Cuzzocrea, D Dou, D Slezak, W Wang, A Gruca, JC-W Lin & R Agrawal (eds), Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023. Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023, Institute of Electrical and Electronics Engineers Inc., pp. 4646-4653, 2023 IEEE International Conference on Big Data, BigData 2023, Sorrento, Italy, 15/12/23. https://doi.org/10.1109/BigData59044.2023.10386846

Unsupervised Style Transfer of Modern Hebrew using Generative Language Modeling and Zero-Shot Prompting. / Kaganovich, Pavel; Munz-Manor, Ophir ; Tsur, Elishai Ezra.
Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023. ed. / Jingrui He; Themis Palpanas; Xiaohua Hu; Alfredo Cuzzocrea; Dejing Dou; Dominik Slezak; Wei Wang; Aleksandra Gruca; Jerry Chun-Wei Lin; Rakesh Agrawal. Institute of Electrical and Electronics Engineers Inc., 2023. p. 4646-4653 (Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Unsupervised Style Transfer of Modern Hebrew using Generative Language Modeling and Zero-Shot Prompting

AU - Kaganovich, Pavel

AU - Munz-Manor, Ophir

AU - Tsur, Elishai Ezra

PY - 2023

Y1 - 2023

N2 - Style transfer is one of the most intriguing hallmarks of natural language processing. It involves the semantic preserving conversion of artistic 'style'. Style transfer of the Hebrew language is an exceptionally challenging task due to the language's intricate morphology, inflectional structure, and orthography, which have undergone significant transformations throughout its history. In this work, we present the first generative language model for unsupervised textual style transfer for modern Hebrew, which rewrites sentences in a target style in the absence of parallel style corpora. We create a pseudo-parallel corpus through back translation, fine-tunes a pre-trained Hebrew language model, and leverages zero-shot learning. Our results demonstrate the first significant results in Hebrew style transfer in terms of transfer accuracy, semantic similarity, and fluency.

AB - Style transfer is one of the most intriguing hallmarks of natural language processing. It involves the semantic preserving conversion of artistic 'style'. Style transfer of the Hebrew language is an exceptionally challenging task due to the language's intricate morphology, inflectional structure, and orthography, which have undergone significant transformations throughout its history. In this work, we present the first generative language model for unsupervised textual style transfer for modern Hebrew, which rewrites sentences in a target style in the absence of parallel style corpora. We create a pseudo-parallel corpus through back translation, fine-tunes a pre-trained Hebrew language model, and leverages zero-shot learning. Our results demonstrate the first significant results in Hebrew style transfer in terms of transfer accuracy, semantic similarity, and fluency.

KW - Computational Literary Studies

KW - Hebrew Language

KW - Language Model

KW - Machine Learning

KW - Modern Hebrew Literature

KW - Natural Language Processing

KW - Style Transfer

UR - http://www.scopus.com/inward/record.url?scp=85184988084&partnerID=8YFLogxK

U2 - 10.1109/BigData59044.2023.10386846

DO - 10.1109/BigData59044.2023.10386846

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85184988084

T3 - Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023

SP - 4646

EP - 4653

BT - Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023

A2 - He, Jingrui

A2 - Palpanas, Themis

A2 - Hu, Xiaohua

A2 - Cuzzocrea, Alfredo

A2 - Dou, Dejing

A2 - Slezak, Dominik

A2 - Wang, Wei

A2 - Gruca, Aleksandra

A2 - Lin, Jerry Chun-Wei

A2 - Agrawal, Rakesh

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2023 IEEE International Conference on Big Data, BigData 2023

Y2 - 15 December 2023 through 18 December 2023

ER -

Kaganovich P, Munz-Manor O , Tsur EE. Unsupervised Style Transfer of Modern Hebrew using Generative Language Modeling and Zero-Shot Prompting. In He J, Palpanas T, Hu X, Cuzzocrea A, Dou D, Slezak D, Wang W, Gruca A, Lin JCW, Agrawal R, editors, Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023. Institute of Electrical and Electronics Engineers Inc. 2023. p. 4646-4653. (Proceedings - 2023 IEEE International Conference on Big Data, BigData 2023). doi: 10.1109/BigData59044.2023.10386846

Unsupervised Style Transfer of Modern Hebrew using Generative Language Modeling and Zero-Shot Prompting

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this