Exploring Universal Sentence Encoders for Zero-shot Text Classification

Souvika Sarkar; Dongji Feng; Shubhra Kanti Karmaker Santu

Exploring Universal Sentence Encoders for Zero-shot Text Classification

Souvika Sarkar, Dongji Feng, Shubhra Kanti Karmaker Santu

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Universal Sentence Encoder (USE) has gained much popularity recently as a general-purpose sentence encoding technique. As the name suggests, USE is designed to be fairly general and has indeed been shown to achieve superior performances for many downstream NLP tasks. In this paper, we present an interesting “negative” result on USE in the context of zero-shot text classification, a challenging task, which has recently gained much attraction. More specifically, we found some interesting cases of zero-shot text classification, where topic based inference outperformed USE-based inference in terms of F1 score. Further investigation revealed that USE struggles to perform well on data-sets with a large number of labels with high semantic overlaps, while topic-based classification works well for the same.

Original language	American English
Title of host publication	Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing
Editors	Yulan He, Heng Ji, Sujian Li, Yang Liu, Chua-Hui Chang
Publisher	Association for Computational Linguistics
Pages	135-147
Volume	2
State	Published - Nov 2022
Externally published	Yes

Access to Document

https://aclanthology.org/2022.aacl-short.18/

Cite this

Sarkar, S., Feng, D., & Karmaker Santu, S. K. (2022). Exploring Universal Sentence Encoders for Zero-shot Text Classification. In Y. He, H. Ji, S. Li, Y. Liu, & C.-H. Chang (Eds.), Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Vol. 2, pp. 135-147). Association for Computational Linguistics. https://aclanthology.org/2022.aacl-short.18/

Exploring Universal Sentence Encoders for Zero-shot Text Classification. / Sarkar, Souvika; Feng, Dongji; Karmaker Santu, Shubhra Kanti.
Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing. ed. / Yulan He; Heng Ji; Sujian Li; Yang Liu; Chua-Hui Chang. Vol. 2 Association for Computational Linguistics, 2022. p. 135-147.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Sarkar, S, Feng, D & Karmaker Santu, SK 2022, Exploring Universal Sentence Encoders for Zero-shot Text Classification. in Y He, H Ji, S Li, Y Liu & C-H Chang (eds), Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing. vol. 2, Association for Computational Linguistics, pp. 135-147. <https://aclanthology.org/2022.aacl-short.18/>

Sarkar S, Feng D, Karmaker Santu SK. Exploring Universal Sentence Encoders for Zero-shot Text Classification. In He Y, Ji H, Li S, Liu Y, Chang CH, editors, Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing. Vol. 2. Association for Computational Linguistics. 2022. p. 135-147

Sarkar, Souvika ; Feng, Dongji ; Karmaker Santu, Shubhra Kanti. / Exploring Universal Sentence Encoders for Zero-shot Text Classification. Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing. editor / Yulan He ; Heng Ji ; Sujian Li ; Yang Liu ; Chua-Hui Chang. Vol. 2 Association for Computational Linguistics, 2022. pp. 135-147

@inproceedings{80527899447041c1942c4a97de13b1dc,

title = "Exploring Universal Sentence Encoders for Zero-shot Text Classification",

abstract = "Universal Sentence Encoder (USE) has gained much popularity recently as a general-purpose sentence encoding technique. As the name suggests, USE is designed to be fairly general and has indeed been shown to achieve superior performances for many downstream NLP tasks. In this paper, we present an interesting “negative” result on USE in the context of zero-shot text classification, a challenging task, which has recently gained much attraction. More specifically, we found some interesting cases of zero-shot text classification, where topic based inference outperformed USE-based inference in terms of F1 score. Further investigation revealed that USE struggles to perform well on data-sets with a large number of labels with high semantic overlaps, while topic-based classification works well for the same.",

author = "Souvika Sarkar and Dongji Feng and \{Karmaker Santu\}, \{Shubhra Kanti\}",

year = "2022",

month = nov,

language = "American English",

volume = "2",

pages = "135--147",

editor = "Yulan He and Heng Ji and Sujian Li and Yang Liu and Chua-Hui Chang",

booktitle = "Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing",

publisher = "Association for Computational Linguistics",

}

TY - GEN

T1 - Exploring Universal Sentence Encoders for Zero-shot Text Classification

AU - Sarkar, Souvika

AU - Feng, Dongji

AU - Karmaker Santu, Shubhra Kanti

PY - 2022/11

Y1 - 2022/11

N2 - Universal Sentence Encoder (USE) has gained much popularity recently as a general-purpose sentence encoding technique. As the name suggests, USE is designed to be fairly general and has indeed been shown to achieve superior performances for many downstream NLP tasks. In this paper, we present an interesting “negative” result on USE in the context of zero-shot text classification, a challenging task, which has recently gained much attraction. More specifically, we found some interesting cases of zero-shot text classification, where topic based inference outperformed USE-based inference in terms of F1 score. Further investigation revealed that USE struggles to perform well on data-sets with a large number of labels with high semantic overlaps, while topic-based classification works well for the same.

AB - Universal Sentence Encoder (USE) has gained much popularity recently as a general-purpose sentence encoding technique. As the name suggests, USE is designed to be fairly general and has indeed been shown to achieve superior performances for many downstream NLP tasks. In this paper, we present an interesting “negative” result on USE in the context of zero-shot text classification, a challenging task, which has recently gained much attraction. More specifically, we found some interesting cases of zero-shot text classification, where topic based inference outperformed USE-based inference in terms of F1 score. Further investigation revealed that USE struggles to perform well on data-sets with a large number of labels with high semantic overlaps, while topic-based classification works well for the same.

M3 - Conference contribution

VL - 2

SP - 135

EP - 147

BT - Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing

A2 - He, Yulan

A2 - Ji, Heng

A2 - Li, Sujian

A2 - Liu, Yang

A2 - Chang, Chua-Hui

PB - Association for Computational Linguistics

ER -