Negation Disrupts Compositionality in Language Models: The Czech Usecase
| Authors | |
|---|---|
| Year of publication | 2024 |
| Type | Article in Proceedings |
| Conference | The Eighteenth Workshop on Recent Advances in Slavonic Natural Language Processing |
| MU Faculty or unit | |
| Citation | VRABCOVÁ, Tereza and Petr SOJKA. Negation Disrupts Compositionality in Language Models: The Czech Usecase. In Aleš Horák, Pavel Rychlý, Adam Rambousek. The Eighteenth Workshop on Recent Advances in Slavonic Natural Language Processing. Brno: Tribun EU, 2024, p. 17-24. ISSN 2336-4289. |
| web | fulltext PDF |
| Keywords | negation; language models; machine learning |
| Description | In most Slavic languages, the negation is expressed by short “ne” tokens that do not affect discrete change in the meaning learned distributionally by language models. It manifests in many problems, such as Natural Language Inference (NLI). We have created a new dataset from CsFEVER, the Czech factuality dataset, by extendingitwithnegatedversionsofhypothesespresentinthe dataset. We used this new dataset to evaluate publicly available language models and study the impact of negation on the NLI problems. We have confirmed that compositionally computed representation of negation in transformers causes misunderstanding problems in Slavic languages such as Czech: The reasoning is flawed more often when the information is expressed using negation than when it is expressed positively without it. Our findings highlight the limitations of current transformer models in handling negation cues in Czech, emphasizing the need for further improvements to enhance language models’ understanding of Slavic languages. |
| Related projects: |