Negation Disrupts Compositionality in Language Models: The Czech Usecase

Warning

This publication doesn't include Faculty of Sports Studies. It includes Faculty of Informatics. Official publication website can be found on muni.cz.
Authors

VRABCOVÁ Tereza SOJKA Petr

Year of publication 2024
Type Article in Proceedings
Conference The Eighteenth Workshop on Recent Advances in Slavonic Natural Language Processing
MU Faculty or unit

Faculty of Informatics

Citation
web fulltext PDF
Keywords negation; language models; machine learning
Description In most Slavic languages, the negation is expressed by short “ne” tokens that do not affect discrete change in the meaning learned distributionally by language models. It manifests in many problems, such as Natural Language Inference (NLI). We have created a new dataset from CsFEVER, the Czech factuality dataset, by extendingitwithnegatedversionsofhypothesespresentinthe dataset. We used this new dataset to evaluate publicly available language models and study the impact of negation on the NLI problems. We have confirmed that compositionally computed representation of negation in transformers causes misunderstanding problems in Slavic languages such as Czech: The reasoning is flawed more often when the information is expressed using negation than when it is expressed positively without it. Our findings highlight the limitations of current transformer models in handling negation cues in Czech, emphasizing the need for further improvements to enhance language models’ understanding of Slavic languages.
Related projects:

You are running an old browser version. We recommend updating your browser to its latest version.

More info