Removing spam from web corpora through supervised learning using FastText

Warning

This publication doesn't include Faculty of Sports Studies. It includes Faculty of Informatics. Official publication website can be found on muni.cz.
Authors

SUCHOMEL Vít

Year of publication 2017
Type Article in Proceedings
MU Faculty or unit

Faculty of Informatics

Citation
Web Sborník konference
Keywords Text corpora;Web spam;Supervised learning;FastText
Description Unlike traditional text corpora collected from trustworthy sources, the content of web based corpora has to be filtered. This study briefly discusses the impact of web spam on corpus usability and emphasizes the importance of removing computer ge- nerated text from web corpora. The paper also presents a keyword com- parison of an unfiltered corpus with the same collection of texts cleaned by a su- pervised classifier trained using FastText. The classifier was able to recognise 71 % of web spam documents similar to the training set but lacked both precision and recall when applied to short texts from another data set.
Related projects:

You are running an old browser version. We recommend updating your browser to its latest version.

More info