A Symmetric Term Weighting Scheme for Text Categorization Based on Term Occurrence Probabilities

Abstract

Term weighting schemes used in text categorization can be considered as functions of term occurence probabilities in positive and negative classes. In this paper, widely used weighting schemes are firstly evaluated from this perspective. Then, a novel feature weighting scheme based on term occurrence probabilities is proposed. Experiments conducted using SVM classifier on the Reuters-21578 ModApte Top10 dataset shows that the proposed method outperforms other well known measures such as CHI, IG, OR and RF in terms of macro-F-1 and micro-F-1 scores.