Non-negative matrix factorization (NMF) has proven to be useful in image classification applications such as face recognition. We propose a novel discriminative basis selection method for classification of image categories based on the popular term frequency-inverse document frequency (TF-IDF) weight used in information retrieval. We extend the algorithm to incorporate color, and overcome the drawbacks of using unaligned images. Our method is able to choose visually significant bases which best discriminate between categories and thus prune the classification space to increase correct classifications. We apply our technique to ETH80, a standard image classification benchmark dataset. Our results show that our algorithm outperforms other state-of-the-art techniques.