Problem
Classify news articles as real or fake from their text alone.
Data
WELFake (Kaggle): 72,134 articles, title and body concatenated. 8,577 were exact duplicates of another article after text cleaning; under the original random split, 2,656 test articles (18.4%) had an identical copy in the training set. I deduplicate before splitting, leaving 63,557 articles.
What I built
Text cleaning, TF-IDF with unigrams and bigrams (10,000 features), and logistic regression, compared against a title-only baseline and Multinomial Naive Bayes on the same features.
How I evaluated it
80/20 random split (seed 42) after deduplication.
Results
| LR, title + body | 95.22% |
| LR, title only | 88.91% |
| Naive Bayes, title + body | 86.27% |
| LR, 38 source/format tokens removed | 93.54% |
| LR, before deduplication (leaky split) | 95.76% |
What broke
- The dataset's labels are the reverse of what I'd assumed (0 = real, 1 = fake), so my first per-class report had Fake and Real swapped.
- Duplicates: nearly a fifth of the original test set had an exact copy in training. Removing them cost 0.54 points of accuracy that was never real.
- The most heavily weighted features were source and formatting cues — 'reuters', 'washington reuters', 'york times', and weekday names on the real side; 'image via', 'video', 'images' on the fake side. Removing a fixed list of 38 of them drops accuracy only 1.7 points, because the model falls back on other style cues: how people are named ('hillary' and 'president trump' vs 'president donald' and 'us president'), attribution language ('said', 'spokesman'), and captions ('watch', 'photo'). None of its top features are about whether a claim is true.