The proliferation of fake news on digital media has led to concerns about the credibility of information and trust. Conventional rule-based filtering may not be effective against sophisticated fake news. Here we build a fully integrated end-to-end fake news detection system utilising TinyBERT (prajjwal1/bert-tiny), a distilled transformer model. Our system is deployed in Google Colab, and involves data collection from a Kaggle source, data preparation (title-text concatenation), tokenization with the Hugging Face AutoTokenizer and supervised fine-tuning with the Trainer API. We compare our proposed system with a TF-IDF and Logistic Regression based baseline. The model achieves an accuracy of 99% and a macro F1-score of 0.99 with a test set of 8,980 samples. The confusion matrix shows only 9 wrong predictions, showing the power of the proposed method. Our findings show that smaller transformer models can produce high accuracy and be deployed in practice.

Index Terms—fake news, TinyBERT, transformers, natural language processing, text classification, knowledge distillation, misinformation