Why I Built This
When I started planning my undergraduate thesis at Telkom University, I wanted to tackle misinformation on social media. Most existing studies I read looked only at tweet text. In practice, however, false claims on platforms like X (Twitter) rarely rely on text alone. People share misleading screenshots, edited images, and memes alongside their captions, while their account history provides subtle cues about authenticity.
I wanted to see whether fusing text, image features, and user metadata into a single deep learning pipeline could improve classification accuracy over standard text-only baselines.
How I Built the Pipeline
1. Data Collection and Representation
I assembled a dataset of 23,564 annotated tweets with a balanced 50:50 split between credible and non-credible content. Each record paired the post text with its associated image and account metadata.
Because standard pre-trained embeddings often struggle with Indonesian social media slang, I built a custom GloVe embedding model. I trained it across a combined corpus of 62,274 records, pairing the tweet dataset with 38,710 articles from IndoNews to capture both formal news reporting and informal social vocabulary.
2. Multimodal Fusion Architecture
I combined three feature streams into a single classification network:
- Text Stream: Combined TF-IDF for lexical frequency statistics and my custom GloVe embeddings for semantic relationships.
- Visual Stream: Extracted lightweight image feature maps using MobileNetV1.
- Metadata Stream: Encoded account age, verification status, follower ratios, and posting velocity.
3. Automated Tuning with Particle Swarm Optimization
Instead of tuning hyperparameters through manual trial and error, I implemented Particle Swarm Optimization (PSO). The swarm explored the parameter search space to find optimal combinations for learning rate, dense layer dimensions, and dropout probabilities.
Findings and Results
| Model Architecture | Accuracy |
|---|---|
| CNN-BiGRU + PSO (Proposed) | 79.09% |
| BiGRU-CNN + PSO | 77.93% |
Integrating visual and account metadata produced a +4.95% accuracy gain compared to text-only classification. The model achieved 80.38% precision on credible posts and 80.97% recall on non-credible posts, confirming that visual context helps catch misleading content that text analysis misses.
Publication
This research was accepted and published in the peer-reviewed proceedings of the IEEE International Conference on ICT for Smart Society (ICICyTA):
- DOI: 10.1109/ICICyTA68677.2025.11362759
- Conference: IEEE International Conference on ICT for Smart Society (ICICyTA)
- Institution: Telkom University
- Academic Honors: Summa Cum Laude (GPA 3.96/4.00)
Technologies: Python, PyTorch, TensorFlow, CNN, BiGRU, GloVe, TF-IDF, MobileNetV1, Particle Swarm Optimization
Last updated on September 16, 2026 at 12:23 PM UTC+7. See Changelog