Predicting viral memes from image, text and metadata, with an ensemble and a hypergraph model.
Hypergraph NNsCLIPBERTPyTesseractSHAP
The problem
Virality depends on visuals, text, timing and platform dynamics together. A model that looks at only one of them misses most of the signal. The task was to classify memes as viral or not using all three.
Approach
Built a dataset of 5,600+ Reddit memes, with text extracted from images by OCR plus timestamps, subreddit and score.
Approach 1: separate text (BERT), image (CLIP) and metadata classifiers, combined by weighted averaging.
Approach 2: a hypergraph where each meme is a node (CLIP 512-d + BERT 768-d = 1,280-d features) and hyperedges link memes that share subreddit, time bin or score cluster.
Used SHAP and attention visualisation to explain the text and metadata models.
Results
Ensemble: about 69% accuracy, the best overall.
Hypergraph model: about 66 to 67% accuracy, with high recall on viral memes.
Single-modality baselines: text (BERT) 0.62, image (CLIP) 0.68.
Built for the Deep Learning on Graphs course (UE22AM342BA2).
At a glance
Accuracy by approach
Text only (BERT)62%
Image only (CLIP)68%
Hypergraph NN~67%
Multimodal ensemble~69%
Accuracy on the meme dataset, from the project README (rounded).
