Team HausaNLP at SemEval-2026 Task 4: Narratives via Semantic Embeddings
Résumé
This paper presents Team HausaNLP's submission to SemEval-2026 Task 4 (Track A), which requires identifying the more narratively similar of two candidate stories relative to an anchor.Narrative similarity is defined along three dimensions: abstract theme, course of action, and story outcomes.We conduct a systematic ablation comparing five approaches: a lexical TF-IDF baseline, two bi-encoder SBERT variants (all-MiniLM-L6-v2 and all-mpnet-base-v2), a paraphrase-focused embedding model, and a cross-encoder reranker.On the 200-instance development set, all-mpnet-base-v2 achieves the best performance (61.5% accuracy, 61.48 macro-F1), outperforming both TF-IDF (54.5%) and the official SBERT baseline (55.0%).Surprisingly, the cross-encoder re-ranker (55.5%) does not improve on the bi-encoders, which we attribute to the long-document nature of Wikipedia story summaries exceeding the model's effective context window.On the official test set, our primary SBERT MiniLM submission achieved 61.50% accuracy (33rd of 44 teams).Our error analysis over 200 development instances identifies five systematic failure categories, distinct from the All Correct / Partial cases, including 23 Lexical Trap cases, 23 Hard Cases, and 24 Proposed-Recovery cases, thereby informing concrete directions for future work.
Citer ce document
Accès au document
Texte intégral en lecture en ligne, réservé aux abonnés SPHAERO et aux membres de l'institution. Se connecter
Voir l'article sur le site de la revueAuteur(s)
Statistiques
Consultations : 1
Téléchargements : 0