LLM Benchmarking Framework for Financial Sentiment Analysis
// machine learning / evaluation
Built a financial sentiment analysis benchmarking framework comparing OpenAI GPT-4o and Google Gemini 2.5 Flash on real-world finance datasets. The project automates dataset processing, prompt standardisation, prediction validation, evaluation metrics, and result visualisation to analyse how different LLMs perform on sentiment classification tasks.
Implemented a full evaluation pipeline using Python and Jupyter across the Financial PhraseBank and FiQA datasets, covering standardised 3-class sentiment prediction (positive, negative, neutral), automated response validation, and label normalisation. Evaluation included accuracy, macro F1, per-class F1, latency, and hallucination analysis, with statistical significance testing via McNemar's Test and comparative confusion matrix visualisations.
The benchmark surfaced a substantial difference in classification behaviour while Gemini 2.5 Flash returned lower average latency in the measured sample. The project focused on reproducible benchmarking, clean evaluation methodology, and structured LLM comparison workflows rather than relying on anecdotal model outputs.
- →Financial PhraseBank and FiQA dataset handling
- →Standardised 3-class sentiment prediction across both models
- →Accuracy, macro F1, per-class F1, latency, and hallucination analysis
- →Automated response validation and label normalisation
- →Statistical significance testing using McNemar's Test
- →Confusion matrices and comparative visualisations
- →Optimised API usage to reduce failed calls and improve cost efficiency
Model Evaluation Evidence
Comparative benchmark outputs, classification metrics, and latency analysis artifacts from the financial sentiment LLM evaluation framework.
select artifact to expand