Final Project Report

Bal-Wijzer: Chatbot for Sports Analytics using Deep Learning

By
Muhammad Momin Rauf 366523
Muhammad Ehtisham Raza 374965
Muhammad Taha Khan 393503

Supervisor:
Dr. Rafia Mumtaz

Co-Supervisors:
Dr. Mehdi Hussain & Dr. Daud Abdullah

Bachelor of Eng. in Software Engineering (2021-2025)

Department of Computing
School of Electrical Engineering and Computer Science
National University of Sciences & Technology
Declaration

It is declared that this final report for the final year project titled “Chatbot for Sports Analytics - Balwijzer”
submitted to SEECS is a record of original work done by team member Muhammad Momin Rauf,
Muhammad Ehtisham Raza and Muhammad Taha Khan under the combined guidance of supervisor Dr.
Rafia Mumtaz and Co-Supervisors Dr. Mehdi Hussain and Dr. Daud Abdullah Asif. No part has been
plagiarized without citations, and this project is submitted in partial fulfillment of the requirements for the
degree Bachelor of Engineering in Software Engineering.

Team Member(s):
Muhammad Momin Rauf
Muhammad Taha Khan
Muhammad Ehtisham Raza

Supervisor:
Dr. Rafia Mumtaz

Date: 15/5/2025
Place: SEECS, NUST H-12 Campus, Islamabad
 Dedicated to

Allah Almighty
without the Power of whom this project would not have been possible,
To
our dear parents, advisor, co-advisors,
whose help makes this project successful.
Acknowledgements

This project has been completed under the mentorship and guidance of our respected Advisor Dr. Rafia
Mumtaz and Co-Advisors Dr Mehdi Hussain and Dr Daud Abdullah Asif. We extend our deepest gratitude
to all for their continued advice throughout this project. We also extend our heartiest appreciation for our
dream university, National University of Sciences and Technology (NUST) and school, School of Electrical
Engineering and Computer Science (SEECS) for providing me with the platform to develop our skills to
contribute as multidimensional persons to the world.
Table of Contents
Declaration
Acknowledgements
Abstract
1. Introduction
2. Literature Review
3. Problem Statement
4. Project Overview and Goals
5. Project Development Methodology
6. Project Milestones and Deliverables
7. Tools, Technologies, and Method
8. Overall Description
9. User Classes and Characteristics
10. Operating Environment
11. User Documentation
12. External Interface Requirements
13. System Features
14. Non-Functional Requirements
15. Other Requirements
16. Design Methodology and Software Process Model
17. System Overview
18. Design Models
19. Data Design
20. User Interface Design
21. Project Management Tool
22. Testing…………………………………………………………………………………………………..
Abstract

Targeting young adults, football fanatics, and sports analysts, this project aims to develop a product that
provides personalized follow-up to stats through customized data augmentation and chatbot-based user
interaction, with a multi-object tracking (MOT) module at the back end. This product will offer users
multiple options to analyze and enjoy the game, with real-time results being one of the primary challenges
the project will address. The solution will integrate advanced AI techniques and real-time data processing to
enhance user experience, making the platform not just informative but also highly interactive. By offering a
seamless blend of data visualization and natural language processing, the project aspires to set a new
standard for engaging with live football broadcasts.
1. Introduction
1.1 Purpose
This Report - Version 1.0, marks a critical phase in the development lifecycle of the Computer Vision-Based
Sports Analytics Chatbot, a core component of the ongoing Final Year Project, “Chatbot for Sport Analytics
-Bal Wijzer" The primary goal of this report is to provide a comprehensive blueprint outlining the design,
architecture, and technical details essential for implementing the proposed system. Building on the
objectives defined in the Software Requirements Specification (SRS) Version 1.0, this document serves as a
structured guide for the development team, facilitating a systematic approach to the implementation phase.
The project aims to revolutionize sports analysis by integrating computer vision techniques with LLM
enabling a chatbot to analyze and interpret sports videos in real time while providing users with actionable
insights and intuitive responses. This innovative solution leverages state-of-the-art computer vision
techniques (e.g., object detection, tracking, and event detection) alongside conversational AI to process
sports videos dynamically. Users can interact with the chatbot to gain insights into player performance,
game events, and team strategies through natural language queries. The overarching objective is to create a
real-time, user-friendly platform that bridges advanced analytics with accessibility for sports enthusiasts and
professionals alike. The SDS outlines the foundational design principles for the automation of video analysis
and user interaction, ensuring scalability, efficiency, and precision. By reducing the complexity of traditional
sports analysis workflows, this project empowers users to make informed decisions effortlessly, fostering
deeper engagement and understanding of sports events. This document serves as a cornerstone for the
software development process, providing detailed insights into the architectural framework, system
components, and technical implementation strategies. It ensures that the development process aligns with
the stipulated requirements in the SRS, paving the way for the successful creation and deployment of the
Computer Vision-Based Sports Analysis Chatbot.

1.2 Document Conventions

This final report document is prepared by following the software and systems engineering standards of the
Institute of Electrical and Electronics Engineers (IEEE). Each system requirement has been assigned a
unique requirement number and priority level.

1.2.1 Font Styles and Sizes

[Link] Heading Styles:

Heading 1: Times New Roman, 18 Font-Size, Bold, Black, Title Case.
Heading 2: Times New Roman, 16 Font-Size, Bold, Black, Title Case.
Heading 3: Times New Roman, 14 Font-Size, Bold, Black, Title Case.
Heading 4: Times New Roman, 12 Font-Size, Bold, Black, Title Case.

[Link] Body Text:

Body text, including descriptions and content: Times New Roman, 12 Font-Size.
1.2.2 Notations and Symbols
[1.0]: SRS Version/Release number

1.3 Intended Audience and Reading Suggestions

This report document is intended for the following audiences:
1. Developers: To understand the system requirements, design constraints, and implementation details
necessary for developing the software.
2. Users: Specifically, end-users like sports analysts, football fans to understand the system’s
capabilities and features they will interact with.
3. Testers: To ensure the system meets the requirements through comprehensive testing.
4. Documentation Writers: To create user manuals, help guides, and other relevant documentation
based on the specified requirements.

1.4 Product Scope

The Bal wijzer is designed to enhance the football experience by providing real-time analysis of match
events, player statistics, and interactive visualizations using augmented reality. The software leverages state
of the art YOLO model for object detection and tracking algorithms like Deepsort and Bytetrack, enabling it
to detect and track players and key events (e.g., goals, fouls, tackles etc.) using Wide range of Soccernet
Dataset. Additionally, the product integrates a chatbot using an LLM to provide users with interactive and
data-rich insights, such as real-time match stats and historical comparisons.

2. Literature Review
2.1 Research based
2.1.1 Sporthesia: Augmenting Sports Videos Using Natural Language
Augmentation for sports analysis has always been engaging for sports enthusiasts around the world, but
visualization has never been easy; it demands skill in video editing and graphics. This solution provides a
way to visualize data using natural language.
This advanced system is specifically designed for racket-based sports such as table tennis, squash, and
tennis.
The system comprises three key components:
• Entity Detector
• Entity Visualizer
• Visualization Scheduler.
The Entity Detector identifies and extracts visualizable entities from the text, while the Entity Visualizer
maps these entities into dynamic visualizations. Finally, the Visualization Scheduler coordinates the timing
of these visualizations with the sports video, ensuring seamless integration.
Utilizing state-of-the-art natural language processing (NLP) models, this system enhances sports analysis
with advanced data augmentation.

2.1.2 The quest for Omniocular: Embedded visualization for augmenting basketball
game viewing experiences.
Omniocular is the embedding visualization for augmenting basketball game viewing experiences. It is used
for in-game analysis of basketball live matches. It uses visual analysis to show different types of
visualization according to the preference of the audience which includes basketball fans and expert analysts.
• It has conducted user surveys to get know about which types of information the User wants to
see while watching an online match.
• They shortlisted five importants types of visualization which includes Shooting, Offense
Trajectory, Defense Form, Short Chart and Team Stats.
• User Study is conducted which includes basketball fans to see how they find this new system.
The Result shows that they like this new visual data analysis to be beneficial in understanding
the game better.

2.1.3 CourtVision: New Visual and Spatial Analytics for the NBA
CourtVision is a new analytical technique which combines visual and spatial analysis of the National
Basketball Association (NBA). It is presented by Kirk Goldsberry. It shows the use of visual and spatial
analytics to better understand the game.
• It is based on finding the answer to the question “Who is the best Shooter?” Previously used
metrics like Field Goal Percentage (FG%) were failing to identify the best shooter.
• In order to find the answer of the question a spatial analysis of the basketball court is conducted.
Along with the visual analysis of the players playing the game which include their shooting for
the game.
• Its metrics were created which includes a Spread which shows unique shooting areas where a
player scores at least one goal. The second was the Range which showed the spatial influences
on the shooting area.
• Using these metrics, a visual representation is created which is beneficial in finding the best
shooter who was Steve Nash.
• This result concluded by showing the importance of visual and spatial analysis in sports.

2.1.4 A deep learning ball tracking system in soccer videos (DLBT)

Abstract: This paper enhances the object tracking by using Deep learning-based models. addressing
challenges such as occlusions, varying object appearances, and motion blur using advanced deep learning
model and increasing the tracking efficiency
Methodology: The proposed method integrates convolutional neural networks (CNNs) for feature
extraction and object representation, combined with traditional tracking algorithms like Kalman filters or
particle filters. The system uses end-to-end training on large-scale annotated datasets to optimize
performance.
Dataset: benchmark tracking datasets: OTB (Object Tracking Benchmark) and VOT (Visual Object
Tracking)

2.1.5 FOOTYVISION BY PETER ANDREWS

Abstract: FootyVision is a state-of-the-art computer vision module designed for tracking, detection, and
perspective transformation of players and the ball in football games. It utilizes a YOLOv7 backbone and
assigns identities via the Hungarian algorithm based on feature embeddings. FootyVision is an all-in-one
model for Multi-Object Tracking (MOT) and perspective transformations for uncalibrated broadcast video.
Methodology: The FootyVision module incorporates three main components:
• YOLOv7 Network: For player and ball detection.
• Tracking Module: Assigns identities to detections using the Hungarian algorithm based on
feature embeddings.
• Perspective Transformation Module: Computes homographies for top-down localization of
tracks.
Dataset: For validation, FootyVision is evaluated using the ISSIA and SoccerNet datasets. Specifically, for
SoccerNet, FootyVision achieves a MOTA of 94.04% and scores competitive results for ISSIA. It also
scores a HOTA(0) of 93.1% and an overall HOTA of 72.16%. also trained on soccernet and ISSIA
Performance: FootyVision sets new benchmarks in performance, achieving a mean average precision
(mAP) of 95.7% and an F1-score of 95.5% in object detection.

2.1.6 Revealing the invisible: Visual analytics and explanatory storytelling for advanced
team sport analysis.
Abstract: The paper "Revealing the Invisible: Visual Analytics and Explanatory Storytelling for Advanced
Team Sport Analysis" presents a system for enhancing the understanding of team sports through visual
analytics and explanatory storytelling. Focused on soccer, the most widely played game, the system is
designed to improve sports analysis by providing in-depth insights.
Methodology: The system consists of four key components:
• Classification: Assesses the available visualizations.
• Specification: Provides familiarity with the situation context.
• Explanation: Involves storytelling to convey insights.
• Alteration: Includes "What if" analysis to explore different scenarios.
Dataset: The analysis is conducted on soccer data, focusing on player passes and interactions between
players from both teams.
Performance: The study demonstrates how the visual analytics and explanatory storytelling system
effectively highlights key aspects of team sports analysis, receiving positive feedback from former coaches
who were interviewed for the study.

2.1.7 Laliga chatbot

Abstract: The paper "Chatbol: A Chatbot for the Spanish ‘La Liga’" introduces Chatbot, an interactive
conversational model designed to provide insights on La Liga , trained on football content it enhances the
user experience by allowing him to chat and extract insights about football data
Dataset: The chatbot utilizes comprehensive football data from La Liga, including historical match data,
player statistics, and real-time updates to deliver relevant and precise information to users.
Performance: Chatbol demonstrates effective user interaction and engagement by accurately responding to
inquiries related to La Liga, enhancing the overall experience for football enthusiasts and providing a
valuable resource for league-related information.

2.1.8 Ball and Player Detection & Tracking in Soccer Videos Using Improved YOLOV3
Model
Abstract: The paper "Ball and Player Detection & Tracking in Soccer Videos Using Improved YOLOv3
Model" presents an enhanced version of the YOLOv3 model specifically tailored for detecting and tracking
players and the ball in soccer videos. The improved model aims to address the challenges of real-time
tracking and accurate detection in dynamic sports environments.
Methodology: The research introduces modifications to the YOLOv3 architecture to enhance its
performance for soccer video analysis. These improvements include adjustments to the network's layers and
training processes to better handle the complexities of ball and player detection in varying conditions.
Dataset: The study utilizes a dataset of soccer videos to train and evaluate the improved YOLOv3 model.
This dataset includes various match scenarios to ensure the model's robustness across different types of
soccer footage.
Performance: The enhanced YOLOv3 model achieves significant improvements in detection and tracking
accuracy compared to the standard YOLOv3 model. It demonstrates better performance in identifying and
following players and the ball throughout soccer games, making it effective for real-time sports analytics.

2.1.9 A Survey on Object Detection and Tracking in Soccer Videos

Abstract: Detecting and tracking players and the ball in soccer videos is challenging due to issues like
shadows, overlapping objects, and varying ball visibility. This paper reviews various detection and tracking
methods used in soccer videos.
Methodologies:
• YOLOv3: For object detection.
• Kalman Filter: For ball tracking.
 • Particle Filter: For ball tracking.
• Multiple Hypothesis Tracking: For player tracking.
Performance:
• YOLOv3: High accuracy in detection.
• Kalman Filter: Ball tracking accuracy: 81% to 89.85%.
• Particle Filter: Ball tracking accuracy: ~92%.
• Multiple Hypothesis Tracking: Effective but can fail with extreme occlusions.
Datasets: No specific datasets mentioned; focuses on general methodologies and their performance in
soccer videos.

2.1.10 YOLOv3-SORT: detection and tracking player/ball in soccer sport

Abstract: Soccer player and ball detection and tracking are crucial for evaluating team performance and
aiding coaches in decision-making. Existing methods struggle with high-velocity ball tracking and player
tracking under occlusion. This study proposes a YOLOv3 and SORT-based approach to address these
challenges by accurately detecting and tracking players and the ball in soccer videos.
Methodologies:
• YOLOv3: For detecting and classifying objects (players, ball, background) and filtering out non-
playfield objects.
• SORT (Simple Online and Realtime Tracking): For tracking using Kalman filtering and
bounding box overlap.
Performance:
• Tracking Accuracy: 93.7% on multiple objects tracking metrics.
• Detection Speed: 23.7 frames per second (FPS).
• Tracking Speed: 11.3 FPS.

2.1.11 SportVis: Discovering Meaning in Sports Statistics Through Information

Visualization
Abstract: SportVis leverages information visualization to analyze and extract insights from extensive sports
statistics, addressing challenges such as deriving patterns and answering complex queries from large
datasets. It focuses on baseball and provides interactive visual tools to enhance the understanding of team
and player performance.
Methodologies: SportVis utilizes two main visualizations:
• Baseline Bar Display: Represents games with bars indicating outcomes, run margins, and scores,
enabling easy trend detection.
• Player Map: Employs treemaps to visualize player performance metrics and playing time,
highlighting areas of inefficiency or potential improvement.
Performance: SportVis provides interactive and dynamic exploration of data, allowing users to filter, sort,
and analyze performance metrics. It effectively facilitates the discovery of trends and patterns in large
datasets, though its current implementation focuses on aggregate rather than detailed game-level analysis.
Datasets: The visualizations use data from Retrosheet and Baseball Databank, covering game results, player
statistics, and other relevant metrics for comprehensive season and game-level analysis.

2.1.12 Deep learning in sport video analysis: a review

Sport is a competitive field, where it is an element of measurement for a countries development. Due to this
reason, sport analysis has become one of the major contributions in analyzing and improving the
performance level of an athlete. Video-based modality has become a crucial tool used in sport analysis by
coaches and performance analysis. There were wide variety of techniques used in sport video analysis. The
main purpose of this review paper is to compare and update review between traditional handcrafted
approach and deep learning approach in sport video analysis based on human activity recognition, overview
of recent study in video based human activity recognition in sport analysis and finally concluded with future
potential direction in sport video analysis.

2.1.13 A Comprehensive Review of Computer Vision in Sports: Open Issues, Future

Trends, and Research Directions
Recent advancements in sports video analysis and computer vision techniques have significantly improved
operations such as player detection, tracking, trajectory prediction, strategy recognition, and event
classification in sports like soccer, basketball, and cricket. This paper reviews various high-level
applications of sports video analysis, discussing existing work and perspectives on AI applications, GPU-
based systems, and embedded platforms. It also highlights research directions, challenges, and future trends
in sports visual recognition.

2.1.14 Bring it to the pitch: Combining video and movement data to enhance team
sport analysis.
The movement data of different sports helps in understanding the movement behaviors of different teams.
For the sport analysts and different teams they need to study these behaviors to understand the strategy
involved to better analyze.
• Embedded visualization helps in understanding these tactics.
• The videos of the game taken from a single camera on the pitch are used and visualizations are
created to study the strategy adopted by the teams during the game.
• Different analysis is conducted using these embedded visualizations of how the teams’ players
react in different events of passing, shooting and goal scoring.
• Free Spaces, Interaction Spaces and Dominant regions are embedded so that gives a good
understanding of the events made by the player.
• This helps in better understanding the game tactics and making good decisions in their upcoming
games using the embedded visualization on the video.
2.2 SELECTED RESEARCH PAPER:
FootyVision, which provides tracking along with perspective transformation, is the research-based solution
closest to our problem. It has a good balance between accuracy and real-time performance and has been able
to achieve state-of-the-art results.
Improvement Proposed:
Although the FootyVision module has achieved state-of-the-art results in tracking and detection, it is still
not highly efficient for real-time processing. Based on YOLOv7 as the backbone, we will fine-tune newly
released state-of-the-art CNN models like YOLOv8 or YOLOv10. Along with preprocessing and optimized
real-time processing, we aim to deliver a model with state-of-the-art accuracy and real-time performance.

2.3 Product based

2.3.1 Fotovision
Fotovision uses state-of-the-art computer vision models to analyze the performance of players and teams on
a user-interactive dashboard. They cover all ball and non-ball events.
Data Collection
 From any video feed.
 Tracking of players, referees, and the ball with over 2 million data points per match.
Video and Performance Analysis
 900 performance metrics
 Compute game events with and without the ball.
Event Detection
 They detect all events and phases of the game, such as attack, counterattack, possession, etc.

2.3.2 AI Commentator by MediaFutures/ACM

Redesigning the commentary role in football games with AI-Commentator - MediaFutures
This solution is based on the state-of-the-art module FootyVision (by Andrews), which provides perspective
transformation along with tracking of players and the ball.
“FootyVision benefits from the ability to localize players in a top-down viewpoint even when there is a lack
of visual information. It outperforms prior tracking algorithms, setting a comprehensive benchmark for
others working in football tracking,”
(Source: Andrews, Peter. "FootyVision." MediaFutures).
They provide an interactive user interface linked with Discord, which helps users interact with the live
stream using a chatbot. The chatbot offers two ways to interact with the system: menu-based and natural
language using NLP, augmenting the response as an overlay on the screen.
One disadvantage of this solution is that it still has some problems when dealing with real-time data.
It is yet to be presented in the market.

2.3.3 GameBot
GameBot, developed by Qiyu Zhi and Ronald Metoyer at the University of Notre Dame, enhances fan
engagement by providing detailed statistical information and visualizations related to sports games (Source:
Zhi, Qiyu, and Metoyer, Ronald. "GameBot: A Visualization-Augmented Chatbot for Sports Games,"
CHI’20 Extended Abstracts). Key features include:
• Direct Answers: Specific answers to stats-related queries.
• Visual Context: Data visualizations like game flow charts and performance bar charts.
• System Architecture: Data scraping, intent classification, stats extraction, and output generation.
• Future Work: Evaluating visualization effectiveness, improving intent classification, and
exploring narrative visualization.

3. Problem Statement
3.1 Unmet Need or Problem
Main problem this project is trying to solve is decline in live viewership of football game, particularly
among Generation Z. Recent studies and surveys suggest that only 63% of the Gen Z prefer to watch live
games , the numbers for Boomers are comparatively high (90%) (Source:pwc-survey-2021). Furthermore, It
has been surveyed that almost 77% of the young adults prefer to use a secondary screen for the follow up of
the stats this a problem when it comes to user experience. According to PwC's sports survey, 79.4% of
respondents believe that improving on-site and online fan experiences is crucial (Source: PwC Analysis, N =
593). Additionally, 89.3% of industry leaders view enhanced digital media fan experiences as a top
opportunity to increase revenues (Source: PwC Sports Survey). The survey also reveals that 82.1% support
transforming the media landscape as a key market force in the sports industry (Source: PwC).
One of the best ways to enhance the fan experience is using AI with other digital technologies, AI-based
chatbots have shown promise in improving fan engagement, with about 87% of people enjoying their use
(Source: AI Journal).

3.2 Who Needs It

Young adults who want more personalized and engaging content during live broadcasts are the most
potential buyers. Besides this, the solution can be used by sports analysts and sports journalists. According
to PwC's fan behavior fact book:

Age:9-24 Age:25-40 Age:41-56 Age:57-75

Generation Millennials Generation Boomers
 Z X
Overall level of 60% 55% 49% 47%
interest in sport
(global)
Willingness to pay for 44% 40% 35% 22%
sports content
(percentage of fans)
Share of time- 35% 44% 50% 50%
consuming sport, Live

4 Project Overview and Goals

The main goal of the project is to create a solution that enhances the user experience for young adults during
the broadcast of games. This solution provides a mobile application chatbot that allows users to interact
using natural language, similar to any other chatbot. The chatbot can respond with the help of a Large
Language Model (LLM) when the required answer is in text form.
Additionally, users can augment the data on the screen by tracking players or fetching and displaying
relevant stats through embedded visualization or Augmented Reality (AR). This can be done by interacting
with provided templates in the form of a menu. An OTT streaming service will also be available, where
users can view the broadcast with the augmented data.
The key difference this project aims to achieve is better real-time performance while tracking and
augmenting real-time data. Unlike existing solutions, this platform is designed to deliver real-time results by
fine-tuning models on benchmark datasets and applying real-time data processing.

4.1 Value Proposition

4.1.1 Real-Time Performance:
The platform ensures that visual augmentations and data overlays are synchronized with the live action,
avoiding delays that are common in existing solutions.

4.1.2 Clean and User-Friendly Interface:

A customizable and intuitive interface for real-time interaction during live broadcasts.

4.1.3 Personalized Viewing Experience:

Fans can watch their games in a personalized environment with customizable options.

4.2 Final Project Output:

The final project will consist of a web application and a mobile application, both integrated with Computer
Vision and LLM technologies in the backend, along with other relevant modules such as visualization and
real-time data processing.
Hardware Components: The system will rely primarily on server-side processing, requiring a robust
backend server to handle video processing, data analysis, and API management. Users will need standard
devices like smartphones or computers to access the platform.
Software Components: The solution will include modules for real-time data processing (using technologies
like OpenCV and deep learning models), a web-based UI (using [Link] or similar frameworks), and a
mobile chatbot application (built with React Native).

5 Project Development Methodology:

5.1 Objectives and Modules
Objective 1: Select and Fine Tune a model for real time Detection and MOT
Deliverables: A fully functional MOT system that processes live sports feeds and outputs player positions
and movements along with Perspective Transformation.

Objective 2: Select and Fine tune a LLM for chatbot : .

Deliverables: LLM model with Documentation showing results.

Objective 3: Build the Mobile Chatbot Application.

Deliverables: A chatbot app that allows users to query stats and receive data updates during the game.

Objective 4: Implement the logic for Real time Data processing

Deliverables: Real time Data Processing Module with documentation.

Objective 5: Implement the Data Visualization and Augmentation

Deliverables: Data Visualization and Augmentation module after testing with documentation

Objective 6: Developing and deploying a platform to display augmented videos and integrating it with
backend
Deliverables: Complete Deployed Product with Documentation

5.1.2 System-Level Block Diagram

6. Project Milestones and Deliverables
6.1 Initiation and Planning:
Initiation
Define the main idea for the project, highlighting how it will overcome the issues in sports analysis.
Improve the project concept to ensure clarity and feasibility.

Develop the Problem Statement and Project Plan

Examine the issue this project aims to address and create a detailed plan to achieve the project’s goals. Make
sure the plan covers all critical elements of the project.

Gather the Requirements

Gather and write down the requirements to understand the user’s needs and expectations. Use the
information to direct the project's development.

Deliverables:
Project Proposal Draft, SRS, and Project Management doc, which outline the project’s goals, requirements,
and schedule.
6.2 System Architecture and Design
Design System Architecture
Create a comprehensive structure of the system. Define how this architecture will interact to support the
project’s goals.

Design Detailed Components

Create detailed specifications for each system component, including interfaces and interactions. Ensure that
each component is designed to meet specific project requirements.

Create Architectural Diagrams

Develop diagrams to visualize the system’s design, including system architecture, component interactions,
deployment setup, and activity flows.

Develop High-Level Design and Architecture

Map out a comprehensive high-level design that outlines system modules, responsibilities, and interactions.
Give an overview that includes design patterns, scalability strategies, and other considerations.

Deliverables:
System Design Specification (SDS) Document, which includes all details of the system’s architecture,
components, and design.

6.3 Computer Vision Model Development

Collecting Datasets
Using existing Datasets such as SOCCERNET ISSIA ROBOFLOX

Frame Augmentation
Use data augmentation techniques to enhance the dataset with different transformations like rotation and
scaling etc. This helps improve the model’s performance and generalization.

Design and Implement Model Architecture

Develop and implement the deep learning model architecture tailored for sports analysis which gives
effective detection and tracking.

Model Training and Optimization

Use the data which is processed and train the model and evaluate the efficiency by performance metrics.
Optimize the model by finetuning the hyperparameter for better efficiency.

Apply Tracking Algorithm

Tracking algorithms like deep-sort or byte-track etc., to monitor and track objects across frames. Check the
algorithm works perfectly with the detection model for accurate tracking.
Deliverables:
The Computer Vision Model which is implementing the Sport Analysis of Football videos including the
performance metric of the model.

6.4 Chatbot Development and Integration

Design Chatbot Architecture and Conversation Flow
Determine the chatbot’s interaction and design its conversation flow to manage different user interactions
efficiently. Develop the architecture that supports the chatbot’s functionality.

Train Large Language Model (LLM)

Train an LLM model to generate accurate and relevant responses. Adapt the model to address queries related
to sports analysis.

Implement Core Chatbot Functionality

Develop and integrate core features of the chatbot. Ensure these features work together to provide effective
user interaction.

Test and Debug the Chatbot

Conduct thorough testing of the chatbot’s functionality to identify and resolve any issues. Optimize and
debug to ensure smooth and accurate performance.

Integrate it with Computer Vision Model

Combine the chatbot with the computer vision model to make interactive sports analysis of Football. Ensure
it works correctly.

Deliverables:
The chatbot with the model integration will give the interactive model of the project.

6.5 Testing and Deployment

Unit Testing
Perform unit testing on the various functions of the model to verify that each function operates correctly on
its own.

Integration Testing
Conduct testing on different parts of the system so that the integrated components work according to the
requirements.

System Testing
Direct the testing of the complete system to ensure all components work together effectively. Check
functionality, performance, and reliability of the system.

User Acceptance and Performance Testing

Perform user acceptance testing to check it meets the users’ needs and expectation and check the system
reliability and robustness using performance testing.

Prepare and Deploy the System

Set up the deployment of the complete model including the chatbot. Deploy the system and perform initial
monitoring to ensure successful operation.

Deliverables:
Testing report and the Final Detailed Report which encompasses the complete project.

7. Tools, Technologies, and Method

7.1 Tools & Technologies:
Programming Languages:
Python for Training and Fine-Tuning Models, JavaScript/TypeScript for Web and mobile products

Frameworks:
[Link] for the web application, React-native for the mobile app

Libraries:
[Link] or [Link] for data visualization along with other AR technologies

APIs:
Third Party Apis for real time data fetching along with websockets

Datasets :
SOCCERNET, ISSIA, ROBOFLOW for player and football detection

8. Overall Description:
8.1 Product Perspective
The Bal wijzer is a self-contained system designed to perform real-time football match analysis using deep
learning, computer vision, chatbot interaction, and augmented reality (AR). It integrates a computer vision
model with a chatbot powered by a large language model (LLM), specifically tailored to analyze football
matches and interact with users. This product is intended to serve as a comprehensive solution for sports
analysis, targeting football matches with features like event detection (e.g., goals, fouls, tackles), player
tracking, and real-time match insights.
The system is composed of several components:
1. Computer Vision Subsystem: Uses YOLO for object detection and Bytetrack for tracking players
and the ball.
2. Event Detection Subsystem: Processes tracked data to identify key match events.
3. Chatbot Subsystem: Fine-tunes the LLM model to respond to football-related queries based on real-
time and historical data.
4. Augmented Reality Subsystem: Embeds interactive visual elements into video feeds for enhanced
viewing experiences.

8.2 Product Functions

The major functions which we are going to accomplish through our product are following:

Real-time Object Detection and Tracking:

Detect players, ball, and other relevant objects on the field.

Event Detection:
Identify key match events like goals, fouls, and offside situations based on tracked data.

Chatbot Interaction:
Provide users with information and statistics about the match, players, and teams through natural language
queries.

Augmented Reality Visualization:

Embed real-time visualizations such as player and team stats etc. on video content for an enchanting user
experience.

Data Integration and Retrieval:

Fetch real-time and historical match data from various football APIs for chatbot responses.

User Authentication:
The system will allow users to sign up or login to their accounts, ensuring privacy and proper profile
management.

Help and documentation:

A link will be provided allowing users to access any aid they need in using the system.

9. User Classes and Characteristics

The Users of our project are following:
9.1 Sports Analysts:
Individuals with a high level of expertise in football analysis. They will use the product frequently to gain
insights into matches and events, using both chatbot and AR features for advanced analysis.

9.2 Football Fans:

Football enthusiasts who want a richer match-watching experience. They may not use the product as
frequently but will interact with the chatbot to access stats and utilize the AR features for a more engaging
view.

9.3 Coaches and Teams:

Users who need to review match data for player performance analysis and strategical adjustments. They will
interact with the product functions, such as detailed event logs and player stats.

9.4 Technical Support Staff:

Individuals with a high level of technical expertise who will handle the maintenance, deployment, and
integration of the system components.

10. Operating Environment

The software will operate in the following environment:

10.1 Hardware Platform:

High-performance servers for processing video streams and running deep learning models. Mobile and web-
based platforms for AR and chatbot functionalities accessible to end-users.

10.2 Operating System:

Linux-based servers for deploying deep learning models and event detection systems, while Windows,
Android, and iOS platforms support client-side applications for AR and chatbot access.

10.3 Software Components:

Includes frameworks like YOLO (for object detection), Bytetrack (for tracking objects), [Link] (for web
application) and Flutter (for mobile application). The other AR technologies for embedded visualization.

10.4 Design and Implementation Constraints

The development of the system is subject to the following constraints:

Hardware Limitations:
Real-time processing requires high computational power and memory, limiting deployment to high-
performance server environments.
Interfaces to Other Applications:
The system will interface with football data APIs for real-time and historical data, and the integration of AR
features must be compatible with mobile/web platforms.

Parallel Operations:
The system must handle simultaneous processing of video feeds, chatbot interactions, and AR
visualizations, requiring efficient multithreading and message queuing mechanisms.

Security Considerations:
User data and system access will be secured with encryption and authentication measures, ensuring the
protection of sensitive information and preventing unauthorized access.

Design Standards:
The software will adhere to agile development practices and coding standards for consistency,
maintainability, and scalability.

11. User Documentation

The user documentation components that will be delivered along with the football analysis software include:

11.1 User Manuals

A comprehensive user guide covering installation, setup, and operation of the system, including detailed
sections for different user classes such as Sports Analysts, Casual Viewers, and Technical Support Staff.

11.2 Assumptions and Dependencies

The following assumptions and dependencies are relevant to the project:

11.2.1 Assumptions:
 The users will have access to devices capable of handling high computational requirements for AR
visualization and real-time data processing, such as modern smartphones, tablets, or computers with
GPUs.
 Users interacting with the chatbot will have stable internet connections to access real-time data and
historical match information.
 Third-party APIs providing real-time football data will remain operational and compatible with the
software throughout its use.
 The hardware and software platforms specified (e.g., Linux-based servers, Android/iOS devices) will
be available and supported throughout the development and deployment phases.
 The development team will have access to the necessary resources, such as GPU-enabled servers for
model training and real-time processing.
11.2.2 Dependencies
 The project depends on external APIs for accessing live and historical football data, which are
essential for chatbot functionalities. Changes or disruptions to these APIs could affect the product’s
performance.
 The use of YOLO and Bytetrack models requires specific dependencies. Version changes or
deprecations in these libraries could necessitate modifications in the codebase.
 The chatbot's reliance on fine-tuning the Llama model assumes the availability of training resources
and compatibility with deployed hardware.

12. External Interface Requirements

12.1 User Interfaces
A GUI will be made considering all the protocols of Product design, It will be made in the following manner
using figma:

12.1.1 Design Process

 Sitemap
 Sketch
 Wireframe
 Prototype

12.2 User Interfaces

12.2.1 Chatbot Interfaces
 Chatbot Interface: Where users interact with the bot.
 Splash Screen: Initial loading screen for the application.
 Loading Screen: Displays while data is being fetched or processed.
 Dashboard: Overview of all live matches and relevant statistics.
 Sign Up Page: User registration interface.
 Login Page: User authentication interface.

12.2.2 Web Portal Interfaces

 Home Page: Main landing page of the portal.
 About Us Page: Information about the product or company.
 Contact Page: Form for users to get in touch.
 Sign In Page: Alternative access for existing users.
 Registration Page: Detailed sign-up interface.
 Pricing Page: Information on subscription plans and costs.
 User Dashboard: Interface for users to manage personal data.
  Live Matches Menu Page: Displays current live matches.
 Streaming Page: Streaming interface with stats and data overlays.

12.2 Hardware Interfaces

The system interfaces with several hardware components to support its real-time and AR functionalities:

12.2.1 Servers:
High-performance servers with GPU support are required for processing computer vision tasks and
deploying deep learning models. The software communicates with servers to manage data processing and
ensure efficient performance.

12.2.2 User Devices:

The software supports interaction through mobile devices (Android and iOS) and desktop systems
(Windows and Linux). AR features require compatible hardware.

12.2.3 Display Devices:

The system interfaces with large display screens or monitors to provide an enhanced viewing experience,
allowing users to view AR-enhanced live feeds or historical analysis.

12.3 Software Interfaces

The system connects and integrates with multiple software components, as detailed below:

12.3.1 Operating Systems:

The product supports windows, Linux, android and iOS based all types of environments.

12.3.2 Databases:
The system interfaces with databases to store and retrieve match statistics, historical data, and user
preferences.

12.3.3 APIs:
The product integrates with third-party football data APIs to fetch real-time and historical match
information. These APIs provide data such as match scores, player stats, and event logs, which are used by
the chatbot.

12.3.4 Deep Learning Libraries:

The software uses YOLO for object and event detection and bytetrack for tracking the objects. It also
integrates with the LLM model for chatbot functionalities.
12.3.5 AR Frameworks:
Data like player positions, match statistics, and event detections are shared across components ensuring
consistency and real-time updates. The system ensures secure data exchange using encryption protocols for
sensitive data and user information.

12.4 Communications Interfaces

This system uses following communication interfaces:

12.4.1 HTTP/HTTPS Protocols:

The system uses HTTP/HTTPS for communication with third-party APIs and databases, ensuring secure
data retrieval and updates.

12.4.2 WebSocket:
For real-time communication between the server and client-side applications, such as delivering live match
data and AR visualizations. WebSocket provides continuous data transfer for high performance.

12.4.3 Security:
All communication between the software components, user devices, and servers will secure data transfer
and user privacy protection.

13. System Features

13.1 Sign in & Login
13.1.1 Description and Priority
This feature allows users to create an account or log in to the system, enabling them to access personalized
content like match history, video analyses, and real-time football updates.
Priority: High
Benefit: 8, Penalty: 9, Cost: 4, Risk: 2

13.1.2 Stimulus/Response Sequences

Stimulus: The user enters the application and navigates to the login/signup page.
Response:
 For login: The user inputs their credentials and is authenticated to access the system.
 For signup: The user fills in the necessary details (email, username, password) and an account is
created.

13.1.3 Functional Requirements

  REQ-1: The system shall provide a login form where the user inputs their username and password.
 REQ-2: The system shall provide a signup form requesting the username, email, and password,
enforcing strong password guidelines (minimum 8 characters, with numbers and special characters).
 REQ-3: The system shall encrypt passwords using secure hashing algorithms.
 REQ-4: The system shall allow users to reset their password using a secure recovery method.

13.2 General Football Information

13.2.1 Description and Priority
This feature allows users to ask football-related questions via a chatbot, which provides general information
such as team histories, player statistics, and football rules using an LLM.
Priority: Medium
Benefit: 7, Penalty: 6, Cost: 5, Risk: 3

13.2.2 Stimulus/Response Sequences

Stimulus: The user initiates a chat with the system and asks a football-related question.
Response: The system responds with the requested information based on its LLM knowledge.

13.2.3 Functional Requirements

 REQ-1: The system shall allow users to enter free-form football-related queries into the chatbot.
 REQ-2: The chatbot shall retrieve relevant football information and provide an accurate, context-
appropriate response.
 REQ-3: The system shall respond within 2-3 seconds to general information requests.

13.3 Real-Time Football Information

13.3.1 Description and Priority
This feature enables users to request real-time football match updates, player performance, and key match
events via a chatbot integrated with external football data APIs.
Priority: Very High
Benefit: 9, Penalty: 10, Cost: 6, Risk: 4

13.3.2 Stimulus/Response Sequences

Stimulus: The user asks the chatbot for real-time updates on a match or player.
Response: The system fetches the requested data from an external API and provides real-time match
updates.

13.3.3 Functional Requirements

 REQ-1: The system shall be integrated with live football data APIs to fetch real-time scores, events,
and player performance data.
  REQ-2: The system shall process the API data and deliver responses to user queries within 2
seconds.
 REQ-3: The system shall handle errors such as API downtime by providing fallback responses or
retry mechanisms.

13.4 Real-Time Video Analysis with Computer Vision

13.4.1 Description and Priority
This feature allows users to upload or stream football match videos, where the system performs real-time
player and ball tracking, providing key match insights using MOT models.
Priority: Very High
Benefit: 10, Penalty: 9, Cost: 8, Risk: 5

13.4.2 Stimulus/Response Sequences

Stimulus: The user uploads a football match video or selects a live stream.
Response: The system processes the video in real time, displaying player positions, movements, and key
match events.

13.4.3 Functional Requirements

 REQ-1: The system shall accept video uploads or live streams for real-time analysis.
 REQ-2: The system shall use multi-object tracking (MOT) models to track players and the ball in
real time.
 REQ-3: The system shall overlay relevant statistics like player heatmaps and key events during live
matches.

13.5 Data Visualization and Augmentation

13.5.1 Description and Priority
This feature offers users various data visualizations, such as player heatmaps, shot trajectories, and team
formations, overlaid on live matches or match recordings.
Priority: High
Benefit: 8, Penalty: 7, Cost: 6, Risk: 3

13.5.2 Stimulus/Response Sequences

Stimulus: The user selects an option to view a data visualization while watching a match.
Response: The system displays the chosen data visualization over the match video.
13.5.3 Functional Requirements
 REQ-1: The system shall generate and overlay visualizations like player heatmaps, shot trajectories,
and team formations.
 REQ-2: The system shall ensure smooth rendering of visualizations with no visible lag.

13.6 Computer Vision and LLM Integration for Video Understanding

13.6.1 Description and Priority
This feature integrates the computer vision model with the LLM, allowing users to query the chatbot about
match events based on real-time video analysis.
Priority: Very High
Benefit: 9, Penalty: 10, Cost: 7, Risk: 4

13.6.2 Stimulus/Response Sequences

Stimulus: The user asks the chatbot specific questions regarding the ongoing match video, such as “Who
scored the last goal?”
Response: The system processes the query, retrieves relevant video data, and responds to the user with
accurate information.

13.6.3 Functional Requirements

 REQ-1: The system shall enable users to ask specific video-based queries like “Which player passed
the ball?” or “What’s the current team formation?”
 REQ-2: The system shall combine video analysis data with LLM processing to answer user queries
within 5 seconds.
 REQ-3: The system shall overlay visual highlights, such as player positions or key events, when
responding to video-based queries.

14. Non-Functional Requirements

14.1 Performance Requirements
 The system must respond to general user queries within 1000 milliseconds.
 Real-time video analysis should not exceed 10 seconds of delay.
 API responses for real-time football match updates must be retrieved within 5 seconds.
 Data visualizations must render within 5 seconds after selection.

14.2 Safety Requirements

 User data, such as match videos, must be securely stored, and only used for processing purposes.
 The system must maintain data integrity and provide accurate information based on video analysis
and football data.
  Users should be informed if their data is collected for further usage or shared with third-party
services.

14.3 Security Requirements

 Strong user authentication measures must be in place, requiring passwords with a minimum of 8
characters and complexity requirements.
 All user data, including videos and credentials, must be encrypted using industry-standard
encryption algorithms.
 Secure communication with external APIs and servers using encrypted channels (e.g., HTTPS).

14.4 Software Quality Attributes

 Correctness: The system must ensure accuracy in football data and video analysis, with an F1-
score of at least 0.80 for tracking models.
 Portability: The system should be accessible via web and mobile platforms with consistent
performance.
 Scalability: The system must handle multiple concurrent users without performance degradation.
 Reliability: The system must ensure at least 95% uptime during football matches and high-traffic
events.
 Usability: The UI should be intuitive, allowing non-technical users to interact with the chatbot and
visualizations with ease.

14.5 Business Rules

 Only authenticated users can upload match videos or request real-time video analysis.
 Users are restricted from accessing each other's data unless explicit permission is given (data privacy
enforcement).
 Admin-level users will have privileges to manage external API connections and system updates.

15. Other Requirements

The following business rules are to be implemented by the system:

15.1 User Privacy:

Data specific to each user, such as uploaded videos, match history, and personalized analyses, must only be
accessible by that user. This must be enforced using a secure authentication and authorization system.
Any shared data, such as highlights or stats, must require explicit user consent if they involve user-generated
content.

15.2 Overloading:
If the system experiences an overload (e.g., too many concurrent users), users should be informed and
prompted to try again or switch to a low-latency mode where detailed analyses are deferred until server
capacity is available.

16. Design Methodology and Software Process Model

16.1 Design Methodology -- Mixed Approach: MVC with OOP and PP
In our project, we are adopting a mixed approach using the MVC (Model-View-Controller) design pattern,
which combines the strengths of both Object-Oriented Programming (OOP) principles and Procedural
Programming (PP). This approach ensures modularity, flexibility, and efficiency for our chatbot and video
analysis system.

16.1.1Separation of Concerns (SoC):

The MVC design pattern divides the system into three components:
 Model: Handles data management, storage, and business logic. This will embody OOP principles
such as encapsulation, abstraction, and inheritance. For example, classes will be used to model
entities like "Player," "Team," or "Match."
 View: Manages the user interface, which will render dynamic data received through APIs. This is
implemented on the front-end using frameworks like [Link] for web applications and Flutter/React
Native for mobile.
 Controller: Acts as the intermediary, managing input, processing business logic, and connecting the
Model and View. This component will flexibly incorporate both OOP for complex operations and PP
for simpler, sequential tasks.

16.1.2 Flexibility and Adaptability:

OOP principles (e.g., encapsulation and inheritance) will be used for the Model to manage entities and
relationships efficiently.
PP concepts will simplify straightforward tasks such as API calls, server-side scripts, and specific business
logic in the Controller, ensuring efficiency for real-time operations.
This hybrid approach allows the use of the most suitable paradigm for each task.

16.1.3 Enhanced Modularity and Reusability:

Using classes and objects in the Model promotes modularity and reusability of components like player stats,
match data, and analysis logic.
Simpler, reusable functions in PP style can handle tasks like data parsing, API requests, and basic
calculations, reducing complexity for specific workflows.

16.1.4 Compatibility with [Link] and System Scalability:

The [Link] framework aligns seamlessly with MVC principles for structured development of server-side
and front-end operations.
By keeping the Model independent of the View and Controller, the application becomes easier to maintain,
test, and scale as requirements evolve.

16.1.5 Problem-Specific Solutions:

For computationally intensive processes like video analysis and player tracking, OOP will encapsulate
algorithms and their parameters within structured classes.
For tasks like API integration, data transformation, and user input validation, PP will provide clear,
sequential logic, ensuring optimal execution.
Summary of Benefits:

Key Benefit Explanation

Separation of Concerns Clear division between Model, View, and Controller improves maintainability
and scalability.
Flexibility and Combines OOP for data complexity and PP for simpler workflows, choosing
Adaptability the best-suited paradigm.
Enhanced Modularity Promote code reuse through objects, classes, and reusable functions.
Efficiency Simplifies real-time API calls, server-side tasks, and UI rendering for a
responsive system.
Table 1: Summary of Benefits

16.2 Software Process Model — Agile

In alignment with the principles of adaptive and iterative development, the "Bal Wijzer" project adopts the
Agile Process Model. This deliberate choice reflects the methodology’s focus on collaboration, flexibility,
and continuous feedback, aligning with the dynamic requirements of the project.
Agile sprint-based approach facilitates iterative development, breaking the project into manageable
increments that focus on delivering core features such as chatbot interfaces, LLM integration, and YOLO
model enhancements. This iterative process ensures measurable progress while accommodating changes in
APIs, datasets, or system requirements without disrupting the overall workflow.
Regular sprint reviews foster collaboration among team members, enabling the identification and resolution
of challenges while incorporating feedback from stakeholders. Continuous testing throughout the
development cycle ensures the functionality and reliability of each module, contributing to a seamless and
efficient development journey.
By adopting the Agile Process Model, the project benefits from adaptability, structured development cycles,
and a proactive approach to challenges, ensuring steady progression toward delivering a high-quality
solution.
17. System Overview
The project is a cross-platform football analytics system combining LLM-based chatbot functionality, real-
time match streaming, and advanced visualizations. Built using [Link] for accessibility, the chatbot utilizes
LLM for natural language responses and Tivaly for web searches, integrated with Rapid-API for live
football data. Fine-tuned YOLO models detect and track players and events like goals or fouls, enabling
real-time visualizations, while WebGL adds immersive 3D match visualizations for an enhanced user
experience.

17.1 Architectural Design Chatbot Interface:

The Chatbot is a cross-platform application built using [Link], ensuring accessibility on both mobile and
PC web platforms. The chatbot leverages ChatGPT 4.0 for generating general responses and Tivaly for web
searches. The integration with the Football API from RapidAPI ensures real-time live data during football
matches. The chatbot interface allows users to interact with the system and receive updates, insights, and
answers in natural language.

17.1.1 Detection and Tracking

The chatbot is seamlessly integrated with another web application responsible for streaming live football
matches. This interface not only enables users to watch the game but also incorporates real- time
visualizations based on data received from fine-tuned YOLO models. These models are used for detecting
and tracking players, as well as event detection (fouls, goals, etc.), which trigger real- time visualizations on
the match.

17.1.2 WebGL Integration for 3D Visualizations:

A separate module is implemented using WebGL, allowing for the integration of 3D visualizations within
the football match stream. This feature enhances the user experience by providing immersive and interactive
visualizations, offering a deeper understanding of the match and events in real-time.

17.1.3 Architecture diagram

Our system consists of two microservices: a Chatbot and a Detection and Tracking service. The Chatbot is
built with [Link], utilizing ChatGPT 4.0 and Tivaly for real-time responses and football data integration.
The Detection and Tracking service streams live football matches, using fine-tuned YOLO models for
player detection and event tracking, enhanced with real-time 3D visualizations via WebGL. Both services
follow a layered architecture for modularity and maintainability.
18. Design Models
Design models are given below:

Class Diagram:
Sequence
Diagrams:
Chatbot Tracking Sequence Diagram

Chatbot-Live Info. Sequence Diagram

State Transition Diagram
Activity Diagrams:
Data Flow Diagrams:
19. Data Design
We have two separate databases:
One for storing user data and its relevant information, and the other is provided by Football-API to
fetch real-time data during football games.
We are using Bunny-CDN for storing images such as logos and player/team pictures for fast
processing.
In addition, [Link] is also being used for video streaming purposes.
ERD:
NoSQL Data Format:
User Info:

. User Information
{
"user": {
"user_id": "12345",
"email": "user@[Link]",
"password": "hashed_password",
"name": "John Doe",
"age": 30,
"gender":
"Male",
"favorite_team": "Manchester United",
"interactions": [
{
"interaction_id": "abc123",
"message": "Who won the last
match?",
"response": "Manchester United won 3-1.",
"timestamp": "2024-12-17T[Link]Z"
}
]
}
}

Live Event Detection:

{
"live_event":
{ "event_id":
"event123", "type":
"Goal",
"description": "Rashford scored a
goal", "timestamp": "2024-12-
17T[Link]Z",
"video_url": "[Link]
}
}

Data Dictionary
System Data Dictionary for Authentication
Purpose: Authentication for the football chatbot and stream site.

Field Type Description

id uuid Unique identifier for the user/admin
 username varchar Unique username for the user
email varchar Email address for user authentication
password varchar Encrypted password for authentication
token string JWT or session token generated upon successful login
refresh_token string Token used to refresh expired access tokens
created_at datetime Timestamp when the user account was created
updated_at datetime Timestamp of the last update to the user account
last_login datetime Timestamp of the user's last successful login
role varchar Role of the user (e.g., "admin", "user")
is_active boolean Indicates if the user account is active
Table 2: System Data Dictionary for Authentication

System Data Dictionary for Live Match Data Endpoint

Endpoint: GET/odds/live

Description: Retrieves live match odds and status for a specific fixture.

General Response Fields

Field Type Description

get string The endpoint called (e.g., "odds/live")
[Link] string Fixture ID for which live odds are requested
errors array List of errors (if any)
results integer Number of results returned
[Link] integer Current page number
[Link] integer Total number of pages
Table 3: General Response Fields

Response Object Fields

Field Type Description

[Link] integer Unique ID of the fixture
[Link] string Descriptive match status (e.g., "Second Half")
[Link] integer Minutes elapsed in the match
[Link] string Seconds elapsed in the match in mm:ss format
[Link] integer Unique ID of the league
[Link] integer Current season year for the league
[Link] object Details of the home team (currently empty placeholder)
[Link] object Details of the away team (currently empty placeholder)
[Link] boolean Indicates if the match is temporarily stopped
[Link] boolean Indicates if the match is blocked
[Link] boolean Indicates if the match has finished
update datetime Timestamp of the last data update in ISO 8601 format
Table 4: Response Object Fields

Odds Array (Nested Fields)

 Field Type Description
[Link] integer Unique ID for the odds entry
[Link] string Name of the odds market (e.g., "Match Corners")
[Link] array Values associated with the odds (currently empty)
Table 5: Odds Array

Example Data

Field Value
get "odds/live"
[Link] "721238"
errors []
results 1
[Link] 1
[Link] 1
[Link] 721238
[Link] "Second Half"
[Link] 62
[Link] "62:14"
[Link] 30
[Link] 2022
[Link] {}
[Link] {}
[Link] false
[Link] false
[Link] false
update "2022-01-27T[Link]+00:00"
odds[0].id 20
odds[0].name "Match Corners"
odds[0].values []

Table 6: Example Data

20. User Interface Design

Below is the system’s functionality from the user’s perspective:

20.1 Login and Authentication

Supports login via credentials (email/password) or third-party OAuth (Google, Discord).
Simple registration for new users with quick account creation options.

20.2 Home Screen

Displays current football news and live/upcoming matches with key details.
Direct navigation to the chatbot UI for detailed match insights.

20.3 Chatbot Interaction

 20.3.1 Multi-Modal Interface:
Text-based, speech input, and theme customization (light/dark mode).

20.3.2 Core Features:

 General football information (team stats, standings, player performance).
 Live match tracking (goals, possession stats, real-time updates).
 Real-time event detection (goals, fouls) with video highlights powered by [Link].

20.3.3 Game Streaming and Visualization

 Watch live games via integrated [Link] streams.

 Visual overlays for player stats, tactical insights, and match progress updates.
 Receive notifications for live events with graphical highlights like heatmaps and event replays.

20.3.4 User Profile and Accessibility

 Profile Management: Update details, manage security, and customize chatbot settings.
 Accessibility Features: Speech interaction, cross-platform availability, and customizable
themes.

20.3.5 User Flow

Log in → Browse news/matches → Interact with chatbot → Watch live streams with analytics
→ Manage profile settings.
20.4 Screen Images

This is Splash Screen of Mobile View

Page Before Authentication
Login Page
Sign-up page
Chatbot Page
Chatbot page theme 2
Profile update page.
Home Page after Auth
Home page of streaming site

Video before overlay

 Video after stats overlay

Screen Objects and Actions

This part of chatbot helps user to add query Via text or Speech to text User can also add
emoji showing inclusiveness.

This is header of the chatbot it includes a menu, and a theme change button for better readability.
The speaker emoji in this section helps to receive the info via voice.

This menu allows users to navigate home or log out the session.
This part of the homepage gives the latest football related news to the user.

In the login page user can login through their social media accounts with these buttons.
21. Project Management Tool:
22. Testing:
22.1 Chatbot Backend Testing:
22.1.1 Test Categories
 Integration Tests: End-to-end testing of API endpoints and chatbot flows
 Unit Tests: Individual component testing
 API Tests: Football API integration testing
 Handler Tests: Core functionality testing

22.1.2 Test Environment:

# Environment Requirements

Python Version: 3.13.1

Pytest Version: 8.3.5

Required Packages:

- pytest-asyncio

- pytest-cov

- pytest-mock

22.1.3 Test Implementation

[Link] Integration Tests
Location: tests/integration/

API Endpoints (test_api_endpoints.py)

# Key Test Cases

1. Root Endpoint

- Verifies API status

- Checks environment variables

- Validates welcome message

2. Chatbot Endpoint
 - General Mode: Basic query handling

- Search Mode: Information retrieval

- Live Mode: Real-time match data

- Fallback: Search mode when uncertain

3. Live Chat Endpoint

- Match-specific queries

- Real-time data processing

- Visualization support

Chatbot Flow (test_chatbot_flow.py)

# Test Scenarios

1. Live Match Query Flow

- Initial live match detection

- Specific match queries

- Match statistics visualization

2. Multi-mode Conversation

- Mode switching

- Context preservation

- Response consistency

[Link] Unit Tests

Location: tests/unit/

API Football (test_api_football.py)

1. Live Matches

- Successful retrieval

- No matches scenario

- API error handling

- Rate limit handling

2. Live Fixtures

- Successful retrieval

- Not found handling

- Request error handling

ChatGPT Handler (test_chatgpt_handler.py)

# Test Cases

1. Response Generation

- Valid input processing

- Concise response flag

- Structured data handling

- Memory management

- Error handling

Tavily Search (test_tavily_search.py)

# Test Cases

1. Search Functionality

- Successful search

- Empty results

- Tool errors

- Exception handling

Uncertainty Detection (test_uncertainty_detection.py)

# Test Cases

1. Detection Scenarios

- Obvious uncertainty phrases

- Confident responses

- Redirect phrases

- Error handling
22.1.4 Test Data
[Link] Mock Data

# Example Mock Data Structure

mock_match_data = {

"league": "Premier League",

"country": "England",

"status": "In Play",

"home_team": "Team A",

"away_team": "Team B",

"home_score": 1,

"away_score": 0,

"elapsed_time": 45

[Link] Test Queries

# Standard Test Queries

test_queries = [

"Who won the recent match?",

"Show me live matches",

"What's the score?",

"Who scored the last goal?"

22.1.5 Running Tests

[Link] Basic Test Execution

# Run all tests

pytest tests/

# Run specific test category

pytest tests/integration/

pytest tests/unit/

# Run with verbose output

pytest -v tests/

[Link] Test Coverage

# Generate coverage report

pytest --cov=chatbot tests/

22.1.6 Known Issues and Limitations

[Link] Current Warnings
1. Pytest Configuration
 Issue: Unset asyncio_default_fixture_loop_scope
 Impact: Future compatibility
 Resolution: Set explicit scope
2. Pydantic Deprecation
 Issue: Missing type_params
 Impact: Python 3.15 compatibility
 Resolution: Update type parameter handling

22.1.7 Output:
22.2 Website Backend Testing:
22.2.2 Test Categories

 Integration Tests:
Validates API endpoints with real or in-memory data layers (e.g., MongoMemoryServer).
  Unit Tests:
Verifies business logic, response structure, and service behavior in isolation using
mocking.

 API Handler Tests:

Directly tests request handlers (e.g., POST, PUT, GET, DELETE) for input/output
behavior.

22.2.2 Test Environment

 Framework: Jest

 Mocking Tools: [Link], custom mocks

 In-Memory DB: MongoMemoryServer for integration

 Mocked Dependencies:

o Mongoose models

o JWT

o Bcrypt

o Email transporter

22.2.3 Test Implementation:

Route / Endpoint Test Case Setup Result

/api/check- In-memory DB, no matching 200 OK – Username

Unique username
username-unique user available

Invalid format 200 OK – Still treated

Contains invalid characters
username as available

DB query failure Simulate [Link] 500 Error – "Error

Route / Endpoint Test Case Setup Result

throwing error checking username"

[Link]() throws 500 Error – JSON

Invalid JSON body
SyntaxError parse failure

/api/check- 409 Conflict –

Verified user exists [Link] returns user
username-unique Username taken

Username is 200 OK – Username

[Link] returns null
available available

No Authorization
/api/delete-user Missing Authorization header 401 Unauthorized
header

Invalid
Authorization Authorization: Basic token 401 Unauthorized
format

Valid token, user 404 Not Found – User

findByIdAndDelete returns null
not found not found

204 No Content +
/api/delete-user CORS preflight No auth required
CORS headers

400 Invalid –
/api/login User not found findOne returns null
Email/user not found

User found with isVerified: 400 Invalid – User not

User not verified
false verified

400 Invalid – Wrong

Invalid password [Link] returns false
password
Route / Endpoint Test Case Setup Result

200 OK – Returns JWT

Valid credentials All mocks pass, JWT generated
token

204 No Content +
/api/login CORS preflight Valid request
CORS headers

/api/reset- Invalid or expired 400 Error – Invalid or

[Link] returns null
password token expired token

/api/profile- Valid JWT, valid profilePhoto, 200 OK – Updated user

Successful update
picture update succeeds returned

Missing Bearer
Invalid header format 401 Unauthorized
token

Invalid JWT [Link] throws error 401 Invalid token

Missing 400 Profile photo

Empty JSON body
profilePhoto required

User not found findByIdAndUpdate returns

404 Not Found
during update null

findByIdAndUpdate throws
DB update error 500 Internal error
error

/api/profile- Retrieve existing 200 OK – Returns

User found with profile photo
picture photo profilePhoto

200 OK – profilePhoto:
Retrieve null photo User found, no profilePhoto
null

Invalid auth header Not starting with Bearer 401 Unauthorized

Route / Endpoint Test Case Setup Result

Invalid JWT [Link] throws 401 Invalid token

User not found

findById returns null 404 Not Found
during retrieval

500 Internal server

DB retrieval error findById throws error
error

New user 201 Created –

/api/signup Valid data, email sent
registration Verification email sent

Email already 400 Conflict – Email

findOne returns verified user
registered already taken

Unverified user 201 Created – Email

Updates user & sends email
exists resent

500 Error – Failed to

Email send fails sendVerificationEmail fails
send email

500 Error – User

JSON parse error [Link]() throws error
registration failed

22.2.4 Output
Coverage:
22.3 Frontend Testing

22.4 Mobile Testing

22.5 Non-Functional Testing
22.5.2 Website
The Summary Report is as follows:

The Aggregate Report is as follows:

The Aggregate Graph is as follows:

The Response Time Graph is as follows:

22.5.2 Chatbot:

The Summary Report is as follows:

The Aggregate Report is as follows:

The Aggregate Graph is as follows:

The Response Time Graph is as follows:

23. Deployment
Each component has been strategically deployed across modern cloud platforms to optimize for
performance, scalability, and ease of maintenance.

1. Mobile Application
 Framework: React Native
 Deployment Target: iOS and Android devices
 Purpose: Provides users with a mobile interface to access core features of the application
including chat functionality.
 Deployment Channel: Distributed via Apple App Store and Google Play Store (or side-
loaded during development/testing phases).
  CI/CD (if applicable): Integrated with services like Expo, EAS, or Fastlane for
automated builds and OTA (Over-The-Air) updates.

2. Chatbot Backend
 Platform: Railway
 Purpose: Hosts the backend logic and infrastructure for the AI chatbot that powers both
the mobile app and the web interface.
 Features:
o API endpoints for chatbot interaction
o Integration with NLP/AI models
o Real-time or asynchronous messaging logic
 Benefits of Railway:
o Fast deployment with auto-scaling
o Integrated observability (logs, metrics)
o GitHub CI/CD integration for streamlined updates

3. Chat Application Backend (Mobile App & Website)

 Platform: Vercel
 Purpose: Acts as the common backend for both the mobile and web chat clients,
handling authentication, user management, and real-time chat services.
 Technologies: Likely uses serverless functions (e.g., [Link] API routes or edge
functions) for scalable, efficient backend logic.
 Endpoints: Secure REST or WebSocket APIs exposed to clients.

4. Website Frontend
 Platform: Vercel
 Framework: Likely built with [Link] for SSR (Server-Side Rendering) or SSG (Static
Site Generation).
 Purpose: Public-facing site allowing users to access the application via a browser.
 Features:
o User login and chat interface
o SEO-optimized landing pages
o Responsive and fast-loading UI
 Benefits of Vercel for Frontend:
o Blazing-fast CDN deployment
o Easy integration with GitHub for automatic redeployments
o Edge rendering for performance

Summary of Deployment Flow

Component Technology Platform Purpose
Mobile App React Native iOS & Native mobile access
Android to core app
Chatbot Backend Custom Node/Express or Railway AI/chat logic and NLP
 AI backend handling
Chat App Backend API Routes / Serverless Vercel User auth, messaging
(Mobile/Web) API
Web Frontend React / [Link] Vercel Public website and
chat UI

24. Conclusion and Future works

To further enhance the system's efficiency and user experience, future efforts will focus on
optimizing algorithms for improved real-time performance, particularly within the chatbot and
chat application layers. This includes reducing latency in message processing and response
generation to ensure seamless user interactions. Additionally, exploring and integrating more
powerful computational resources—such as higher-tier cloud instances or specialized hardware
accelerators—will enable the platform to scale more effectively and maintain high performance
under increased user load.

References:
 PwC Sports Survey. PwC Sports Survey 2021
 AI Journal. "How Do Customers Feel About AI Chatbots?"
 Andrews, Peter. "FootyVision." MediaFutures. PDF
 Zhi, Qiyu, Metoyer, Ronald. "GameBot: A Visualization-Augmented Chatbot for Sports
Games." CHI’20 Extended Abstracts.
 Fotovision. Fotovision Website
 RoboFlow. "Football Players Detection Dataset.".
 SoccerNet. "SoccerNet Dataset."
 Augmentation for Sports Analysis Paper