1
2
3
4
5
6
7
8
9
10
11
12
13
14
Lab Topics
Lab 1: Python Intro, setting up the environment, version control, code completion with AI
Lab 2: Web development 101. DOM, HTML, CSS, JS. Page organisation and styling
Lab 3: Reading and writing data. Common data formats, querieng databases (MongoDB, SQL)
Lab 4: Web scraping with Python (Beautiful Soup). Advanced scraping with Scrapy
Lab 5: Setting up local LLM server. Data parsing through LLM agents
Lab 6: Data Cleaning & Preprocessing: Introduction to NumPy and pandas. Cleaning data with pand
Midterm exam
Lab 8: Visualizing data with Matplotlib, Seaborn, ProPlot, Bokeh and Plotly
Lab 9: Delivering data with Flask
Lab 10: Visualizing data with Three.JS
Lab 11: Visualizing data with D3.js
Lab 12: Building a complete visualization project
Flipped class. Presentation of projects
Final exam
Concepts covered in Lectures and Labs:
1. Lab 1: Python Intro, Setting up the Environment, Version Control, Code Completion with AI, Testi
Python Basics:
Syntax and Structure: Variables, data types (integers, floats, strings, booleans, lists, tuples, diction
Modules and Packages: Importing and using standard libraries (e.g., math, random, os).
Error Handling: Understanding and handling exceptions (try/except blocks).
Basic Debugging: Simple print statements, using a debugger (if time allows).
Setting up the Environment:
Python Installation: Choosing a Python distribution (e.g., Anaconda, miniconda), installing Python o
Virtual Environments: Creating and managing virtual environments using venv or conda to isolate
Package Management: Using pip or conda to install, update, and manage libraries.
Text Editors and IDEs: Introduction to basic code editors like VSCode, Sublime Text, or PyCharm an
Version Control (Git):
Introduction to Version Control: Why use version control? Explain the purpose of Git and its benefit
Git Basics: Cloning repositories, adding files, committing changes, checking the status, viewing his
Branching and Merging: Creating branches, merging branches, resolving merge conflicts.
GitHub/GitLab/Bitbucket: How to use these platforms, creating repositories, pushing changes, pull r
Basic Workflows: Teach a basic branching model, using main and develop branches.
Code Completion with AI:
Introduction to Code Completion tools: Explain what they are and why they are helpful.
Tools in the chosen IDE: Explain how to use and configure the code completion tools, setup self-ho
How to utilise the tools: How to use code completion to write code more efficiently.
Limitations: Explain how to not rely solely on the code completion tools, when to use and when not
Introduction to Testing:
Why testing is important: the importance of testing in software development and data analysis.
Basic Types of tests: Introduction to unit tests, integration test and sanity checks.
Using pytest: basic usage of pytest for simple testing.
2. Lab 2: Web Development 101. DOM, HTML, CSS, JS. Page Organisation and Styling
HTML (Structure):
Basic Tags: <html>, <head>, <body>, headings (h1 to h6), paragraphs (<p>), links (<a>), image
Semantic HTML: Using tags like <nav>, <article>, <aside>, <footer> for better structure and acc
HTML Attributes: class, id, src, href, alt, placeholder, etc.
HTML Forms: Creating forms with different input types (text, password, checkbox, radio buttons, su
CSS (Styling):
Selectors: Tag selectors, class selectors, ID selectors, attribute selectors, pseudo-classes, pseudo-e
Properties: color, font-size, margin, padding, border, background, display, position, float, etc.
Box Model: Understanding content, padding, border, and margin.
Layout: Techniques for layout using Flexbox and CSS Grid.
CSS Frameworks: Basic introduction to CSS frameworks like Bootstrap (or other similar framework)
Responsive Design: Using media queries to make pages responsive for different screen sizes.
JavaScript (Interactivity):
Basic Syntax: Variables (let, const, var), data types, operators, control flow (if/else, for loops, while
DOM Manipulation: Selecting elements, changing content, modifying styles, event handling (e.g., o
Working with the console: debugging using the browser console.
Introduction to async and await: Making asynchronous API calls, working with promises.
DOM (Document Object Model):
Understanding the DOM Tree: How HTML is represented as a tree structure.
Accessing DOM Elements: Using JavaScript to interact with the DOM (e.g., querySelector, querySele
Page Organisation and Styling:
Planning a Simple Layout: Creating a basic webpage with a navigation, header, content section, an
Responsive Design: Using CSS media queries to make pages responsive for different screen sizes.
Basic Design Principles: Introduction to basic visual design principles, hierarchy, spacing, color, fon
3. Lab 3: Reading and Writing Data. Common Data Formats, Querying Databases (Mong
Common Data Formats:
CSV (Comma Separated Values): Reading and writing CSV files using Python's csv module or panda
JSON (JavaScript Object Notation): Reading and writing JSON data using Python's json module, work
Other Formats (Optional): Introduction to other common formats like XML, YAML (if time permits), p
Reading and Writing Files:
File I/O: Opening files, reading from files, writing to files, closing files (using open(), read(), write(),
Error Handling: Handling potential file-related errors, and ensuring that the data is correctly format
Querying Databases:
SQL (Relational Databases):
Introduction to SQL: Basic concepts of databases, tables, rows, columns, schema.
SQL Queries: SELECT, FROM, WHERE, ORDER BY, GROUP BY, JOIN (inner, left, right, outer joins), HA
Connecting with Python: Using a library like sqlite3 or SQLAlchemy to connect to and query a SQL d
MongoDB (NoSQL Database):
Introduction to MongoDB: Basic concepts of documents, collections, embedding vs. referencing doc
CRUD Operations: Creating, reading, updating, deleting documents, filtering using find, and using u
Querying with MongoDB: Using the MongoDB query language, using $match, $project, $group, $so
Connecting with Python: Using pymongo to connect to and query a MongoDB database.
Introduction to APIs
What are APIs: Understanding how APIs work, including RESTful APIs.
Making API calls: How to use the requests library to make GET and POST requests to an API.
API responses: understanding the data format from API responses.
API Authentication: understanding how to use api keys or tokens.
4. Lab 4: Web Scraping with Python (Beautiful Soup), Advanced Scraping with Scrapy, A
Web Scraping Fundamentals:
Ethical Considerations: Respecting website terms of service, robots.txt, avoiding overloading serve
HTTP Requests: Making requests to web servers using requests library, handling different response
HTML Structure Review: How HTML is parsed and structured.
Web Scraping with Beautiful Soup:
Beautiful Soup: Parsing HTML and XML content, understanding different parsers.
Navigating the HTML tree: Finding elements using tags, attributes, CSS selectors, traversing DOM e
Extracting text and data: getting the data and saving it in a structured format, handling inconsisten
Advanced Scraping with Scrapy:
Introduction to Scrapy: Creating a Scrapy project, creating Scrapy spiders, working with item pipeli
Defining rules: Specifying which pages to scrape, and how to follow links using LinkExtractor, using
Extracting data: extracting data using CSS selectors, and xpaths, item loaders for structured data.
Data Pipelines: processing the scraped data, cleaning the data, saving it in a structured format, ex
Handling pagination and dynamic content: Strategies to scrape websites with multiple pages, and
Handling different data types, images, and other assets.
Deploying the scrapy spider and monitoring its status.
API Scraping:
Understanding the difference between API and Web Scraping why API scraping is easier and prefer
Scraping APIs: Using the same techniques as web scraping to get data from APIs, focusing on the r
API Pagination and Rate Limiting how to manage rate limiting and pagination when using APIs.
Rate Limiting and Avoiding Bans: Implementing strategies to avoid getting IP banned, user agents,
5. Lab 5: Setting up Local LLM Server. Data Parsing through LLM Agents
Introduction to LLMs:
What are LLMs: Explain the basic concepts, how they work, what are their uses and limitations, wh
Local LLM Server:
Choosing a LLM model: Choosing an open source model that can run locally, understanding the siz
Setting up the Environment: Installing the necessary libraries and frameworks (like llama.cpp, ollam
Running a Local LLM Server: Using Python to set up the server and interact with it, setting up the s
Parameter configuration: Explain how to configure parameters like temperature, top-p and others.
LLM Agents:
Introduction to Agents: Explain the concept of an agent, how to use LLMs with agents, using LangC
Prompt Engineering: Writing effective prompts for LLMs to perform specific tasks, zero-shot and few
Parsing with LLMs: Using LLMs to extract data from unstructured or semi-structured text, using cha
Examples of Data Parsing: Extracting entities, key-value pairs, summarizing text, data transformat
Error handling and debugging: How to fix errors when the LLMs does not behave as expected, how
6. Lab 6: Data Cleaning & Preprocessing: Introduction to NumPy and pandas. Cleaning d
Introduction to NumPy:
NumPy Arrays: Creating, manipulating, and indexing NumPy arrays, different types of arrays, multi
Basic Operations: Mathematical operations, broadcasting, slicing, reshaping.
Linear Algebra: Basic linear algebra operations if time permits, dot products, matrix multiplication,
Introduction to Pandas:
DataFrames: Creating, accessing, and modifying DataFrames, reading data from different files, diff
Series: Understanding Series and how they relate to DataFrames, working with index, renaming co
Indexing and Selecting Data: Different ways to access, filter and slice data, loc and iloc.
Data Cleaning with Pandas:
Handling Missing Values: Identifying and dealing with missing data (imputation with different strate
Data Type Conversions: Changing data types (e.g., string to integer, string to datetime), using pan
Duplicate Data: Finding and removing duplicate rows, dropping or keeping duplicate rows based on
Data Formatting: Cleaning up inconsistencies in text, dates, using regular expressions for text man
Data Validation: Checking if the data is in correct ranges, using assert statements, validating speci
Data Preprocessing with pandas:
Outlier Detection: Identifying and handling outliers using IQR, z-score, visualisations, and trimming
Data Transformation: Applying techniques like scaling (min-max normalization, standardization), lo
Exploring data (embeddings):
Introduction to Embeddings: Understanding what embeddings are, their uses, working with dense a
Generating Embeddings: Using libraries like SentenceTransformers, HuggingFace Transformers to
Basic analysis with embeddings: Applying techniques like cosine similarity to explore relationships
Visualizing the embeddings: Using dimensionality reduction techniques to visualize embeddings.
Basic Statistics and EDA
Descriptive Statistics: Calculating and interpreting descriptive statistics (mean, median, standard d
Distributions: Understanding different types of distributions (normal, uniform, skewed), and how to
Correlation: Measuring and interpreting the correlation between variables, using pearson, spearma
Relationships between variables: Exploring the relationship between categorical variables and num
Data distributions using histograms and kernel density plots.
Hypothesis testing (briefly): Introduction to basic hypothesis tests, significance and p-values.
7. Midterm Exam
Comprehensive assessment of concepts covered in Labs 1-6.
8. Lab 8: Visualizing data with Matplotlib, Seaborn, ProPlot, Bokeh and Plotly
Data Visualization Principles:
Choosing the Right Chart Type: Selecting appropriate visualizations for different types of data (bar
Effective Use of Color: Using color to highlight important information, selecting the right color map
Labels and Annotations: Adding clear labels, titles, legends, and annotations to visualizations, addi
Accessibility: Design for accessibility, colour contrast, using labels and captions, using alternative t
Matplotlib:
Basic Plotting: Creating plots, setting axis labels, titles, legends, saving figures, working with multip
Customization: Changing colors, markers, line styles, fonts, grids, annotations, and text, changing
Subplots and Figures: Creating multiple plots in the same figure, setting up layout, creating grids fo
Working with subplots, and axes objects: how to control the axes and the subplots.
Seaborn:
Statistical Plots: Creating distribution plots, relational plots, categorical plots, heatmaps, and pairpl
Theme Customization: Changing default styles, using seaborn themes, making custom themes.
ProPlot:
Introduction to ProPlot: how to use it and its benefits, setting up the environment.
Advanced Customization: Use the advanced features of ProPlot, using its easy API to generate publ
Bokeh:
Interactive Plots: Creating interactive charts and dashboards with zooming, panning, hovering, and
Bokeh Server: Deploying Bokeh applications, creating live applications.
Customisation: Customising bokeh figures, adding glyphs, annotations, and setting up layout.
Plotly:
Interactive Plots: Creating interactive plots, adding interaction using event handlers, zooming, pan
Customization: Using plotly for more interactive plots, changing the theme and look and feel of a p
Dashboards: Creating interactive dashboards with Plotly Dash (optional), building complex applicat
9. Lab 9: Delivering data with Flask
Introduction to Web Frameworks: What are frameworks and why are they used, the diff
Flask Basics:
Setting up a Flask application: Creating a basic Flask application, running it, and setting it up, work
Routes: Defining URL routes using decorators, working with variables in routes, using different HTT
Templates: Using Jinja2 templating to generate dynamic HTML content, passing data to templates,
Requests: Accessing data from HTTP requests (GET/POST), parsing url parameters, working with re
Responses: Returning JSON data, redirecting user, returning html and other types of data, managin
Delivering data with Flask
Returning JSON data: Returning data from the server, serialising python objects to json data.
Creating a Basic API endpoint: Creating an endpoint that can return data, creating multiple endpoin
Error Handling: Handling errors and returning appropriate HTTP responses, creating custom error p
Deploying Flask Applications: Basic overview of deploying Flask applications to a server, using wsg
Security: Introduction to securing Flask applications, avoiding common vulnerabilities, using sessio
Configuration: Configuring Flask application for different environments, setting up database connec
10. Lab 10: Visualizing data with Three.JS
Introduction to WebGL: What is WebGL and why it is used for 3D graphics, understandin
Three.js Basics:
Scene, Camera, and Renderer: Setting up the basic scene for 3D graphics, adjusting camera view,
3D Objects: Creating different geometries (cube, sphere, cone), materials (basic, mesh, phong), lig
Basic Transformations: Translating, rotating, and scaling objects, applying matrix operations for ad
Working with textures: Adding images to the models.
Data Visualization with Three.js:
Creating simple 3D charts: bar charts, line charts, scatter plots in a 3D space, using custom geome
Interactivity: Adding interactive elements using mouse events (hover, click, drag), keyboard events
Loading models: Loading external 3D models (OBJ, GLTF), using loaders.
Animations: Animating objects and changes in the scene, working with the animation loop and requ
Optimizing the performance: Rendering performance, working with LOD for complex scenes.
11. Lab 11: Visualizing data with D3.js
Introduction to D3.js:
Selection: Selecting HTML elements based on CSS selectors, selecting groups of elements.
Data Binding: Binding data to HTML elements, using enter, update and exit, understanding differen
Scales and Axes: Creating scales for mapping data to visual properties, linear, log, and ordinal scal
Creating different types of charts
Basic Chart Types: Creating bar charts, line charts, scatter plots, using svg for drawing, working wi
Transitions: Adding transitions and animations to data visualizations, using different types of transi
Interactivity: Adding interactive elements with mouse events and zooming, using event handlers, m
Layouts: Using layout generators (e.g., force-directed layout, tree layout, pack layout), understand
Advanced visualizations: Introducing more complex charts (if time allows), using different types of
Performance: Optimizing the performance for large datasets, working with virtual DOM, using web
Accessibility: Making accessible visualizations.
12. Lab 12: Building a Complete Visualization Project
Project Selection: Defining the scope and objectives of the visualization project, identifying the aud
Data Gathering: Finding and cleaning relevant data for the project, working with different data sou
Visualization Design: Choosing the most appropriate visualizations to present the data, understand
Implementation: Using any of the tools covered in the labs to build a comprehensive data visualiza
Presentation: Presenting the visualization, explain the design choices and their impact, presenting
Documentation: Writing documentation for the project, explaining how to use the visualization, com
13. Flipped Class. Presentation of Projects
Students present their projects.
Feedback from instructors and other students, evaluating the project based on the criteria that wer
14. Final Exam
Comprehensive assessment of all the concepts covered throughout the course.
Date TA 0
27-Jan Mahmoud 1
3-Feb Roman Project Proposals out 2
10-Feb Mahmoud 3
17-Feb Roman Task 1 out Project Proposals due 4
24-Feb Mahmoud 5
3-Mar Roman Task 1 due 6
10-Mar Everyone 7
17-Mar Mahmoud Task 2 out 8
24-Mar Roman 9
31-Mar Mahmoud Task 2 due 10
7-Apr Roman 11
14-Apr Mahmoud Projects are due 12
21-Apr Rustam 13
TBD Everyone 14
, for/while loops), functions, basic input/output, list comprehensions.
and spans, forms (<form>), and semantic HTML tags: <nav>, <article>, <aside>, <footer>
method, action.
o avoid memory issues.
gations (COUNT, SUM, AVG, MIN, MAX), update and delete statements.
ent, avoiding SQL injection.
ascript, using selenium or playwright.
embedding, and their limitations.
different licenses.
ain multiple LLM calls.
practices.
erent data formats.
lities of LLMs.
se and when not to use LLMs.
beddings), Basic Statistics and EDA
fill, or removal, using masks.
g lambda functions.
nsform, and encoding (one hot encoding, label encoding, ordinal encoding), using sklearn library.
duction for embeddings.
ms, box plots, violin plots, heatmaps, pie charts, treemaps, etc.), avoiding misleading plots.
ps, and creating custom themes.
working with template inheritance.
querystrings.
hic cameras.
tanding different types of lights.
x visualization, using maps for geographical visualizations.
d handling messy data.
nderstanding the impact of each visual element, selecting the best tool for the job.
dular approach.
torytelling to present the data.
the project.
Lectures
Introduction, overview of the course
Maps of knowledge / Python vs JS
WebDev 101. DOM, HTML, CSS, JS
Databases for datavis
Data collection
Advanced data collection with LLM's
EDA, ML for EDA
Midterm
Vis. Frameworks
Flask
Three.JS
D3.JS
Connecting the dots
Flipped class
Final exam
learn library.
