AD3301-Data Exploration and Visualization
List of Experiments
S.No Date Experiment Name Page Marks
No
1. Installation of the data Analysis and
Visualization tool Power BI
2. Implementation of exploratory data
analysis (EDA) on email data set.
3. Implementation of Numpy arrays,
Pandas data frames, Basic plots using
Matplotlib
4. Implementation of various variable and
row filters in R and various plot
features in R
5. Implementation of Time Series
Analysis and various visualization
techniques.
6. Implementation of Data Analysis and
representation on a Map using various
Map data sets
7. Implementation of cartographic
visualization for multiple datasets
8. Implementation of EDA on Wine
Quality Data Set.
9. Case study on a data set and apply the
various EDA and visualization
techniques and present an analysis
report.
Total
Staff Incharge
1
Ex.No: Installation of the data Analysis and Visualization tool
01 Power BI
AIM
ALGORITHM
PROGRAM
1. Download and Install Power BI Desktop
1. Download Power BI Desktop:
o Go to the Power BI Desktop download page.
o Click on “Download Free” to start the download.
2. Run the Installer:
2
o Locate the downloaded installer file (usually in your
Downloads folder).
o Double-click the file to start the installation process.
o Follow the on-screen instructions to complete the installation.
You may need to accept the license agreement and choose an
installation location.
3
3. Launch Power BI Desktop:
o After installation, open Power BI Desktop from the Start menu
or desktop shortcut.
2. Initial Configuration
1. Configure Initial Settings:
o When Power BI Desktop opens for the first time, you may be
prompted to sign in. You can sign in with a Microsoft account,
but this is optional for local use.
o Choose any initial settings based on your preferences or leave
them as defaults.
4
3. Import Data
1. Load Data:
o Click on the “Home” tab in the ribbon.
o Click “Get Data” to open the data source options.
o Choose the type of data source (e.g., Excel, CSV, SQL Server)
and follow the prompts to connect to your data source.
2. Transform and Clean Data:
o After importing the data, you may need to perform data
cleaning or transformation.
o Use the “Transform Data” button to open Power Query Editor,
where you can clean and transform the data (e.g., remove
duplicates, filter rows).
5
4. Create Visualizations
1. Add Visualizations:
o Once your data is loaded and cleaned, you can start creating
visualizations.
o In the “Report” view, drag and drop fields from your data onto
the report canvas.
o Choose from various visualization types (e.g., bar charts, line
charts, pie charts) from the “Visualizations” pane.
6
2. Customize Visualizations:
o Click on a visualization to configure its properties (e.g., axis
titles, colors).
o Use the “Format” pane to adjust visual settings.
7
3. Arrange Visualizations:
o Arrange and resize visualizations on the canvas to create a
meaningful report layout.
5. Publish and Share Reports
1. Publish to Power BI Service (Optional):
o Click the “Publish” button on the “Home” tab.
o Sign in with your Microsoft account if required.
o Choose a workspace in Power BI Service where you want to
publish the report.
8
2. Share Reports:
o After publishing, you can share the report with others by
providing them access through the Power BI Service.
o Use sharing options available in Power BI Service to control
who can view or interact with the report.
RESULT
Ex.No: Implementation of exploratory data analysis (EDA) on
02 email data set
AIM
ALGORITHM
9
PROGRAM
import pandas as pd
# Load the email dataset
df = pd.read_csv('emails.csv')
# Display the first few rows of the dataset
print(df.head())
# Display basic information about the dataset
print(df.info())
# Check for missing values
print(df.isnull().sum())
# Convert the 'Date' column to datetime format
df['Date'] = pd.to_datetime(df['Date'], errors='coerce')
# Fill missing values in the 'Sender' and 'Subject' columns if any
df['Sender'].fillna('Unknown', inplace=True)
df['Subject'].fillna('No Subject', inplace=True)
# Check for any remaining missing values
print(df.isnull().sum())
import matplotlib.pyplot as plt
# Plot the number of emails over time
plt.figure(figsize=(10, 6))
df['Date'].groupby(df['Date'].dt.to_period('M')).count().plot(kind='bar')
plt.title('Number of Emails Over Time')
plt.xlabel('Date')
plt.ylabel('Number of Emails')
plt.show()
# Top 10 senders
top_senders = df['Sender'].value_counts().head(10)
10
# Plotting the top senders
plt.figure(figsize=(10, 6))
top_senders.plot(kind='bar', color='skyblue')
plt.title('Top 10 Senders')
plt.xlabel('Sender')
plt.ylabel('Number of Emails')
plt.show()
from wordcloud import WordCloud
# Generate a word cloud from the subject lines
wordcloud = WordCloud(width=800, height=400,
background_color='white').generate(' '.join(df['Subject']))
# Display the word cloud
plt.figure(figsize=(10, 6))
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off')
plt.title('Word Cloud of Email Subjects')
plt.show()
# Extract the hour from the 'Date' column
df['Hour'] = df['Date'].dt.hour
# Plot the number of emails by hour
plt.figure(figsize=(10, 6))
df['Hour'].value_counts().sort_index().plot(kind='bar', color='orange')
plt.title('Emails Received by Hour of the Day')
plt.xlabel('Hour')
plt.ylabel('Number of Emails')
plt.show()
# Calculate the length of each email body
df['Body_Length'] = df['Body'].apply(lambda x: len(str(x).split()))
# Plot the distribution of email lengths
plt.figure(figsize=(10, 6))
df['Body_Length'].plot(kind='hist', bins=30, color='green')
plt.title('Distribution of Email Lengths')
plt.xlabel('Number of Words')
plt.ylabel('Number of Emails')
plt.show()
11
OUTPUT
12
13
RESULT
Ex.No: Implementation of Numpy arrays, Pandas data frames ,
03 Basic plots using Matplotlib
AIM
ALGORITHM
PROGRAM
1. NumPy Arrays
pip install numpy
import numpy as np
# 1D Array
arr_1d = np.array([1, 2, 3, 4, 5])
print("1D Array:", arr_1d)
# 2D Array
arr_2d = np.array([[1, 2, 3], [4, 5, 6]])
print("2D Array:\n", arr_2d)
# Array with specific shape and values
zeros = np.zeros((2, 3))
ones = np.ones((3, 2))
identity = np.eye(3)
print("Zeros Array:\n", zeros)
print("Ones Array:\n", ones)
14
print("Identity Matrix:\n", identity)
# Element-wise operations
arr_add = arr_1d + 10
arr_mult = arr_2d * 2
print("Array after addition:\n", arr_add)
print("Array after multiplication:\n", arr_mult)
# Matrix operations
matrix_mult = np.dot(arr_2d, np.array([[1, 0], [0, 1], [1, 0]]))
print("Matrix Multiplication:\n", matrix_mult)
2. Pandas DataFrames
pip install pandas
import pandas as pd
# Creating DataFrame from a dictionary
data = {
'Name': ['Alice', 'Bob', 'Charlie'],
'Age': [25, 30, 35],
'City': ['New York', 'Los Angeles', 'Chicago']
}
df = pd.DataFrame(data)
print("DataFrame:\n", df)
# Creating DataFrame from a CSV file
# df = pd.read_csv('file.csv') # Uncomment and use if you have a CSV
file
# Basic statistics
print("Statistics:\n", df.describe(include='all'))
# Accessing specific columns
print("Names:\n", df['Name'])
# Filtering rows
filtered_df = df[df['Age'] > 28]
print("Filtered DataFrame:\n", filtered_df)
# Adding a new column
df['Country'] = 'USA'
print("DataFrame with New Column:\n", df)
15
3. Basic Plots Using Matplotlib
pip install matplotlib
import matplotlib.pyplot as plt
# Line Plot
x = np.linspace(0, 10, 100)
y = np.sin(x)
plt.figure(figsize=(10, 6))
plt.plot(x, y, label='Sine Wave', color='blue', linestyle='-')
plt.title('Line Plot')
plt.xlabel('X axis')
plt.ylabel('Y axis')
plt.legend()
plt.grid(True)
plt.show()
# Scatter Plot
x = np.random.rand(50)
y = np.random.rand(50)
plt.figure(figsize=(10, 6))
plt.scatter(x, y, color='red', alpha=0.5)
plt.title('Scatter Plot')
plt.xlabel('X axis')
plt.ylabel('Y axis')
plt.grid(True)
plt.show()
# Histogram
data = np.random.randn(1000)
plt.figure(figsize=(10, 6))
plt.hist(data, bins=30, edgecolor='black')
plt.title('Histogram')
plt.xlabel('Value')
plt.ylabel('Frequency')
plt.grid(True)
plt.show()
# Bar Plot
categories = ['A', 'B', 'C']
16
values = [10, 20, 15]
plt.figure(figsize=(10, 6))
plt.bar(categories, values, color='green')
plt.title('Bar Plot')
plt.xlabel('Category')
plt.ylabel('Value')
plt.grid(True)
plt.show()
OUTPUT
17
18
RESULT
Ex.No: Implementation of various variable and row filters in R
04 and various plot features in R
AIM
ALGORITHM
19
PROGRAM
Step 1: Data Loading and Initial Inspection
# Load a sample dataset (e.g., the 'mtcars' dataset)
data(mtcars)
# View the first few rows of the dataset
head(mtcars)
# Get a summary of the dataset
summary(mtcars)
Step 2: Variable Filtering
# Select specific columns (e.g., 'mpg', 'hp', 'wt')
selected_vars <- mtcars[, c('mpg', 'hp', 'wt')]
# View the filtered dataset
head(selected_vars)
Step 3: Row Filtering
# Filter rows where mpg (miles per gallon) is greater than 20
filtered_rows <- mtcars[mtcars$mpg > 20, ]
# Filter rows where 'cyl' (cylinders) equals 4
filtered_rows_cyl <- mtcars[mtcars$cyl == 4, ]
# View the filtered dataset
head(filtered_rows)
head(filtered_rows_cyl)
Step 4: Data Cleaning
# Remove rows with missing values (if any)
cleaned_data <- na.omit(mtcars)
# Rename columns for clarity
colnames(cleaned_data) <- c('Miles_Per_Gallon', 'Cylinders',
'Displacement',
'Horsepower', 'Rear_Axle_Ratio', 'Weight',
'Quarter_Mile_Time', 'Transmission', 'Gears',
'Carburetors')
# View the cleaned dataset
head(cleaned_data)
Step 5: Data Visualization
# Load necessary libraries
library(ggplot2)
20
# Scatter plot of Horsepower vs Miles Per Gallon
ggplot(mtcars, aes(x = hp, y = mpg)) +
geom_point(color = 'blue') +
labs(title = "Horsepower vs Miles Per Gallon", x = "Horsepower", y =
"Miles Per Gallon")
# Histogram of Miles Per Gallon
ggplot(mtcars, aes(x = mpg)) +
geom_histogram(binwidth = 2, fill = 'green', color = 'black') +
labs(title = "Distribution of Miles Per Gallon", x = "Miles Per Gallon", y =
"Frequency")
# Boxplot of Miles Per Gallon by Cylinder
ggplot(mtcars, aes(x = factor(cyl), y = mpg)) +
geom_boxplot(fill = 'orange', color = 'black') +
labs(title = "Miles Per Gallon by Cylinder", x = "Number of Cylinders", y
= "Miles Per Gallon")
OUTPUT
21
22
23
RESULT
Ex.No: Implementation of Time Series Analysis and various
05 visualization techniques
AIM
ALGORITHM
PROGRAM
pip install pandas numpy matplotlib
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
# Create a time series dataset
date_rng = pd.date_range(start='2023-01-01', end='2023-12-31',
freq='D')
24
data = pd.DataFrame(date_rng, columns=['date'])
data['value'] = np.sin(np.linspace(0, 10, len(date_rng))) +
np.random.normal(0, 0.5, len(date_rng))
# Set the date column as the index
data.set_index('date', inplace=True)
print(data.head())
plt.figure(figsize=(12, 6))
plt.plot(data.index, data['value'], label='Value')
plt.title('Time Series Plot')
plt.xlabel('Date')
plt.ylabel('Value')
plt.legend()
plt.grid(True)
plt.show()
# Compute rolling statistics
data['rolling_mean'] = data['value'].rolling(window=30).mean()
data['rolling_std'] = data['value'].rolling(window=30).std()
plt.figure(figsize=(12, 6))
plt.plot(data.index, data['value'], label='Value', alpha=0.5)
plt.plot(data.index, data['rolling_mean'], label='Rolling Mean', color='red')
plt.plot(data.index, data['rolling_std'], label='Rolling Std Dev',
color='orange')
plt.title('Rolling Statistics')
plt.xlabel('Date')
plt.ylabel('Value')
plt.legend()
plt.grid(True)
plt.show()
from pandas.plotting import autocorrelation_plot
plt.figure(figsize=(12, 6))
autocorrelation_plot(data['value'])
plt.title('Autocorrelation Plot')
plt.grid(True)
plt.show()
# Extract month from the index
25
data['month'] = data.index.month
# Plot by month
plt.figure(figsize=(12, 6))
for month in range(1, 13):
monthly_data = data[data['month'] == month]
plt.plot(monthly_data.index, monthly_data['value'], label=f'Month
{month}')
plt.title('Seasonal Plot by Month')
plt.xlabel('Date')
plt.ylabel('Value')
plt.legend()
plt.grid(True)
plt.show()
OUTPUT
26
RESULT
27
Ex.No: Implementation of Data Analysis and representation on
06 a Map using various Map data sets
AIM
ALGORITHM
PROGRAM
pip install folium geopandas plotly pandas
import folium
import geopandas as gpd
import pandas as pd
# Step 2: Load Geographic Data
# Example dataset: Load a sample GeoDataFrame
world = gpd.read_file(gpd.datasets.get_path('naturalearth_lowres'))
# Load some data for analysis (e.g., population or any other metric)
# For this example, we'll use a simple dataset:
data = pd.DataFrame({
'iso_a3': ['USA', 'CAN', 'MEX'],
'population': [331002651, 37742154, 126190788]
})
# Merge the data with the GeoDataFrame
world = world.merge(data, how='left', on='iso_a3')
# Step 3: Create a map with Folium
m = folium.Map(location=[20, 0], zoom_start=2)
# Step 4: Add Mouse Rollover and User Interaction
28
# Add a Choropleth map
folium.Choropleth(
geo_data=world,
name="choropleth",
data=world,
columns=["iso_a3", "population"],
key_on="feature.properties.iso_a3",
fill_color="YlGn",
fill_opacity=0.7,
line_opacity=0.2,
legend_name="Population by Country",
).add_to(m)
# Add GeoJson for mouse rollover effect
folium.GeoJson(
world,
name="Population Info",
tooltip=folium.GeoJsonTooltip(
fields=["name", "population"],
aliases=["Country", "Population"],
localize=True,
),
).add_to(m)
# Step 5: Save and Display the Map
m.save("interactive_map.html")
# Display map in Jupyter Notebook (if using Jupyter)
m
OUTPUT
29
RESULT
Ex.No: Implementation of cartographic visualization for
07 multiple datasets
AIM
ALGORITHM
30
PROGRAM
pip install geopandas folium matplotlib
import geopandas as gpd
import folium
import matplotlib.pyplot as plt
# Load the world map dataset
world = gpd.read_file(gpd.datasets.get_path('naturalearth_lowres'))
# Display the first few rows
print(world.head())
# Plot the world map
world.plot(figsize=(15, 10), edgecolor='black')
plt.title('World Map')
plt.show()
# Create a base map centered around the world
m = folium.Map(location=[20, 0], zoom_start=2)
# Add countries as GeoJson to the map
folium.GeoJson(world).add_to(m)
# Save the map as an HTML file
m.save("world_map.html")
# Example: Adding population data to the world map
# Add a column to the world dataset for population (you may need to add
real data here)
world['pop_est'] = world['pop_est'].fillna(0) # Fill NA values with 0
# Plot using matplotlib with color based on population
world.plot(column='pop_est', figsize=(15, 10), legend=True,
cmap='OrRd', edgecolor='black')
31
plt.title('World Population Map')
plt.show()
# Display in Jupyter Notebook (if using Jupyter)
m # m is the folium map object
OUTPUT
32
RESULT
Ex.No: Implementation of EDA on Wine Quality Data Set
08
AIM
ALGORITHM
33
PROGRAM
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
# Load the wine quality dataset
url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/wine-
quality/winequality-red.csv'
wine_data = pd.read_csv(url, sep=';')
# Display the first few rows of the dataset
print(wine_data.head())
# Basic information about the dataset
print(wine_data.info())
# Summary statistics
print(wine_data.describe())
# Check for missing values
print(wine_data.isnull().sum())
# Distribution of wine quality
plt.figure(figsize=(8, 6))
sns.countplot(x='quality', data=wine_data, palette='viridis')
plt.title('Distribution of Wine Quality Ratings')
plt.show()
# Correlation heatmap
plt.figure(figsize=(10, 8))
corr = wine_data.corr()
sns.heatmap(corr, annot=True, cmap='coolwarm', linewidths=0.5)
plt.title('Correlation Heatmap of Wine Quality Data')
plt.show()
# Pairplot of selected features
sns.pairplot(wine_data, vars=['fixed acidity', 'volatile acidity', 'citric acid',
'quality'], hue='quality', palette='viridis')
plt.show()
# Alcohol content vs quality
plt.figure(figsize=(10, 6))
sns.boxplot(x='quality', y='alcohol', data=wine_data, palette='viridis')
34
plt.title('Alcohol Content vs Wine Quality')
plt.show()
# Fixed acidity vs quality
plt.figure(figsize=(10, 6))
sns.boxplot(x='quality', y='fixed acidity', data=wine_data,
palette='viridis')
plt.title('Fixed Acidity vs Wine Quality')
plt.show()
# Example of handling imbalance (if necessary)
from sklearn.utils import resample
# Separate the minority and majority classes
df_majority = wine_data[wine_data.quality >= 6]
df_minority = wine_data[wine_data.quality < 6]
# Upsample minority class
df_minority_upsampled = resample(df_minority,
replace=True, # sample with replacement
n_samples=len(df_majority), # to match majority
class
random_state=42) # reproducible results
# Combine majority class with upsampled minority class
wine_data_balanced = pd.concat([df_majority, df_minority_upsampled])
# Display new class counts
print(wine_data_balanced['quality'].value_counts())
# 3D Scatter plot
fig = plt.figure(figsize=(10, 8))
ax = fig.add_subplot(111, projection='3d')
ax.scatter(wine_data['sulphates'], wine_data['alcohol'],
wine_data['quality'], c=wine_data['quality'], cmap='viridis')
ax.set_xlabel('Sulphates')
ax.set_ylabel('Alcohol')
ax.set_zlabel('Quality')
plt.show()
OUTPUT
35
36
37
38
RESULT
Ex.No: Case study on a data set and apply the various EDA and
09 visualization techniques and present an analysis report.
39
AIM
ALGORITHM
PROGRAM
The Titanic dataset is a classic dataset often used to demonstrate various
data analysis techniques. This dataset provides information on the
passengers aboard the Titanic, including whether they survived the
disaster, their age, gender, class, fare, and other details. This case study
will apply various EDA and visualization techniques to uncover insights
from this data and present an analysis report.
Dataset Overview
Dataset: Titanic passenger data.
Features:
o Survived: Survival (0 = No, 1 = Yes)
o Pclass: Ticket class (1 = 1st, 2 = 2nd, 3 = 3rd)
o Sex: Sex
o Age: Age in years
o SibSp: Number of siblings/spouses aboard the Titanic
o Parch: Number of parents/children aboard the Titanic
o Fare: Passenger fare
o Embarked: Port of Embarkation (C = Cherbourg; Q =
Queenstown; S = Southampton)
Step 1: Data Loading and Inspection
import pandas as pd
# Load the Titanic dataset
titanic_data =
pd.read_csv('https://raw.githubusercontent.com/datasciencedojo/datasets/
master/titanic.csv')
40
# Display the first few rows of the dataset
print(titanic_data.head())
# Basic information about the dataset
print(titanic_data.info())
# Summary statistics
print(titanic_data.describe())
Observation:
The dataset has 891 entries and several features, some of which
have missing values (e.g., Age, Cabin, and Embarked).
Step 2: Data Cleaning
# Handle missing values by filling or dropping
titanic_data['Age'].fillna(titanic_data['Age'].median(), inplace=True)
titanic_data['Embarked'].fillna(titanic_data['Embarked'].mode()[0],
inplace=True)
titanic_data.drop(columns=['Cabin'], inplace=True) # Drop Cabin due to
too many missing values
# Verify missing values
print(titanic_data.isnull().sum())
Observation:
Missing values have been handled: Age filled with median,
Embarked filled with mode, and Cabin dropped.
Step 3: Univariate Analysis
import seaborn as sns
import matplotlib.pyplot as plt
# Plotting the distribution of survival
sns.countplot(x='Survived', data=titanic_data, palette='pastel')
plt.title('Survival Distribution')
plt.show()
# Distribution of Age
sns.histplot(titanic_data['Age'], bins=30, kde=True, color='blue')
plt.title('Age Distribution of Passengers')
plt.show()
Observation:
The majority of passengers did not survive.
Most passengers are between 20 and 40 years old.
Step 4: Bivariate Analysis
# Survival rate by class
sns.barplot(x='Pclass', y='Survived', data=titanic_data, palette='viridis')
41
plt.title('Survival Rate by Passenger Class')
plt.show()
# Survival rate by gender
sns.barplot(x='Sex', y='Survived', data=titanic_data, palette='viridis')
plt.title('Survival Rate by Gender')
plt.show()
# Age distribution by survival
sns.boxplot(x='Survived', y='Age', data=titanic_data, palette='pastel')
plt.title('Age Distribution by Survival')
plt.show()
Observation:
Passengers in 1st class had a higher survival rate.
Females had a significantly higher survival rate than males.
Younger passengers were more likely to survive.
Step 5: Multivariate Analysis
# Survival rate by class and gender
sns.catplot(x='Pclass', hue='Sex', col='Survived', kind='count',
data=titanic_data, palette='pastel')
plt.show()
# Correlation heatmap
plt.figure(figsize=(10, 8))
corr = titanic_data.corr()
sns.heatmap(corr, annot=True, cmap='coolwarm', linewidths=0.5)
plt.title('Correlation Heatmap')
plt.show()
Observation:
The survival rate is highest for females in 1st class.
There is a strong negative correlation between Pclass and Survived,
indicating that higher-class passengers were more likely to survive.
Insights
1. Class and Survival: Passengers in higher classes had better
survival chances, with 1st class being the safest.
2. Gender and Survival: Females had a much higher survival rate,
especially in the 1st and 2nd classes.
3. Age Factor: Younger passengers had a better survival rate, with
children particularly having a higher chance of survival.
4. Embarkation Point: Passengers who embarked at Cherbourg (C)
had a higher survival rate compared to other embarkation points.
5. Multivariate Interaction: The combination of gender, class, and
age played a crucial role in determining the survival of a passenger.
42
OUTPUT
43
44
45
RESULT
46
