Module 14 - Module & Packages

1. Working with import statement

In Python, the import statement is used to bring modules, packages, or specific components of
modules into your code, so you can use their functions, classes, and variables. Here's how you
can work with it:

1. Importing Entire Modules

import math

● You can now use the math module by prefixing its functions or constants with math.:

print(math.sqrt(16)) # Output: 4.0

print(math.pi) # Output: 3.141592653589793

2. Importing Specific Components

from math import sqrt, pi

● Now you can directly use sqrt and pi without the math. prefix:

print(sqrt(16)) # Output: 4.0

print(pi) # Output: 3.141592653589793

3. Using Aliases

a. Aliasing Modules:

import numpy as np

● Use the alias np to refer to numpy:

array = np.array([1, 2, 3])

print(array) # Output: [1 2 3]
b. Aliasing Components:

from math import sqrt as square_root

● Use the alias square_root for sqrt:

print(square_root(16)) # Output: 4.0

4. Importing Everything

from math import *

● This brings all components of the module into your code's namespace:

print(sqrt(16)) # Output: 4.0

print(pi) # Output: 3.141592653589793

Caution: This method may lead to namespace pollution, causing conflicts with existing
variables or functions.

5. Importing Custom Modules

If you have a file named my_module.py in the same directory:

# my_module.py
def greet():
return "Hello, World!"

You can import it:

import my_module
print(my_module.greet()) # Output: Hello, World!

6. Importing from Packages

For a package with the following structure:

mypackage/
__init__.py
module1.py
module2.py

You can import like this:

from mypackage import module1

Or access a specific component:

from mypackage.module1 import my_function

7. Dynamic Imports (importlib)

If you need to import a module dynamically at runtime:

import importlib
math_module = importlib.import_module('math')
print(math_module.sqrt(16)) # Output: 4.0

Best Practices

● Use specific imports (from module import component) for clarity and
efficiency.
● Use aliases to improve readability for long module names.
● Avoid using from module import * in large projects.
2. Modules and Packages in Python

In Python, modules and packages are ways to structure and organize code into reusable
components.

1. Modules

A module is a single Python file containing definitions, functions, classes, and executable code.
Modules help you organize and reuse code by breaking it into smaller, manageable files.

Creating and Using Modules

Example:

Create a file named my_module.py:

# my_module.py
def greet(name):
return f"Hello, {name}!"
def add(a, b):
return a + b

Import and use it in another Python script:

# main.py
import my_module
print(my_module.greet("Alice")) # Output: Hello, Alice!
print(my_module.add(5, 3)) # Output: 8
Types of Module Imports

a. Import the entire module:

import my_module
print(my_module.greet("Alice"))
b. Import specific functions or classes:

from my_module import greet

print(greet("Alice"))

c. Alias modules or components:

import my_module as mm
print(mm.greet("Alice"))

d. Import all components (not recommended):

from my_module import *

print(greet("Alice"))

2. Packages

A package is a directory containing a special __init__.py file and one or more module files.
It is used to organize related modules into a hierarchy.

Structure of a Package

mypackage/
__init__.py
module1.py
module2.py

Purpose of __init__.py

The __init__.py file can be empty, or it can include initialization code for the package. It
makes Python treat the directory as a package.

Creating a Package
Directory Structure:

mypackage/
__init__.py
greetings.py
calculations.py

Code for greetings.py:

def say_hello(name):
return f"Hello, {name}!"

Code for calculations.py:

def add(a, b):

return a + b

Importing the Package:

from mypackage import greetings, calculations

print(greetings.say_hello("Alice")) # Output: Hello, Alice!
print(calculations.add(5, 3)) # Output: 8

Relative Imports in Packages

You can use relative imports within a package to import from sibling modules.

Example:

# mypackage/calculations.py
from .greetings import say_hello
def greet_and_add(name, a, b):
greeting = say_hello(name)
total = a + b
return f"{greeting} The sum is {total}."
Built-in and Third-Party Modules

Built-in Modules

Python includes many pre-installed modules like os, sys, math, datetime, etc.

import math
print(math.sqrt(16)) # Output: 4.0

Third-Party Modules

These are installed via pip (Python Package Installer). For example:

pip install requests

Usage:

import requests
response = requests.get('https://example.com')
print(response.status_code)

Key Differences Between Modules and Packages

Aspect Module Package

Structure Single .py file Directory with an __init__.py file

Purpose Organize code into a single file Organize multiple modules into a folder

Import Example import module_name from package_name import

module_name

Best Practices
 1. Use packages to structure large projects.
2. Follow naming conventions: use lowercase for module and package names.
3. Keep modules focused on a single responsibility.
4. Document your modules and packages clearly.

3. Working with calendar modules

The calendar module in Python provides functions for working with calendars and dates. It
includes methods for creating, manipulating, and formatting calendar data, making it useful for
date-related operations.

1. Importing the Calendar Module

import calendar

2. Key Functions and Features

a. Generate Text and HTML Calendars

Text Calendar

import calendar
# Print a text calendar for a specific month
print(calendar.month(2024, 12))
# Print a text calendar for a whole year
print(calendar.calendar(2024))

Output:
 December 2024
Mo Tu We Th Fr Sa Su
1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31

HTML Calendar

html_calendar = calendar.HTMLCalendar()
html_code = html_calendar.formatmonth(2024, 12)
print(html_code)

b. Working with Days and Weeks

Weekday Constants

The calendar module provides constants for days of the week:

● calendar.MONDAY = 0
● calendar.SUNDAY = 6

Get the Weekday of a Date

# Get the weekday (0 = Monday, 6 = Sunday)

weekday = calendar.weekday(2024, 12, 25)
print(weekday) # Output: 2 (Wednesday)

Check if a Year is a Leap Year

is_leap = calendar.isleap(2024)
print(is_leap) # Output: True
Count Leap Years in a Range

leap_days = calendar.leapdays(2000, 2025)

print(leap_days) # Output: 6

c. Iterating Over Weeks and Days

Month's Days as a List

month_days = calendar.Calendar().monthdayscalendar(2024, 12)

for week in month_days:
print(week) # Output: Lists of week numbers, 0 for padding
days

Example:

[0, 0, 0, 0, 0, 0, 1]
[2, 3, 4, 5, 6, 7, 8]
[9, 10, 11, 12, 13, 14, 15]
[16, 17, 18, 19, 20, 21, 22]
[23, 24, 25, 26, 27, 28, 29]
[30, 31, 0, 0, 0, 0, 0]
d. Customize Calendar Start Day

By default, weeks start on Monday. You can change it to Sunday

calendar.setfirstweekday(calendar.SUNDAY)
print(calendar.month(2024, 12))

e. Get Calendar Information

First Weekday of the Month and Number of Days

first_weekday, days_in_month = calendar.monthrange(2024, 12)

print(first_weekday) # Output: 6 (Sunday)
print(days_in_month) # Output: 31

f. Iterating Over Dates

The calendar module also provides an iterweekdays() method:

c = calendar.Calendar()
for day in c.iterweekdays():
print(day) # Outputs day numbers for a week starting from
MONDAY

3. Practical Examples

a. List All Mondays in a Month

year, month = 2024, 12

c = calendar.Calendar()
mondays = [day for day in c.itermonthdays(year, month) if day !=
0 and calendar.weekday(year, month, day) == 0]
print(mondays) # Output: [2, 9, 16, 23, 30]

b. Check if a Specific Date is a Holiday

from datetime import datetime

def is_holiday(year, month, day):
holidays = [(12, 25)] # Christmas Day
return (month, day) in holidays

print(is_holiday(2024, 12, 25)) # Output: True

print(is_holiday(2024, 12, 26)) # Output: False
c. Create a Meeting Planner

Generate dates for every Friday in a month:

year, month = 2024, 12

fridays = [day for day in range(1, calendar.monthrange(year,
month)[1] + 1)
if calendar.weekday(year, month, day) == 4]
print(fridays) # Output: [6, 13, 20, 27]

c. First Day of the Month

Scenario: You want to display the first weekday of every month for the year 2024 to decide
which months begin with a Monday.

import calendar

year = 2024

for month in range(1, 13):

first_day, _ = calendar.monthrange(year, month)

if first_day == calendar.MONDAY:

print(f"{calendar.month_name[month]} starts on a
Monday.")

Output:

January starts on a Monday.

April starts on a Monday.

July starts on a Monday.

September starts on a Monday.

December starts on a Monday.

The calendar module is versatile and highly useful for scheduling, formatting dates, and
date-related computations.

4. Working with random numbers

In Python, the random module provides tools for generating random numbers and performing
randomization tasks. Here's how you can work with it effectively.

1. Importing the random Module

import random

2. Generating Random Numbers

a. Random Floating-Point Numbers

● random(): Generates a random float between 0.0 (inclusive) and 1.0 (exclusive).

print(random.random()) # Output: 0.37444887175646646 (example)

● uniform(a, b): Generates a random float between a and b.

print(random.uniform(1, 10)) # Output: 7.5892 (example)

b. Random Integers
 ● randint(a, b): Returns a random integer between a and b (inclusive).

print(random.randint(1, 10)) # Output: 4 (example)

● randrange(start, stop[, step]): Returns a random integer from a range

with a step.

print(random.randrange(0, 10, 2)) # Output: 8 (example)

c. Random Boolean

print(random.choice([True, False])) # Output: True or False

3. Working with Sequences

a. Random Element from a List

● choice(seq): Selects a random element from a sequence.

colors = ['red', 'blue', 'green']

print(random.choice(colors)) # Output: 'green' (example)

b. Shuffle a List

● shuffle(seq): Shuffles the elements of a list in place.

numbers = [1, 2, 3, 4, 5]
random.shuffle(numbers)
print(numbers) # Output: [4, 1, 5, 2, 3] (example)

c. Random Sample from a List

● sample(seq, k): Returns k unique elements as a new list.

numbers = [1, 2, 3, 4, 5]
print(random.sample(numbers, 3)) # Output: [2, 4, 1] (example)

4. Generating Random Values from Distributions

a. Normal Distribution

● gauss(mu, sigma): Generates a random value with a Gaussian (normal)

distribution.

print(random.gauss(0, 1)) # Output: -0.2128 (example)

● normalvariate(mu, sigma): Another way to generate a random value from a

normal distribution.

print(random.normalvariate(0, 1)) # Output: 0.5688 (example)

b. Exponential Distribution

● expovariate(lambda): Generates a random value from an exponential distribution.

print(random.expovariate(1.5)) # Output: 0.2904 (example)

c. Uniform Distribution

● betavariate(alpha, beta): Generates a value from a Beta distribution.

print(random.betavariate(2, 5)) # Output: 0.3741 (example)

5. Controlling Randomness

Setting the Seed

The random module generates pseudo-random numbers based on a seed. You can set the seed to
make results reproducible:
random.seed(42)
print(random.random()) # Output: 0.6394267984578837 (consistent
for the same seed)

6. Generating Secure Random Numbers

For cryptographic applications, use the secrets module instead of random:

import secrets
# Generate a secure random number
print(secrets.randbelow(10)) # Output: 7 (example)
# Generate a secure random token
print(secrets.token_hex(16)) # Output:
'a7e1c2e3f68d9ac3b4d2d234' (example)

7. Using NumPy for Random Numbers

The numpy.random module is more powerful for working with arrays and complex
distributions.

import numpy as np
# Generate an array of random numbers
print(np.random.rand(3)) # Output: [0.3744, 0.9507, 0.7320]
(example)
# Random integer array
print(np.random.randint(0, 10, size=(2, 3))) # Output: 2D array
of random integers

Examples
Generate a Random Password

import random
import string
length = 8
password = ''.join(random.choices(string.ascii_letters +
string.digits, k=length))
print(password) # Output: 'a2Bd5Fg7' (example)

Simulate a Dice Roll

dice_roll = random.randint(1, 6)
print(f"You rolled a {dice_roll}") # Output: 'You rolled a 4'
(example)

Simulating a Lottery Draw

Scenario: Simulate a lottery draw where 6 unique numbers are drawn from a range of 1 to 49.

import random
lottery_numbers = random.sample(range(1, 50), 6)
print(f"Lottery numbers: {sorted(lottery_numbers)}")

Randomized Seating Arrangement

Scenario: A teacher wants to randomize the seating arrangement for 5 students in a row.

import random
students = ['John', 'Emily', 'Sophia', 'Liam', 'Noah']
random.shuffle(students)
print(f"Randomized seating: {students}")

Random Weather Simulation

Scenario: Simulate random weather conditions (Sunny, Rainy, Cloudy) for a week.

import random
weather_conditions = ['Sunny', 'Rainy', 'Cloudy']
weekly_weather = [random.choice(weather_conditions) for _ in
range(7)]
print(f"Weekly weather: {weekly_weather}")

5. Working with dates modules

Python provides several modules for working with dates and times, with the most commonly
used ones being datetime and time. Here's how you can work with dates in Python:

1. Working with datetime Module

a. Getting the Current Date and Time

You can get the current date and time using the datetime.now() function.

from datetime import datetime

current_datetime = datetime.now()

print("Current date and time:", current_datetime)

Output (Example):

Current date and time: 2024-12-21 14:55:23.761912

b. Getting the Current Date

To get just the current date (without time), use the date() method:
from datetime import datetime

current_date = datetime.now().date()

print("Current date:", current_date)

Output (Example):

Current date: 2024-12-21

c. Creating a Specific Date

You can create a datetime object for a specific date using datetime(year, month,
day):

from datetime import datetime

specific_date = datetime(2024, 12, 25)

print("Specific date:", specific_date)

Output:

Specific date: 2024-12-25 00:00:00

2. Working with timedelta for Date Arithmetic

timedelta objects represent differences in dates and times. You can add or subtract days,
hours, etc., from a datetime.

a. Adding/Subtracting Days to a Date

from datetime import datetime, timedelta

current_date = datetime.now().date()

new_date = current_date + timedelta(days=10) # Adding 10 days

print("New date after adding 10 days:", new_date)

previous_date = current_date - timedelta(days=10) # Subtracting

10 days

print("New date after subtracting 10 days:", previous_date)

Output (Example):

New date after adding 10 days: 2024-12-31

New date after subtracting 10 days: 2024-12-11

b. Calculating the Difference Between Dates

from datetime import datetime

date1 = datetime(2024, 12, 21)

date2 = datetime(2024, 12, 25)

difference = date2 - date1

print("Difference between the dates:", difference)

Output:

Difference between the dates: 4 days, 0:00:00

3. Formatting Dates

You can format datetime objects into human-readable strings using strftime().
a. Custom Date Format

from datetime import datetime

current_datetime = datetime.now()

formatted_date = current_datetime.strftime("%A, %B %d, %Y")

print("Formatted date:", formatted_date)

Output (Example):

Formatted date: Saturday, December 21, 2024

b. Extracting Specific Components

You can extract specific parts of a date or time, like the day of the week, year, month, etc.

from datetime import datetime

current_datetime = datetime.now()

print("Year:", current_datetime.year)

print("Month:", current_datetime.month)

print("Day:", current_datetime.day)

print("Hour:", current_datetime.hour)

print("Minute:", current_datetime.minute)

print("Second:", current_datetime.second)

Output (Example):

Year: 2024
Month: 12

Day: 21

Hour: 14

Minute: 55

Second: 23

4. Working with date Class

You can also work with just the date class from the datetime module if you don’t need the
time part.

a. Creating a Date Object

from datetime import date

specific_date = date(2024, 12, 25)

print("Specific date:", specific_date)

Output:

Specific date: 2024-12-25

b. Getting the Current Date

from datetime import date

today = date.today()

print("Today's date:", today)

Output (Example):

Today's date: 2024-12-21

5. Working with time Class

You can also work with just the time class to manipulate times.

a. Creating a Time Object

from datetime import time

specific_time = time(14, 30, 45) # 2:30:45 PM

print("Specific time:", specific_time)

Output:

Specific time: 14:30:45

b. Getting the Current Time

from datetime import datetime

current_time = datetime.now().time()

print("Current time:", current_time)

Output (Example):

Current time: 14:55:23.761912

6. Parsing Strings to Dates

You can convert a string representation of a date into a datetime object using strptime().
from datetime import datetime

date_string = "2024-12-21 14:55:23"

parsed_date = datetime.strptime(date_string, "%Y-%m-%d

%H:%M:%S")

print("Parsed date:", parsed_date)

Output:

Parsed date: 2024-12-21 14:55:23

7. Using calendar for Date Operations

You can also use the calendar module to perform various date-related tasks.

a. Checking if a Year is a Leap Year

import calendar

year = 2024

if calendar.isleap(year):

print(f"{year} is a leap year.")

else:

print(f"{year} is not a leap year.")

Output:

2024 is a leap year.

b. Displaying a Month's Calendar

import calendar

year = 2024

month = 12

print(calendar.month(year, month))

Output:

December 2024

Mo Tu We Th Fr Sa Su

2 3 4 5 6 7 8

9 10 11 12 13 14 15

16 17 18 19 20 21 22

23 24 25 26 27 28 29

30 31

1. Time Zone Conversion Application

Scenario: Convert the current time in UTC to a specified time zone (e.g., New York or Tokyo).

import datetime
import pytz
# Get current time in UTC
utc_time = datetime.datetime.now(pytz.utc)
print("UTC Time:", utc_time)

# Convert UTC time to New York time

new_york_tz = pytz.timezone('America/New_York')
new_york_time = utc_time.astimezone(new_york_tz)
print("New York Time:", new_york_time)

# Convert UTC time to Tokyo time

tokyo_tz = pytz.timezone('Asia/Tokyo')
tokyo_time = utc_time.astimezone(tokyo_tz)
print("Tokyo Time:", tokyo_time)

Output (Example):

UTC Time: 2024-12-21 14:55:23.761912+00:00

New York Time: 2024-12-21 09:55:23.761912-05:00
Tokyo Time: 2024-12-21 23:55:23.761912+09:00

2. Scheduling Future Events in Multiple Time Zones

Scenario: Schedule a meeting 3 hours from now and display the scheduled time in multiple time
zones.

import datetime
import pytz

# Get current UTC time

utc_time = datetime.datetime.now(pytz.utc)
# Schedule the meeting 3 hours from now
future_time = utc_time + datetime.timedelta(hours=3)

# Convert future time to different time zones

london_tz = pytz.timezone('Europe/London')
london_time = future_time.astimezone(london_tz)

tokyo_tz = pytz.timezone('Asia/Tokyo')
tokyo_time = future_time.astimezone(tokyo_tz)

new_york_tz = pytz.timezone('America/New_York')
new_york_time = future_time.astimezone(new_york_tz)

print(f"Meeting scheduled in UTC: {future_time}")

print(f"Meeting in London: {london_time}")
print(f"Meeting in Tokyo: {tokyo_time}")
print(f"Meeting in New York: {new_york_time}")

Output (Example):

Meeting scheduled in UTC: 2024-12-21 17:55:23.761912+00:00

Meeting in London: 2024-12-21 17:55:23.761912+00:00
Meeting in Tokyo: 2024-12-21 02:55:23.761912+09:00
Meeting in New York: 2024-12-21 12:55:23.761912-05:00

3. Time Zone Aware Logging

Scenario: Track events in an application using time zone-aware timestamps, and log the event
times in the local time zone of the event's origin.

import datetime
import pytz

def log_event(event_name, event_time, time_zone):

# Get the local time for the event based on the provided
time zone
local_tz = pytz.timezone(time_zone)
local_time = event_time.astimezone(local_tz)

print(f"Event: {event_name}")
print(f"Time: {local_time.strftime('%Y-%m-%d %H:%M:%S
%Z%z')}")

# Assume the event occurs in Los Angeles time zone

event_time = datetime.datetime.now(pytz.utc) # Start with UTC
time
event_name = "User Login"
log_event(event_name, event_time, 'America/Los_Angeles')

Output (Example):

Event: User Login

Time: 2024-12-21 06:55:23 PST-0800

6. Working with JSON files

Working with JSON files in Python is straightforward using the built-in json module. Here are
some common tasks that you can perform when working with JSON files:

1. Reading JSON Data from a File

To read data from a JSON file, use json.load() to parse the contents into a Python
dictionary or list.

import json

# Read JSON data from a file

with open('data.json', 'r') as file:
data = json.load(file)

# Print the loaded data

print(data)

Example of data.json file:

{
"name": "John Doe",
"age": 30,
"city": "New York"
}

Output:

{'name': 'John Doe', 'age': 30, 'city': 'New York'}

2. Writing JSON Data to a File

To write a Python object (like a dictionary or list) to a JSON file, use json.dump().

import json
# Python dictionary
data = {
"name": "Jane Smith",
"age": 28,
"city": "Los Angeles"
}

# Write the data to a JSON file

with open('output.json', 'w') as file:
json.dump(data, file, indent=4)

# The JSON file will now contain the following:

# {
# "name": "Jane Smith",
# "age": 28,
# "city": "Los Angeles"
# }

3. Converting a Python Object to JSON String

Sometimes, you may want to convert a Python object to a JSON-formatted string without writing
it to a file. You can use json.dumps() for this.

import json

# Python dictionary
data = {
"name": "Alice",
"age": 25,
"city": "Seattle"
}

# Convert to JSON string

json_string = json.dumps(data, indent=4)
print(json_string)

Output:

{
"name": "Alice",
"age": 25,
"city": "Seattle"
}

4. Loading JSON Data from a String

If you have a JSON-formatted string, you can parse it into a Python object using
json.loads().

import json
# JSON string
json_string = '{"name": "Bob", "age": 22, "city": "Chicago"}'

# Convert the JSON string to a Python dictionary

data = json.loads(json_string)
print(data)

Output:

{'name': 'Bob', 'age': 22, 'city': 'Chicago'}

5. Pretty Printing JSON

You can pretty-print JSON data using json.dumps() with the indent argument, which
makes the output more readable.

import json
# Python dictionary
data = {
"name": "Charlie",
"age": 35,
"city": "San Francisco",
"skills": ["Python", "Data Analysis", "Machine Learning"]
}
# Pretty print the JSON data
pretty_json = json.dumps(data, indent=4)
print(pretty_json)

Output:

{
"name": "Charlie",
"age": 35,
"city": "San Francisco",
"skills": [
"Python",
 "Data Analysis",
"Machine Learning"
]
}

6. Handling JSON with Nested Structures

If the JSON data contains nested structures (like lists or dictionaries within dictionaries), it can
be accessed and manipulated similarly.

Example:

import json
# JSON with nested data
data = {
"name": "David",
"age": 40,
"address": {
"street": "123 Main St",
"city": "Boston",
"zipcode": "02101"
},
"phone_numbers": ["555-1234", "555-5678"]
}
# Convert the data to a JSON string with indentation
json_string = json.dumps(data, indent=4)
print(json_string)

Output:
{
"name": "David",
"age": 40,
"address": {
"street": "123 Main St",
"city": "Boston",
"zipcode": "02101"
},
"phone_numbers": [
"555-1234",
"555-5678"
]
}

7. Handling Errors When Reading/Parsing JSON

When reading or parsing JSON, you might encounter errors, such as invalid JSON syntax. You
can handle these errors using try and except.

import json

# Invalid JSON string

invalid_json_string = '{"name": "Eve", "age": 30'

# Try to parse the invalid JSON string

try:
data = json.loads(invalid_json_string)
except json.JSONDecodeError as e:
print(f"Error decoding JSON: {e}")
Output:

Error decoding JSON: Expecting ',' delimiter: line 1 column 30

(char 29)

8. Updating JSON Data

You can update specific values in a loaded JSON object just like you would with any dictionary
in Python.

import json
# Sample JSON data
data = {
"name": "Eve",
"age": 30,
"city": "Paris"
}

# Update a value
data["city"] = "Berlin"

# Write the updated data to a file

with open('updated_data.json', 'w') as file:
json.dump(data, file, indent=4)

9. Example with JSON File Handling in a Function

You can create a function to read a JSON file, manipulate the data, and write it back.

import json
def update_age(filename, new_age):
 with open(filename, 'r') as file:
data = json.load(file)

# Update the age

data['age'] = new_age

# Write the updated data to the file

with open(filename, 'w') as file:
json.dump(data, file, indent=4)

# Call the function

update_age('data.json', 45)

7. Working with CSV files, excel files

Python provides built-in support for reading from and writing to CSV files through the csv
module. Additionally, you can use the pandas library for more advanced handling of CSV data.

1. Reading CSV Files using csv module

To read a CSV file, you can use csv.reader() or csv.DictReader() to load the data
into a list of dictionaries.

import csv
# Read CSV file
with open('data.csv', 'r') as file:
csv_reader = csv.reader(file)
for row in csv_reader:
print(row)
Example of data.csv:

name,age,city
John,30,New York
Jane,28,Los Angeles
Alice,25,Seattle

Output:

['name', 'age', 'city']

['John', '30', 'New York']
['Jane', '28', 'Los Angeles']
['Alice', '25', 'Seattle']

2. Writing to CSV Files using csv module

To write data to a CSV file, use csv.writer() or csv.DictWriter().

import csv

# Data to write to CSV

data = [
["name", "age", "city"],
["John", 30, "New York"],
["Jane", 28, "Los Angeles"],
["Alice", 25, "Seattle"]
]

# Write data to CSV file

with open('output.csv', 'w', newline='') as file:
 csv_writer = csv.writer(file)
csv_writer.writerows(data)

output.csv file will look like:

name,age,city
John,30,New York
Jane,28,Los Angeles
Alice,25,Seattle

3. Reading and Writing CSV Files using pandas

pandas is a powerful library that makes it easier to work with CSV files, especially when
dealing with large datasets or complex operations.

Reading CSV File

import pandas as pd
# Read CSV file into DataFrame
df = pd.read_csv('data.csv')
print(df)

Output:

name age city

0 John 30 New York
1 Jane 28 Los Angeles
2 Alice 25 Seattle

Writing to a CSV File

import pandas as pd
# Create DataFrame
df = pd.DataFrame({
"name": ["John", "Jane", "Alice"],
"age": [30, 28, 25],
"city": ["New York", "Los Angeles", "Seattle"]
})

# Write DataFrame to CSV file

df.to_csv('output_pandas.csv', index=False)

output_pandas.csv file will look like:

name,age,city
John,30,New York
Jane,28,Los Angeles
Alice,25,Seattle

Working with Excel Files in Python

For handling Excel files, you can use libraries like openpyxl (for .xlsx files) or pandas
(for both .xls and .xlsx files).

1. Reading and Writing Excel Files using openpyxl

Reading from an Excel File:

from openpyxl import load_workbook

# Load Excel file

wb = load_workbook('data.xlsx')
# Select the active sheet
sheet = wb.active

# Read data from the sheet

for row in sheet.iter_rows(values_only=True):
print(row)

Example of data.xlsx:

name age city

John 30 New York

Jane 28 Los Angeles

Alice 25 Seattle

Output:

('name', 'age', 'city')

('John', 30, 'New York')
('Jane', 28, 'Los Angeles')
('Alice', 25, 'Seattle')

Writing to an Excel File:

from openpyxl import Workbook

# Create a new workbook and active sheet
wb = Workbook()
sheet = wb.active
# Data to write
data = [
 ("name", "age", "city"),
("John", 30, "New York"),
("Jane", 28, "Los Angeles"),
("Alice", 25, "Seattle")
]
# Write data to sheet
for row in data:
sheet.append(row)
# Save workbook to a file
wb.save('output.xlsx')

2. Reading and Writing Excel Files using pandas

pandas makes working with Excel files even simpler and more versatile.

Reading Excel File:

import pandas as pd
# Read Excel file into DataFrame
df = pd.read_excel('data.xlsx')
print(df)

Output:

name age city

0 John 30 New York
1 Jane 28 Los Angeles
2 Alice 25 Seattle

Writing to an Excel File:

import pandas as pd
# Create DataFrame
df = pd.DataFrame({
"name": ["John", "Jane", "Alice"],
"age": [30, 28, 25],
"city": ["New York", "Los Angeles", "Seattle"]
})

# Write DataFrame to Excel file

df.to_excel('output_pandas.xlsx', index=False)

3. Working with Multiple Sheets in Excel using pandas

You can also work with multiple sheets using pandas.

import pandas as pd
# Create two DataFrames for different sheets
df1 = pd.DataFrame({
"name": ["John", "Jane"],
"age": [30, 28],
"city": ["New York", "Los Angeles"]
})
df2 = pd.DataFrame({
"name": ["Alice", "Bob"],
"age": [25, 35],
"city": ["Seattle", "Chicago"]
})
# Write DataFrames to different sheets
with pd.ExcelWriter('multi_sheet_output.xlsx') as writer:
df1.to_excel(writer, sheet_name='Sheet1', index=False)
df2.to_excel(writer, sheet_name='Sheet2', index=False)

4. Modifying an Existing Excel File

You can read, modify, and save an Excel file using openpyxl or pandas.

Using openpyxl:

from openpyxl import load_workbook

# Load the existing workbook

wb = load_workbook('data.xlsx')

# Select the active sheet

sheet = wb.active

# Modify the value of a cell

sheet['A2'] = 'Michael' # Change 'John' to 'Michael'

# Save the modified workbook

wb.save('modified_data.xlsx')

Using pandas:

import pandas as pd
# Read the Excel file into DataFrame
df = pd.read_excel('data.xlsx')
# Modify a value (change 'John' to 'Michael')
df.loc[df['name'] == 'John', 'name'] = 'Michael'

# Save the modified DataFrame back to Excel

df.to_excel('modified_data_pandas.xlsx', index=False)

These examples demonstrate the basics of working with CSV and Excel files in Python. You can
expand on these concepts for more advanced use cases, such as handling large datasets,
formatting Excel files, or performing complex data manipulation using pandas.

8.Working with OS module commands

The os module in Python provides a way to interact with the operating system, perform file and
directory operations, and execute system-level commands. Here are some common operations
and commands you can use with the os module.

1. Working with Directories

1.1. Get Current Working Directory

You can use os.getcwd() to get the current working directory.

import os
# Get current working directory
current_dir = os.getcwd()
print("Current Directory:", current_dir)

1.2. Change Current Working Directory

Use os.chdir(path) to change the current working directory.

import os
# Change to a specific directory
os.chdir('/path/to/directory')
# Verify the current working directory
print("New Directory:", os.getcwd())

1.3. List Files in a Directory

Use os.listdir(path) to list all files and directories in a specified path.

import os
# List all files and directories in the current directory
files = os.listdir('.')
print("Files and Directories:", files)

1.4. Create a Directory

Use os.mkdir(path) to create a single directory.

import os
# Create a new directory
os.mkdir('new_folder')
# Verify the directory was created
print("Directory created:", os.path.isdir('new_folder'))

1.5. Remove a Directory

Use os.rmdir(path) to remove a directory. The directory must be empty.

import os
# Remove a directory
os.rmdir('new_folder')
# Verify the directory was removed
print("Directory removed:", not os.path.isdir('new_folder'))

2. File Operations

2.1. Check if a File or Directory Exists

Use os.path.exists(path) to check if a file or directory exists.

import os
# Check if a file exists
if os.path.exists('file.txt'):
print("File exists")
else:
print("File does not exist")

2.2. Get File Information

Use os.stat(path) to get details like file size, creation time, and last modified time.

import os
# Get file information
file_info = os.stat('file.txt')
# Print file size in bytes
print("File size:", file_info.st_size)

2.3. Rename a File or Directory

Use os.rename(old_name, new_name) to rename a file or directory.

import os
# Rename a file
os.rename('old_file.txt', 'new_file.txt')
2.4. Remove a File

Use os.remove(path) to delete a file.

import os
# Remove a file
os.remove('file_to_delete.txt')

3. Executing System Commands

You can use os.system() to execute a shell command.

import os
# Run a shell command (list files in the current directory)
os.system('ls')

3.1. Using os.system() for Operating System Commands

import os
# Execute a command to list files on Windows
os.system('dir') # For Windows
# Execute a command to list files on Linux/macOS
# os.system('ls') # For Linux/macOS

3.2. Run a Command and Get Output

To capture the output of a command, use os.popen() or the subprocess module (for more
advanced use cases).

import os
# Run a command and capture output
output = os.popen('echo Hello, World!').read()
print("Command Output:", output)
4. Working with Environment Variables

4.1. Get Environment Variables

You can use os.environ to access environment variables.

import os
# Get an environment variable
path = os.environ.get('PATH')
print("PATH Environment Variable:", path)

4.2. Set Environment Variables

You can use os.environ to set environment variables.

import os
# Set an environment variable
os.environ['MY_VAR'] = 'Some Value'
# Access the newly set environment variable
print("MY_VAR:", os.environ.get('MY_VAR'))

5. Working with Paths

5.1. Join Paths

Use os.path.join() to join directory paths in a platform-independent way.

import os
# Join paths
full_path = os.path.join('folder', 'subfolder', 'file.txt')
print("Full Path:", full_path)
5.2. Get Absolute Path

Use os.path.abspath(path) to get the absolute path of a file or directory.

import os
# Get absolute path of a file
abs_path = os.path.abspath('file.txt')
print("Absolute Path:", abs_path)

5.3. Split Paths

Use os.path.split() to split a path into the directory and the filename.

import os
# Split path into directory and filename
dir_path, file_name = os.path.split('/path/to/file.txt')
print("Directory:", dir_path)
print("File Name:", file_name)

6. Temporary Files

6.1. Create a Temporary File

The tempfile module is often used in combination with os to create and manage temporary
files.

import os
import tempfile
# Create a temporary file
with tempfile.TemporaryFile() as temp_file:
temp_file.write(b'Hello, Temporary File!')
temp_file.seek(0)
 print(temp_file.read())

7. Process Management

7.1. Get Process ID

Use os.getpid() to get the process ID (PID) of the current process.

import os
# Get process ID
pid = os.getpid()
print("Process ID:", pid)

7.2. Get Parent Process ID

Use os.getppid() to get the parent process ID (PPID).

import os
# Get parent process ID
ppid = os.getppid()
print("Parent Process ID:", ppid)

8. File Permissions

8.1. Change File Permissions

Use os.chmod(path, mode) to change the permissions of a file.

import os
# Change file permissions (e.g., make a file read-only)
os.chmod('file.txt', 0o444) # Read-only permission
9. Check Disk Usage

You can check the disk usage of a directory using os.statvfs().

import os
# Check disk usage for the current directory
stat = os.statvfs('/')
print("Disk Block Size:", stat.f_frsize)
print("Total Blocks:", stat.f_blocks)

These are just some of the key functionalities provided by the os module for interacting with the
operating system in Python. The os module is extremely versatile and useful for file and
directory management, running shell commands, process management, and more.

9. OS modules, glob modules etc

The os module and glob module are essential in Python for file management, path operations,
and pattern matching of filenames. Let's explore these modules in more detail.

1. OS Module

The os module provides a way to interact with the operating system. Here are some common
functionalities:

1.1. File and Directory Operations

You can use os to manipulate files and directories.

Create a directory:

import os
# Create a new directory
os.mkdir('new_directory')

Remove a directory:

import os
# Remove an empty directory
os.rmdir('new_directory')

List contents of a directory:

import os
# List files and directories in the current directory
files = os.listdir('.')
print(files)

Remove a file:

import os
# Remove a file
os.remove('file_to_delete.txt')

1.2. Path Operations

Join paths:

import os
# Join two paths
path = os.path.join('folder', 'subfolder', 'file.txt')
print(path)
Get the absolute path:

import os
# Get absolute path of a file
abs_path = os.path.abspath('file.txt')
print(abs_path)

Check if a path exists:

import os
# Check if the path exists
if os.path.exists('file.txt'):
print("File exists")
else:
print("File does not exist")

1.3. Environment Variables

Get an environment variable:

import os
# Get the value of an environment variable
path = os.environ.get('PATH')
print(path)

Set an environment variable:

import os
# Set a new environment variable
os.environ['MY_VAR'] = 'Some Value'
print(os.environ.get('MY_VAR'))
2. Glob Module

The glob module finds all the pathnames matching a specified pattern. It is mainly used for
filename pattern matching.

2.1. Basic Usage of glob

The glob.glob() function returns a list of filenames that match a specified pattern.

import glob
# Get all .txt files in the current directory
txt_files = glob.glob('*.txt')
print(txt_files)

Output:

['file1.txt', 'file2.txt', 'document.txt']

2.2. Matching Files in Subdirectories

You can use glob to match files in subdirectories by using the ** pattern.

import glob
# Get all .txt files in the current directory and all its
subdirectories
txt_files = glob.glob('**/*.txt', recursive=True)
print(txt_files)

Output:

['folder/file1.txt', 'folder/subfolder/file2.txt']

2.3. Wildcard Matching

Use wildcards (*, ?, []) to match files or directories.

import glob
# Match all files that start with 'data' and end with '.csv'
csv_files = glob.glob('data*.csv')
print(csv_files)

Output:

['data1.csv', 'data2.csv', 'data_final.csv']

3. Combining os and glob Modules

You can combine os and glob to perform more advanced operations like iterating through files
and applying certain actions.

3.1. Using os and glob to Delete Specific Files

For example, you can use glob to find all files matching a pattern and os.remove() to delete
them.

import os
import glob
# Find all .tmp files in the current directory
tmp_files = glob.glob('*.tmp')
# Delete each .tmp file
for file in tmp_files:
os.remove(file)
print(f"Deleted {file}")

3.2. Using glob to List Files and os.path for More Details
You can use glob to get a list of files and os.path functions (like os.path.getsize())
to retrieve additional information.

import os
import glob
# List all .txt files
txt_files = glob.glob('*.txt')
# Print the size of each .txt file
for file in txt_files:
size = os.path.getsize(file)
print(f"{file}: {size} bytes")

4. Other Common Operations with OS and Glob Modules

4.1. Checking File Permissions

You can use os.access() to check file permissions before performing file operations.

import os
# Check if a file is readable and writable
file_path = 'file.txt'
if os.access(file_path, os.R_OK | os.W_OK):
print(f"{file_path} is readable and writable.")
else:
print(f"{file_path} is not readable or writable.")

4.2. Matching Files with Specific Patterns Using glob

You can use glob to find files with specific patterns, such as files modified today or files with
specific extensions.
import glob
import os
import time
# Get all .log files modified today
today = time.time() - 24*60*60 # 24 hours ago
log_files = glob.glob('*.log')
for file in log_files:
if os.path.getmtime(file) > today:
print(f"{file} was modified today")

5. Working with Temporary Files

The os module can be used in combination with tempfile to create and manage temporary
files.

import os
import tempfile
# Create a temporary file
with tempfile.NamedTemporaryFile(delete=False) as tmp_file:
print(f"Temporary file created: {tmp_file.name}")
tmp_file.write(b"Some data")

6. Executing Shell Commands with os.system()

You can use os.system() to execute shell commands directly from Python.

import os
# Execute a shell command
os.system('echo "Hello, world!"')
However, note that os.system() is basic and does not capture the output. For more advanced
needs, use subprocess.

7. Platform-Specific Operations

The os module provides platform-specific operations. For instance, checking the operating
system:

import os
# Get the platform
print(os.name) # 'posix' for Linux/MacOS, 'nt' for Windows

You can also execute OS-specific commands:

import os
if os.name == 'nt':
os.system('cls') # Clear the screen for Windows
else:
os.system('clear') # Clear the screen for Linux/MacOS

Summary

● os module: Useful for handling file and directory operations, interacting with
environment variables, running shell commands, and more.
● glob module: Used for matching file patterns using wildcards. It's great for working
with filenames in directories.
● Combining os and glob: You can combine both modules to perform tasks like batch
file deletions, file size calculations, or even iterating over files and directories.
 Module 15 - Exceptions
1. Errors in a Python Program

Exception handling in Python allows you to handle runtime errors gracefully instead of letting
the program crash. Python uses try, except, else, and finally blocks to handle
exceptions effectively.

Common Errors in Python Programs

1. Syntax Errors
○ Occur due to incorrect syntax.
○ Example: Missing a colon at the end of a loop.
for i in range(5) # Missing colon

print(i)

○ Fix: Correct the syntax.

2. Indentation Errors
○ Python enforces indentation; improper alignment causes errors.
○ Example:
def greet():

print("Hello!") # Incorrect indentation

○ Fix: Align the code properly.

3. Type Errors
○ When operations are performed on incompatible data types.
○ Example:

print("Age: " + 25) # Can't concatenate str and int

○ Fix: Convert the data type or use formatted strings.

print(f"Age: {25}")
 4. Name Errors
○ Occur when a variable or function is used before declaration.
○ Example:

print(name) # name is not defined

○ Fix: Define the variable before using it.

5. Index Errors
○ Occur when accessing a list or string index out of range.
○ Example:
numbers = [1, 2, 3]

print(numbers[5]) # Index out of range

○ Fix: Check the length before accessing the index.

6. Key Errors
○ Raised when a dictionary key doesn't exist.
○ Example:

data = {"name": "Alice"}

print(data["age"]) # KeyError: 'age'

○ Fix: Use get() to safely access keys.

7. Value Errors
○ Raised when a function gets an argument of the right type but inappropriate value.
○ Example:
int("abc") # ValueError
8. ZeroDivisionError
○ Division by zero is not allowed.
○ Example:
print(10 / 0) # ZeroDivisionError
Exception Handling in Python

Syntax:

try:
# Code that may raise an exception
except ExceptionType:
# Code to handle the exception
else:
# Code to execute if no exception occurs
finally:
# Code to execute regardless of an exception

Example with Multiple Exceptions

try:
num = int(input("Enter a number: "))
result = 10 / num
print("Result:", result)
except ValueError:
print("Invalid input! Please enter a number.")
except ZeroDivisionError:
print("Division by zero is not allowed.")
else:
print("Calculation was successful!")
finally:
print("End of program.")

Best Practices

1. Handle specific exceptions instead of using a generic except.

 2. Use finally to clean up resources (e.g., closing a file or database connection).
3. Avoid using bare except: unless absolutely necessary.
4. Log errors for debugging.

2. Exceptions

Exceptions in Python are errors detected during execution. They disrupt the normal flow of a
program and can be managed using exception handling to avoid crashes and handle errors
gracefully.

Built-in Exceptions

Python provides a variety of built-in exceptions, including:

1. ArithmeticError
○ Base class for arithmetic-related errors.
○ Examples: ZeroDivisionError, OverflowError.
2. IndexError
○ Raised when accessing an invalid index of a sequence (e.g., list, string).
3. KeyError
○ Raised when trying to access a non-existent key in a dictionary.
4. ValueError
○ Raised when an operation receives an argument of the correct type but
inappropriate value.
5. TypeError
○ Raised when an operation is applied to an object of inappropriate type.
6. NameError
○ Raised when a variable or name is not defined.
7. FileNotFoundError
○ Raised when a file operation is requested, but the file does not exist.
8. EOFError
 ○ Raised when the input() function hits end-of-file condition.
9. ImportError
○ Raised when an import statement fails to find the module.
10. RuntimeError
○ Raised when an error does not fall under other specific categories.

Basic Syntax of Exception Handling

You can handle exceptions using try, except, else, and finally blocks.

Structure

try:
# Code that might raise an exception
except ExceptionType:
# Code to handle the exception
else:
# Code to execute if no exception occurs
finally:
# Code to execute no matter what (optional)

Examples

1. Single Exception Handling

try:
x = int(input("Enter a number: "))
print(10 / x)
except ZeroDivisionError:
print("You cannot divide by zero.")

2. Multiple Exceptions

try:
 x = int(input("Enter a number: "))
print(10 / x)
except ZeroDivisionError:
print("Cannot divide by zero!")
except ValueError:
print("Invalid input! Enter a valid number.")

3. Generic Exception

try:
x = int(input("Enter a number: "))
print(10 / x)
except Exception as e:
print(f"An error occurred: {e}")

4. else and finally Usage

try:
x = int(input("Enter a number: "))
print(10 / x)
except ZeroDivisionError:
print("Division by zero is not allowed.")
else:
print("Operation successful!")
finally:
print("This block always executes.")

5. Raising Exceptions

You can raise exceptions manually using the raise statement.

x = -1
if x < 0:
raise ValueError("Negative value not allowed!")

6. Custom Exceptions

Define your custom exceptions by creating a subclass of the Exception class.

class NegativeNumberError(Exception):
pass
try:
x = int(input("Enter a positive number: "))
if x < 0:
raise NegativeNumberError("Negative numbers are not
allowed.")
except NegativeNumberError as e:
print(e)

Key Points

1. Order of Exception Handling Matters

Handle specific exceptions first and generic ones later to avoid overshadowing.
2. Best Practices
○ Use specific exceptions wherever possible.
○ Avoid bare except: as it catches all exceptions, including unexpected ones.
○ Use finally for cleanup operations like closing files or database connections.

Access Exception Information Use as to capture the exception object:

except ValueError as e:

3. print("Error:", e)
3. Exception Handling

Exception handling in Python allows programs to deal with runtime errors gracefully, ensuring
the program doesn’t crash unexpectedly and can recover or exit cleanly. It uses a combination of
try, except, else, and finally blocks.

Why Use Exception Handling?

1. To prevent the program from crashing on encountering errors.

2. To handle errors gracefully, providing meaningful messages to the user.
3. To perform cleanup operations like closing files or database connections.

Basic Syntax

try:
# Code that might raise an exception
except ExceptionType:
# Code to handle the exception
else:
# Code to execute if no exception occurs
finally:
# Code to execute no matter what

Key Blocks in Exception Handling

1. try Block
Contains the code that might raise an exception.
2. except Block
Handles specific or general exceptions. Multiple except blocks can handle different
error types.
3. else Block (Optional)
Executes only if the try block does not raise any exceptions.
 4. finally Block (Optional)
Executes regardless of whether an exception was raised or not. Often used for cleanup.

Best Practices
Handle Specific Exceptions
Always catch specific exceptions rather than a generic one, unless necessary.
except ZeroDivisionError:
...
Avoid Bare except:
It catches all exceptions, including unexpected ones like KeyboardInterrupt.
# Avoid this:
except:
print("Something went wrong!")
Use finally for Cleanup
Close files, release resources, or perform any necessary cleanup.
try:
file = open("example.txt", "r")
# Perform operations
finally:
file.close()
Log Exceptions
Use logging to capture errors for debugging instead of printing them.
import logging

try:
...
except Exception as e:
logging.error(f"Error occurred: {e}")
Rethrow Exceptions if Necessary
If you cannot handle the exception fully, re-raise it for higher-level handling.
try:
...
except Exception:
raise

4. Types of Exceptions

Types of Exceptions in Python

Python provides a rich set of built-in exceptions to handle various types of errors. These
exceptions can be categorized based on their functionality and usage.

1. Arithmetic-Related Exceptions

● ArithmeticError
Base class for all arithmetic-related errors.

ZeroDivisionError
Raised when dividing by zero.
print(10 / 0) # ZeroDivisionError
OverflowError
Raised when a calculation exceeds the maximum limit for a numeric type.
import math
print(math.exp(1000)) # OverflowError

● FloatingPointError
Raised when a floating-point operation fails (rare in modern Python).
2. Indexing and Key Errors

IndexError
Raised when accessing an invalid index in a sequence.
my_list = [1, 2, 3]
print(my_list[5]) # IndexError
KeyError
Raised when accessing a non-existent key in a dictionary.
my_dict = {"a": 1}
print(my_dict["b"]) # KeyError

3. Value-Related Exceptions

ValueError
Raised when a function receives an argument of the correct type but inappropriate value.
print(int("abc")) # ValueError
TypeError
Raised when an operation is applied to an object of inappropriate type.
print("5" + 5) # TypeError

4. Input/Output Exceptions

FileNotFoundError
Raised when trying to access a file that does not exist.
open("non_existent_file.txt", "r") # FileNotFoundError
EOFError
Raised when the input() function hits the end of file.
while True:
input() # Press Ctrl+D to raise EOFError

PermissionError
Raised when trying to perform an operation without the necessary permissions.
5. Import Errors

ImportError
Raised when an import statement fails.
import non_existent_module # ImportError
ModuleNotFoundError
A subclass of ImportError, specifically for missing modules.
import nonexistentmodule # ModuleNotFoundError

6. Name-Related Exceptions

NameError
Raised when a variable or function is not defined.
print(undeclared_variable) # NameError

● UnboundLocalError
A subclass of NameError, raised when trying to use a local variable before assigning it.

7. OS-Related Exceptions

● OSError
Base class for errors related to system operations.

FileExistsError
Raised when trying to create a file or directory that already exists.
import os
os.mkdir("existing_dir") # FileExistsError

● IsADirectoryError
Raised when a directory is used in place of a file.
● NotADirectoryError
Raised when a file is used in place of a directory.
8. Attribute and Lookup Errors

AttributeError
Raised when an invalid attribute is accessed.
my_list = [1, 2, 3]
my_list.append(4)
my_list.nonexistent_method() # AttributeError

● LookupError
Base class for errors related to invalid lookups.
○ Subclasses: IndexError, KeyError.

9. Memory-Related Exceptions

● MemoryError
Raised when an operation runs out of memory.

RecursionError
Raised when the maximum recursion depth is exceeded.
def recursive():
recursive()
recursive() # RecursionError

10. Assertion and Runtime Exceptions

AssertionError
Raised when an assert statement fails.
assert 2 + 2 == 5, "Math is broken!" # AssertionError

● RuntimeError
Generic error when no other specific exception applies.
 ● NotImplementedError
Raised when a method is not implemented in a subclass.

11. Custom Exceptions

You can create custom exceptions by subclassing the Exception class.

class CustomError(Exception):
pass
try:
raise CustomError("This is a custom exception!")
except CustomError as e:
print(e)

Hierarchy of Exceptions

All exceptions in Python inherit from the base class BaseException. The hierarchy looks like
this:

BaseException
├── SystemExit
├── KeyboardInterrupt
├── GeneratorExit
└── Exception
├── ArithmeticError
│ ├── ZeroDivisionError
│ ├── OverflowError
│ └── FloatingPointError
├── LookupError
 │ ├── IndexError
│ └── KeyError
├── OSError
│ ├── FileNotFoundError
│ ├── PermissionError
│ └── ...
├── ImportError
│ └── ModuleNotFoundError
├── ...

5. The Except Block( Use of try except & finally)

The except Block in Python

The except block is used in Python to handle exceptions that occur in the try block. It
ensures the program can handle errors gracefully without crashing.

Using try, except, and finally

● try Block: Contains code that may raise an exception.

● except Block: Handles specific or general exceptions.
● finally Block: Executes regardless of whether an exception occurs or not, often used
for cleanup.

Basic Syntax

try:
# Code that may raise an exception
except ExceptionType:
 # Code to handle the exception
finally:
# Code that always executes

Examples

1. Handling Specific Exceptions

try:
result = 10 / 0
except ZeroDivisionError:
print("Error: Division by zero is not allowed.")
finally:
print("Execution of the `finally` block.")

Output:

Error: Division by zero is not allowed.

Execution of the `finally` block.

2. Handling Multiple Exceptions

You can catch multiple exceptions by using multiple except blocks.

try:
num = int(input("Enter a number: "))
print(10 / num)
except ZeroDivisionError:
print("Cannot divide by zero!")
except ValueError:
print("Invalid input! Please enter a number.")
finally:
 print("Cleanup or final message.")

3. Handling Any Exception

Use a generic except block to catch all exceptions. Caution: This should only be used when
you genuinely need to handle all exceptions.

try:
print("Performing operation...")
result = 10 / "five"
except Exception as e:
print(f"An error occurred: {e}")
finally:
print("This block always executes.")

Output:

An error occurred: unsupported operand type(s) for /: 'int' and

'str'
This block always executes.

4. Using else with try

The else block executes if no exceptions occur in the try block.

try:
result = 10 / 2
except ZeroDivisionError:
print("Division by zero is not allowed.")
else:
print(f"The result is: {result}")
finally:
 print("Execution complete.")

Output:

The result is: 5.0

Execution complete.

5. Cleanup Operations with finally

The finally block is often used to release resources, such as closing files or database
connections.

try:
file = open("example.txt", "r")
# Perform file operations
except FileNotFoundError:
print("File not found.")
finally:
print("Closing the file.")
if 'file' in locals() and not file.closed:
file.close()

Key Notes on finally

1. The finally block always executes, even if there’s an exception or a return

statement in the try or except blocks.
2. If an exception occurs but is not handled, the finally block executes before the
exception propagates.

Use Cases

1. Error Handling: Prevent the program from crashing due to unhandled exceptions.
 2. Resource Cleanup: Close files, network connections, or release locks.
3. Logging: Log errors or critical messages.
4. Guaranteed Execution: Ensure specific code runs no matter what.

6. The assert Statement

The assert statement is used for debugging purposes. It helps you test assumptions in your
code and raises an AssertionError if the condition provided is False. Assertions are
typically used during development to catch bugs early but are not meant for handling runtime
errors in production.

Syntax

assert condition, optional_message

● condition: A boolean expression. If False, it raises an AssertionError.

● optional_message: A message displayed when the assertion fails, making
debugging easier.

Basic Examples

1. Assertion Without a Message

x = 10
assert x > 5 # Passes because 10 > 5
assert x < 5 # Raises AssertionError

Output:

Traceback (most recent call last):

...
AssertionError
2. Assertion With a Message

x = 10
assert x < 5, "x is not less than 5"

Output:

Traceback (most recent call last):

...
AssertionError: x is not less than 5

How It Works

1. If the condition evaluates to True, the program continues execution normally.

2. If the condition evaluates to False, Python raises an AssertionError with the
optional message (if provided).

Usage Scenarios

1. Debugging

Assertions can help ensure certain conditions hold true while testing.

def divide(a, b):

assert b != 0, "Denominator cannot be zero"
return a / b

print(divide(10, 2)) # Works

print(divide(10, 0)) # Raises AssertionError

2. Validating Function Inputs

def factorial(n):
 assert n >= 0, "Factorial is not defined for negative
numbers"
if n == 0:
return 1
return n * factorial(n - 1)

print(factorial(5)) # 120
print(factorial(-1)) # Raises AssertionError

3. Ensuring Invariants

x = 10
y = 5
assert x > y, "x should always be greater than y"

Disabling Assertions

Assertions can be disabled globally when running Python in optimized mode (using the -O flag).

Example

python -O script.py

● This ignores all assert statements in the code, making it suitable for production where
assertions are unnecessary.

Key Points

1. Not for Error Handling: Assertions are for catching bugs during development, not for
handling user input or runtime errors.
○ Use exceptions like ValueError or TypeError for user-facing error
handling.
 2. Performance Impact: Assertions are removed in optimized mode, so they should not be
relied upon for critical functionality.
3. Custom AssertionError Messages: Always include meaningful messages to make
debugging easier.

Best Practices

1. Use assert for conditions that should never fail in a correctly functioning program.
2. Avoid using assert for validating external inputs (like user input or file data).
3. Remove or disable assertions in production for better performance.

7. User-Defined Exceptions

In Python, you can create your own exceptions by subclassing the built-in Exception class (or
any of its subclasses). This is useful when you want to define application-specific error handling.

How to Create User-Defined Exceptions

1. Define a new class that inherits from the Exception class.

2. Optionally, you can add custom attributes or methods to the class to provide more context
about the exception.

Basic Syntax
class CustomException(Exception):
pass

Examples

1. A Simple User-Defined Exception

class CustomError(Exception):
pass
try:
raise CustomError("This is a custom exception.")
except CustomError as e:
print(e)

Output:

This is a custom exception.

2. User-Defined Exception with Custom Attributes

class InsufficientFundsError(Exception):
def __init__(self, balance, amount):
self.balance = balance
self.amount = amount
super().__init__(f"Insufficient funds: Balance =
{balance}, Amount = {amount}")

# Example Usage
try:
balance = 100
withdraw_amount = 200
if withdraw_amount > balance:
raise InsufficientFundsError(balance, withdraw_amount)
except InsufficientFundsError as e:
print(e)

Output:

Insufficient funds: Balance = 100, Amount = 200

3. Exception with Custom Methods

class InvalidAgeError(Exception):
def __init__(self, age):
self.age = age

def age_message(self):
return f"Invalid age: {self.age}. Age must be between 0
and 120."

# Example Usage
try:
age = -5
if age < 0 or age > 120:
raise InvalidAgeError(age)
except InvalidAgeError as e:
print(e.age_message())

Output:

Invalid age: -5. Age must be between 0 and 120.

4. Creating a Hierarchy of Exceptions

You can define a base exception class and create multiple subclasses for specific types of errors.

class ApplicationError(Exception):
"""Base class for all application-related errors."""
pass

class DatabaseError(ApplicationError):
 """Exception raised for database-related errors."""
pass

class ValidationError(ApplicationError):
"""Exception raised for validation-related errors."""
pass

# Example Usage
try:
raise DatabaseError("Database connection failed.")
except DatabaseError as e:
print(f"Database Error: {e}")
except ValidationError as e:
print(f"Validation Error: {e}")

Output:

Database Error: Database connection failed.

Best Practices

1. Meaningful Names: Name your exceptions clearly to reflect their purpose.

2. Inherit from Exception: This ensures compatibility with Python's exception handling
mechanism.
3. Add Context: Include attributes or methods to provide additional information about the
error.
4. Document Exceptions: Make it clear when and why each custom exception might be
raised.
When to Use User-Defined Exceptions

● When built-in exceptions do not clearly express the problem.

● To create domain-specific exceptions (e.g., InsufficientFundsError for a
banking app).
● To improve code readability and maintainability.

EXAMPLE
Core Classes:

● Book represents a book with attributes title, author, and is_borrowed.

● User represents a library user with methods to borrow and return books.
● Library manages the collection of books and registered users.

User-Defined Exceptions:

● BookNotAvailableError for unavailable books.

● UserNotRegisteredError for unregistered users.
● OverdueLimitError for users with too many overdue books.

Built-In Exceptions:

● KeyError when trying to access a book that doesn’t exist.

● Generic Exception to catch any unexpected issues.

Flow:

● Books and users are added to the library.

● Borrowing and returning books involves error handling for various scenarios.

Static Methods:

● For utility tasks, such as checking the total number of books in the library.

Encapsulation:
 ● Use private attributes for sensitive data.
● Provide controlled access using properties.

Properties:

● For attributes like overdue_books or borrowed_books to add custom validation

or logic.

# User-Defined Exceptions

class BookNotAvailableError(Exception):

"""Raised when the requested book is not available."""

def __init__(self, book_title):

super().__init__(f"The book '{book_title}' is currently

unavailable.")

class UserNotRegisteredError(Exception):

"""Raised when an unregistered user tries to borrow a

book."""

def __init__(self, user_name):

super().__init__(f"User '{user_name}' is not registered

in the library.")

class OverdueLimitError(Exception):

"""Raised when a user has too many overdue books."""

def __init__(self, user_name, overdue_count):

super().__init__(f"User '{user_name}' has

{overdue_count} overdue books and cannot borrow more.")

# Classes for the Library System

class Book:

def __init__(self, title, author):

self.title = title

self.author = author

self._is_borrowed = False # Encapsulated attribute

@property

def is_borrowed(self):

"""Property to get the borrowed status."""

return self._is_borrowed

@is_borrowed.setter

def is_borrowed(self, value):

"""Property to set the borrowed status."""

if not isinstance(value, bool):

raise ValueError("is_borrowed must be a boolean

value.")

self._is_borrowed = value

def __str__(self):

return f"{self.title} by {self.author}"

class User:
 def __init__(self, name):

self.name = name

self._borrowed_books = [] # Encapsulated attribute

self._overdue_books = 0 # Encapsulated attribute

@property

def borrowed_books(self):

"""Property to get the list of borrowed books."""

return self._borrowed_books

@property

def overdue_books(self):

"""Property to get the overdue book count."""

return self._overdue_books

@overdue_books.setter

def overdue_books(self, value):

"""Property to set the overdue book count."""

if value < 0:

raise ValueError("Overdue books count cannot be

negative.")

self._overdue_books = value

def borrow_book(self, book):

 if book.is_borrowed:

raise BookNotAvailableError(book.title)

self._borrowed_books.append(book)

book.is_borrowed = True

def return_book(self, book):

if book in self._borrowed_books:

self._borrowed_books.remove(book)

book.is_borrowed = False

def __str__(self):

return f"User: {self.name}, Borrowed Books: {[book.title

for book in self._borrowed_books]}"

class Library:

def __init__(self):

self._books = {} # Encapsulated attribute

self._users = {} # Encapsulated attribute

@staticmethod

def total_books(library):

"""Static method to get the total number of books."""

return len(library._books)
 def add_book(self, book):

self._books[book.title] = book

def register_user(self, user):

self._users[user.name] = user

def borrow_book(self, user_name, book_title):

# Check if the user is registered

if user_name not in self._users:

raise UserNotRegisteredError(user_name)

# Check if the book exists

if book_title not in self._books:

raise KeyError(f"The book '{book_title}' does not

exist in the library.")

user = self._users[user_name]

book = self._books[book_title]

# Check overdue limit

if user.overdue_books > 3:

raise OverdueLimitError(user_name,
user.overdue_books)
 # Borrow the book

user.borrow_book(book)

print(f"{user_name} successfully borrowed

'{book_title}'.")

def return_book(self, user_name, book_title):

# Check if the user is registered

if user_name not in self._users:

raise UserNotRegisteredError(user_name)

# Check if the book exists

if book_title not in self._books:

raise KeyError(f"The book '{book_title}' does not

exist in the library.")

user = self._users[user_name]

book = self._books[book_title]

# Return the book

user.return_book(book)

print(f"{user_name} successfully returned

'{book_title}'.")

# Example Usage
try:

# Create a library instance

library = Library()

# Add books to the library

library.add_book(Book("1984", "George Orwell"))

library.add_book(Book("To Kill a Mockingbird", "Harper

Lee"))

library.add_book(Book("The Great Gatsby", "F. Scott

Fitzgerald"))

# Register users

library.register_user(User("Alice"))

library.register_user(User("Bob"))

# Display total books in the library

print(f"Total books in the library:

{Library.total_books(library)}")

# Borrow books

library.borrow_book("Alice", "1984")

library.borrow_book("Bob", "The Great Gatsby")

# Try borrowing a book that's already borrowed

library.borrow_book("Alice", "1984")
except BookNotAvailableError as e:

print(e)

except UserNotRegisteredError as e:

print(e)

except OverdueLimitError as e:

print(e)

except KeyError as e:

print(f"Error: {e}")

except ValueError as e:

print(f"ValueError: {e}")

except Exception as e:

print(f"An unexpected error occurred: {e}")

finally:

print("\nLibrary System operation completed.")

Sample Output
Total books in the library: 3

Alice successfully borrowed '1984'.

Bob successfully borrowed 'The Great Gatsby'.

The book '1984' is currently unavailable.

Library System operation completed.

 Module 16 - Files in Python
1. Open a file, Reading , writing & Closing of a File

Here's an example demonstrating how to open, read, write, and close a file in Python:

# Writing to a file
file_path = "example.txt"

# Open the file in write mode

with open(file_path, "w") as file:
file.write("Hello, this is the first line.\n")
file.write("This is the second line.")

print(f"Data written to {file_path} successfully.")

# Reading from a file

# Open the file in read mode
with open(file_path, "r") as file:
content = file.read()

print("\nContents of the file:")

print(content)

# Appending to a file
# Open the file in append mode
with open(file_path, "a") as file:
file.write("\nThis is an additional line appended to the
file.")
print("\nNew data appended successfully.")

# Reading the updated file content

with open(file_path, "r") as file:
updated_content = file.read()

print("\nUpdated contents of the file:")

print(updated_content)

# Closing the file is automatically handled by the 'with'

statement

Explanation:

1. Writing (w mode):
○ Opens the file for writing. Creates the file if it doesn't exist, or truncates the file if
it does.
○ Use file.write() to write data to the file.
2. Reading (r mode):
○ Opens the file for reading. Throws an error if the file does not exist.
○ Use file.read() to read the entire file content.
3. Appending (a mode):
○ Opens the file for appending. Creates the file if it doesn't exist.
○ Adds data to the end of the file without overwriting the existing content.
4. Automatic File Closing:
○ Using the with open(...) construct ensures that the file is properly closed
after the block is executed, even if an error occurs.
Output:

Data written to example.txt successfully.

Contents of the file:

Hello, this is the first line.
This is the second line.

New data appended successfully.

Updated contents of the file:

Hello, this is the first line.
This is the second line.
This is an additional line appended to the file.

2. Working with creation of temp files

Python provides a convenient tempfile module to create temporary files. These files are
useful for storing data temporarily and are automatically deleted when closed or when the
program exits. Here's how you can create and work with temporary files:

Example Code: Using Temporary Files

import tempfile

# Create a temporary file

with tempfile.NamedTemporaryFile(mode='w+', delete=False) as
temp_file:
print(f"Temporary file created: {temp_file.name}")
 # Write data to the temp file
temp_file.write("This is some temporary data.\n")
temp_file.write("Temporary files are auto-deleted.\n")

# Move the file cursor to the beginning to read

temp_file.seek(0)

# Read the data

print("Contents of the temporary file:")
print(temp_file.read())

# Note: The 'delete=False' means the file is not automatically

deleted when closed.
# To delete manually:
import os
os.remove(temp_file.name)
print(f"Temporary file {temp_file.name} deleted.")

Explanation:

1. tempfile.NamedTemporaryFile:
○ Creates a temporary file with a unique name.
○ You can use mode='w+' for read/write access.
○ The delete=False parameter ensures the file persists after closing, so it can
be manually deleted later.
2. Attributes:
○ temp_file.name: Provides the full path to the temporary file.
3. Automatic Cleanup:
 ○ If delete=True (default), the temporary file is automatically deleted when the
file is closed.

Example Output:

Temporary file created: /tmp/tmpxlh12z9k

Contents of the temporary file:
This is some temporary data.
Temporary files are auto-deleted.
Temporary file /tmp/tmpxlh12z9k deleted.

Temporary Directory Creation:

You can also create temporary directories using tempfile.TemporaryDirectory:

with tempfile.TemporaryDirectory() as temp_dir:

print(f"Temporary directory created: {temp_dir}")
# You can create files or folders inside this directory

# The directory and its contents are deleted when the block
ends.

This ensures you can safely handle temporary data without manually cleaning up.

3. Working with Text Files Containing Strings

Working with text files containing strings is a common task in Python. Here’s a breakdown of
how to handle reading, writing, appending, and searching strings in a text file:

1. Writing Strings to a Text File

file_name = "example.txt"
# Open the file in write mode
with open(file_name, "w") as file:
file.write("Hello, World!\n")
file.write("Python makes file handling easy.\n")
file.write("This file contains strings for testing
purposes.\n")

print(f"Strings written to {file_name} successfully.")

2. Reading Strings from a Text File

# Open the file in read mode

with open(file_name, "r") as file:
content = file.read()

print("\nContents of the file:")

print(content)

3. Appending Strings to a Text File

# Open the file in append mode

with open(file_name, "a") as file:
file.write("Appending more data to the file.\n")

print("\nNew strings appended successfully.")

4. Searching for a String in a Text File

search_term = "Python"
# Open the file in read mode
with open(file_name, "r") as file:
lines = file.readlines()

print(f"\nSearching for '{search_term}' in the file:")

for line_number, line in enumerate(lines, start=1):
if search_term in line:
print(f"Found on line {line_number}: {line.strip()}")

5. Counting the Occurrences of a String

target_word = "file"
count = 0

# Open the file in read mode

with open(file_name, "r") as file:
for line in file:
count += line.lower().count(target_word.lower())

print(f"\nThe word '{target_word}' appears {count} times in the

file.")

6. Replacing a String in a Text File

old_string = "file"
new_string = "document"

# Read the content and replace strings

with open(file_name, "r") as file:
content = file.read()
content = content.replace(old_string, new_string)

# Write the updated content back to the file

with open(file_name, "w") as file:
file.write(content)

print(f"\nReplaced '{old_string}' with '{new_string}' in

{file_name}.")

Complete Example with Output

1. Write strings: Creates a file and adds initial content.

2. Read strings: Displays the file content.
3. Append strings: Adds more content to the file.
4. Search strings: Finds and displays specific text.
5. Count occurrences: Counts how often a word appears.
6. Replace strings: Substitutes one string with another.

Sample Output

Strings written to example.txt successfully.

Contents of the file:

Hello, World!
Python makes file handling easy.
This file contains strings for testing purposes.

New strings appended successfully.

Searching for 'Python' in the file:
Found on line 2: Python makes file handling easy.

The word 'file' appears 3 times in the file.

Replaced 'file' with 'document' in example.txt.

4. Knowing Whether a File Exists or Not

To check if a file exists in Python, you can use the os.path module or the pathlib module.
Both methods are straightforward and effective. Here are examples:

Using os.path

import os

file_name = "example.txt"

# Check if the file exists

if os.path.exists(file_name):
print(f"The file '{file_name}' exists.")
else:
print(f"The file '{file_name}' does not exist.")

Using pathlib

from pathlib import Path

file_path = Path("example.txt")
# Check if the file exists
if file_path.exists():
print(f"The file '{file_path}' exists.")
else:
print(f"The file '{file_path}' does not exist.")

Additional Checks

Check if it’s a file or a directory (using os.path):

if os.path.isfile(file_name):
print(f"'{file_name}' is a file.")
elif os.path.isdir(file_name):
print(f"'{file_name}' is a directory.")
else:
print(f"'{file_name}' does not exist.")
Check if it’s a file or a directory (using pathlib):
if file_path.is_file():
print(f"'{file_path}' is a file.")
elif file_path.is_dir():
print(f"'{file_path}' is a directory.")
else:
print(f"'{file_path}' does not exist.")

Creating the File if It Doesn’t Exist

If the file does not exist, you can create it:

if not file_path.exists():
with open(file_name, "w") as file:
file.write("This is a newly created file.")
 print(f"File '{file_name}' created successfully.")
else:
print(f"File '{file_name}' already exists.")

Output

For an existing file:

The file 'example.txt' exists.

For a non-existing file:

The file 'example.txt' does not exist.

This approach ensures that you can manage file existence efficiently before performing
operations like reading, writing, or deleting.

5. Working with Binary Files

Working with binary files in Python involves handling files that contain non-text data, such as
images, audio files, or executable files. Python allows you to read and write binary data using rb
(read binary) and wb (write binary) modes.

Here’s how to work with binary files:

1. Writing Data to a Binary File

To write binary data, you can use the wb mode. You can write bytes or byte arrays into the file.

# Example binary data (bytes)

binary_data = bytes([65, 66, 67, 68, 69]) # This represents the
characters 'ABCDE'
file_name = "example.bin"

# Open the file in write binary mode

with open(file_name, "wb") as file:
file.write(binary_data)

print(f"Binary data written to {file_name} successfully.")

2. Reading Data from a Binary File

To read binary data, you use the rb mode, and you can read the data as bytes.

# Open the file in read binary mode

with open(file_name, "rb") as file:
binary_content = file.read()

print("\nContents of the binary file:")

print(binary_content)

3. Appending Data to a Binary File

You can append binary data using the ab mode.

additional_binary_data = bytes([70, 71, 72]) # Represents 'FGH'

# Open the file in append binary mode

with open(file_name, "ab") as file:
file.write(additional_binary_data)

print("\nAdditional binary data appended successfully.")

4. Working with Binary Files Containing Images

Let's say you want to copy an image file as a binary file. You can read the image file and write it
to a new file in binary mode:

# Path to the source image file

source_image = "source_image.jpg"
destination_image = "copied_image.jpg"

# Open the source image in read binary mode and the destination
in write binary mode
with open(source_image, "rb") as source_file:
image_data = source_file.read()

with open(destination_image, "wb") as destination_file:

destination_file.write(image_data)

print(f"Image file copied to {destination_image}.")

5. Working with Binary Files Using bytearray

bytearray is mutable and can be used for modifying binary data:

# Create a bytearray
binary_array = bytearray([1, 2, 3, 4, 5])

# Modify data in the bytearray

binary_array[0] = 255 # Change the first byte to 255

# Write the modified bytearray to a binary file

with open(file_name, "wb") as file:
file.write(binary_array)

print(f"Modified binary data written to {file_name}.")

6. Reading Binary Files in Chunks

If you need to read large binary files, you can read them in chunks:

chunk_size = 1024 # 1 KB chunks

with open(file_name, "rb") as file:

while chunk := file.read(chunk_size):
print(chunk) # Process each chunk

This method is useful for processing large files without loading the entire file into memory.

7. Checking the File Type (Optional)

To verify if the file is binary or text, you can inspect the data for special characters, but generally,
if you open a file in binary mode (rb or wb), you are working with binary data.

Summary of Modes:

● wb: Write binary data (creates the file if it doesn't exist).

● rb: Read binary data.
● ab: Append binary data.
● wb+: Write and read binary data.
● rb+: Read and write binary data.
Example Output

Binary data written to example.bin successfully.

Contents of the binary file:

b'ABCDE'

Additional binary data appended successfully.

Image file copied to copied_image.jpg.

Modified binary data written to example.bin.

This approach allows you to handle binary data effectively, making it easy to read and write files
that contain non-text content.

6. The with Statement

The with statement in Python is used for simplifying the management of resources like file
handling, database connections, and network connections. It ensures that resources are properly
acquired and released, even if an exception occurs during the execution of the block.

Basic Syntax

with expression as variable:

# Do something with the resource

● expression: The expression evaluated to produce a context manager.

● variable: An optional variable that is used to reference the resource managed by the
context manager.
 ● The block of code under the with statement is the context in which the resource is being
used.

Key Benefits of Using with

● Automatic Resource Management: Resources are automatically released when the

block is exited, whether the block completes successfully or an exception occurs.
● Clean Code: It eliminates the need for explicitly calling close(), release(), or
other cleanup functions.

Common Use Cases of the with Statement

1. File Handling

When working with files, the with statement ensures that the file is automatically closed after it
is no longer needed, even if an error occurs during file operations.

# Writing to a file
with open('example.txt', 'w') as file:
file.write('Hello, World!\n')
file.write('Python with statement makes resource management
easy.')

# File is automatically closed when the block exits

In this example:

● open('example.txt', 'w') opens the file in write mode.

● The file is automatically closed after the block, ensuring no resource leaks.
2. Reading from a File

The with statement is equally useful for reading files, ensuring they are closed after reading,
even if an exception occurs during reading.

# Reading from a file

with open('example.txt', 'r') as file:
content = file.read()
print(content)

The file is automatically closed after the reading operation is completed.

3. Using with with Custom Context Managers

You can create your own context managers using the contextlib module or by defining a
class with __enter__() and __exit__() methods.

Example using contextlib:

from contextlib import contextmanager

@contextmanager
def sample_resource():
print("Resource acquired")
yield
print("Resource released")

# Using the custom context manager

with sample_resource():
print("Inside the context block.")
Output:

Resource acquired
Inside the context block.
Resource released

In this example:

● sample_resource is a custom context manager.

● The resource is acquired when entering the block and released when exiting.

4. Database Connections

The with statement is frequently used to manage database connections, ensuring that the
connection is closed after the operations are complete, even in the case of an error.

import sqlite3

# Example of using the `with` statement to manage database

connections
with sqlite3.connect('example.db') as conn:
cursor = conn.cursor()
cursor.execute('CREATE TABLE IF NOT EXISTS users (id INTEGER
PRIMARY KEY, name TEXT)')
cursor.execute("INSERT INTO users (name) VALUES ('Alice')")
conn.commit() # Changes are saved automatically

The connection is automatically committed and closed when the block exits.
How with Works:

The with statement works with context managers. A context manager defines two methods:

1. __enter__(): This method is executed when entering the with block. It acquires the
resource and optionally returns it (e.g., the file object).
2. __exit__(): This method is executed when exiting the with block, even if an
exception occurs. It handles the cleanup, such as closing the file or releasing the resource.

Here's a custom context manager example:

class MyContextManager:
def __enter__(self):
print("Entering the context")
return self # Resource can be returned here

def __exit__(self, exc_type, exc_val, exc_tb):

print("Exiting the context")
# Cleanup code goes here (e.g., closing a file or
releasing a connection)

# Using the custom context manager

with MyContextManager() as cm:
print("Inside the context block.")

Output:

Entering the context

Inside the context block.
Exiting the context
Exception Handling in with Statement

If an exception occurs in the with block, the __exit__() method can handle the exception.

class MyContextManager:
def __enter__(self):
print("Entering the context")
return self

def __exit__(self, exc_type, exc_val, exc_tb):

if exc_type:
print(f"Exception occurred: {exc_val}")
print("Exiting the context")

# Using the custom context manager with an exception

with MyContextManager() as cm:
print("Inside the context block.")
raise ValueError("An error occurred")

Output:

Entering the context

Inside the context block.
Exception occurred: An error occurred
Exiting the context

Summary of Benefits

● Automatic Cleanup: You don't need to explicitly close resources (files, connections,
etc.).
 ● Exception Safety: Resources are properly released, even if an error occurs.
● Cleaner Code: The code is more concise and easier to read.

By using the with statement, Python code becomes more robust and resource-efficient.

7. Pickle in Python

Pickle is a module in Python used to serialize and deserialize Python objects into a byte stream
(binary format). Serialization (also called "pickling") refers to converting a Python object into a
byte stream so that it can be saved to a file, sent over a network, or stored in a database.
Deserialization (also called "unpickling") refers to reconstructing the Python object from the byte
stream.

1. Basic Example of Pickling and Unpickling

Here’s a simple example of how to use pickle for serializing and deserializing Python objects.

Pickling (Serializing) an Object

import pickle

# Data to pickle
data = {'name': 'Alice', 'age': 30, 'city': 'Wonderland'}

# Serialize the data into a binary file

with open('data.pkl', 'wb') as file:
pickle.dump(data, file)

print("Data has been pickled and saved.")

In this example:

● pickle.dump(data, file) serializes the data dictionary and writes it to a file

named data.pkl.

Unpickling (Deserializing) an Object

# Read the pickled data from the file

with open('data.pkl', 'rb') as file:
loaded_data = pickle.load(file)

print("Data has been unpickled and loaded:", loaded_data)

In this example:

● pickle.load(file) reads the binary data from the file and deserializes it back into
a Python object.

2. Pickling a Complex Object

Pickle can handle more complex objects like classes, lists, tuples, and dictionaries. Here’s an
example with a custom class:

import pickle

# Define a custom class

class Person:
def __init__(self, name, age):
self.name = name
self.age = age

def __repr__(self):
 return f"Person(name={self.name}, age={self.age})"

# Create an instance of Person

person = Person('Bob', 25)

# Serialize the object

with open('person.pkl', 'wb') as file:
pickle.dump(person, file)

print("Person object has been pickled.")

# Deserialize the object

with open('person.pkl', 'rb') as file:
loaded_person = pickle.load(file)

print("Unpickled person object:", loaded_person)

In this example:

● The Person class is pickled and unpickled just like any other object. It will retain its
structure and values when loaded back.

3. Pickling and Unpickling with pickle.dumps() and pickle.loads()

You can also pickle and unpickle objects to and from memory (in a string or bytes), without
needing to use files.
Pickling to Memory (dumps()):

# Serialize to memory (bytes)

data = {'name': 'Charlie', 'age': 35}
pickled_data = pickle.dumps(data)

print("Serialized data in memory:", pickled_data)

Unpickling from Memory (loads()):

# Deserialize from memory (bytes)

unpickled_data = pickle.loads(pickled_data)

print("Deserialized data:", unpickled_data)

In this case:

● pickle.dumps(data) serializes the data into a byte stream.

● pickle.loads(pickled_data) deserializes the byte stream back into a Python
object.

4. Pickle Protocols

Pickle supports multiple protocols for serialization. Each protocol has different features and
levels of efficiency.

● Protocol 0: ASCII (text) format. It is the oldest and backward-compatible but not as
efficient.
● Protocol 1: Binary format, introduced in Python 2.
● Protocol 2: More efficient binary format.
● Protocol 3: Introduced in Python 3. It is compatible with Python 3.x but not Python 2.x.
● Protocol 4: Introduced in Python 3.4, it supports more data types and is even more
efficient.
 ● Protocol 5: Introduced in Python 3.8, which adds support for efficient out-of-band data
and other features.

You can specify the protocol version when pickling:

with open('data.pkl', 'wb') as file:

pickle.dump(data, file, protocol=pickle.HIGHEST_PROTOCOL)

● pickle.HIGHEST_PROTOCOL ensures the most efficient protocol available in your

version of Python.

5. Security Warning

Pickle can execute arbitrary code during unpickling, which could be a security risk if the pickle
data is received from an untrusted source. Never unpickle data from an untrusted or
unauthenticated source as it can lead to code execution vulnerabilities.

To safely handle data, consider alternatives like JSON for simpler, non-executable data.

Example of a Security Risk with Pickle

import pickle

# Malicious pickled data could execute arbitrary code

malicious_data = b"cos\nsystem\n(S'ls'\ntR."
with open('malicious.pkl', 'wb') as file:
file.write(malicious_data)

# If the data is unpickled, it could execute harmful code

with open('malicious.pkl', 'rb') as file:
pickle.load(file) # This will attempt to run 'ls' on the
system (OS command).
6. Alternatives to Pickle

While pickle is great for serializing Python objects, consider these alternatives based on use
case:

JSON: Best for exchanging data between systems or when interoperability with other
programming languages is required. It supports basic data types like strings, numbers, lists, and
dictionaries.
import json
data = {'name': 'Alice', 'age': 30}
with open('data.json', 'w') as file:
json.dump(data, file)
shelve: A persistent dictionary that stores Python objects in a file with a simple key-value
interface.
import shelve
with shelve.open('my_shelf') as shelf:
shelf['person'] = person

Summary

● Pickle is used to serialize and deserialize Python objects, making it easy to store and
retrieve complex data structures.
● It can be used with both simple and custom objects.
● Pickle is efficient but should be used cautiously, especially when unpickling data from
untrusted sources.
● Other serialization options like JSON or shelve might be better suited for certain tasks,
especially when dealing with cross-platform or cross-language data exchanges.
8. The seek() and tell() Methods

The seek() and tell() methods in Python are used to manipulate and track the position of
the file pointer in a file. These methods are typically used when working with files in binary
mode or when you need to manage the reading/writing positions in the file manually.

1. seek() Method

The seek() method is used to change the file pointer's position to a specific location within the
file. This is especially useful when you're reading or writing to a specific part of a file without
having to read through the entire file sequentially.

Syntax:

file.seek(offset, whence)

● offset: The number of bytes to move the file pointer. It can be positive (to move
forward) or negative (to move backward).
● whence: (Optional) Specifies from where to start counting the offset. The default
value is os.SEEK_SET (start of the file).
○ os.SEEK_SET: The offset is relative to the beginning of the file (default).
○ os.SEEK_CUR: The offset is relative to the current file position.
○ os.SEEK_END: The offset is relative to the end of the file.

Example of seek():

# Open the file in binary mode

with open('example.txt', 'rb') as file:
# Move the file pointer to the 10th byte
file.seek(10)
print(file.read(5)) # Read 5 bytes from the 10th byte
onwards
 # Move the pointer 5 bytes back from the current position
file.seek(-5, 1)
print(file.read(5)) # Read 5 bytes from this new position

# Move to the end of the file

file.seek(0, 2)
print("End of file reached.")

In this example:

● file.seek(10) moves the file pointer to the 10th byte from the beginning of the file.
● file.seek(-5, 1) moves the pointer 5 bytes backward from the current position.
● file.seek(0, 2) moves the pointer to the end of the file.

2. tell() Method

The tell() method is used to return the current position of the file pointer. This is useful to
know where the pointer is after reading or writing data, especially when working with large files
or when you want to resume reading from a specific location.

Syntax:

file.tell()

● It returns the current position of the file pointer in terms of the number of bytes from the
beginning of the file.

Example of tell():

# Open the file in binary mode

with open('example.txt', 'rb') as file:
print("Current position:", file.tell()) # Get the current
position of the file pointer
 file.seek(10) # Move to the 10th byte
print("Position after seek(10):", file.tell()) # Get the
new position

file.read(5) # Read 5 bytes

print("Position after reading 5 bytes:", file.tell()) # Get
the position after reading

In this example:

● file.tell() returns the current position of the file pointer.

● The position will change after calling seek() and after reading data from the file.

3. Practical Example: Combining seek() and tell()

Here’s a practical example where seek() and tell() are used together to navigate through a
file:

with open('example.txt', 'rb') as file:

# Read first 10 bytes
print(file.read(10))
print("Position after reading 10 bytes:", file.tell())

# Move back to the beginning

file.seek(0)
print("Position after seek(0):", file.tell())

# Read the first 5 bytes again

print(file.read(5))
print("Position after reading 5 bytes:", file.tell())
In this example:

● Initially, the pointer is at the start of the file.

● After reading 10 bytes, the pointer moves forward, and tell() is used to check its new
position.
● After calling seek(0), the pointer is moved back to the beginning, and we read the first
5 bytes again.

4. Use Case for seek() and tell() in Binary Files

When working with binary files, seek() and tell() become more essential, as the position
of the file pointer matters in terms of byte offsets rather than line numbers or character positions.

Example (with a binary file):

# Write binary data

with open('binary_file.bin', 'wb') as file:
file.write(b'HelloWorld')

# Read from a specific position

with open('binary_file.bin', 'rb') as file:
file.seek(5) # Move to the 5th byte
print(file.read(5)) # Should output b'World'

In this case:

● file.seek(5) moves the pointer to the 5th byte in the file.

● file.read(5) then reads the next 5 bytes, returning b'World'.

5. Summary of seek() and tell()

● seek(offset, whence): Moves the file pointer to a specific position in the file.
The position is determined based on offset and the whence parameter.
 ● tell(): Returns the current position of the file pointer.

These methods are essential for file navigation, allowing you to move to specific positions within
the file, read or write data from a particular point, and track the pointer's location within the file.

9. Working with CSV files, excel files

CSV (Comma-Separated Values) files are commonly used for storing tabular data, and Python
provides several ways to work with CSV files. The csv module is built into Python for reading
and writing CSV files, and you can also use pandas for more advanced operations.

1. Working with CSV Files using the csv Module

The csv module allows you to easily read and write CSV files.

Reading a CSV File

To read a CSV file, you can use the csv.reader function. Here's an example:

import csv

# Reading a CSV file

with open('data.csv', mode='r') as file:
reader = csv.reader(file)

# Iterate through rows in the CSV file

for row in reader:
print(row)

● csv.reader(file) reads the CSV file.

● Each row is a list of values corresponding to a line in the CSV file.
Writing to a CSV File

You can write data to a CSV file using the csv.writer function:

import csv

# Data to write to CSV

data = [['Name', 'Age', 'City'],
['Alice', 30, 'New York'],
['Bob', 25, 'Los Angeles'],
['Charlie', 35, 'Chicago']]

# Writing to a CSV file

with open('output.csv', mode='w', newline='') as file:
writer = csv.writer(file)

# Write each row of data

writer.writerows(data)

● writer.writerows(data) writes multiple rows of data.

● newline='' ensures that no extra blank lines are added between rows when writing
the file.

Using Dictionaries with CSV Files

You can also use csv.DictReader and csv.DictWriter to work with CSV files in a
dictionary format:

import csv
# Reading a CSV file using DictReader
with open('data.csv', mode='r') as file:
reader = csv.DictReader(file)

for row in reader:

print(row) # Each row is a dictionary

In this example, the keys of the dictionary correspond to the column names in the CSV file.

Working with Excel Files in Python

To work with Excel files, you can use the openpyxl module for .xlsx files or the pandas
library, which provides a high-level API for both .xlsx and .csv files.

1. Using openpyxl for Excel Files

The openpyxl library is used for reading and writing .xlsx files. First, install openpyxl:

pip install openpyxl

Reading from an Excel File

from openpyxl import load_workbook

# Load the workbook and select the active sheet

workbook = load_workbook('example.xlsx')
sheet = workbook.active

# Iterate through rows

for row in sheet.iter_rows(values_only=True):
print(row)
 ● load_workbook('example.xlsx') loads the Excel workbook.
● sheet.iter_rows(values_only=True) iterates over the rows, and
values_only=True ensures only the cell values are returned (not cell objects).

Writing to an Excel File

from openpyxl import Workbook

# Create a new workbook and select the active sheet

workbook = Workbook()
sheet = workbook.active

# Write data to cells

sheet['A1'] = 'Name'
sheet['B1'] = 'Age'
sheet['A2'] = 'Alice'
sheet['B2'] = 30
# Save the workbook
workbook.save('output.xlsx')

● sheet['A1'] and similar statements assign values to specific cells.

● workbook.save('output.xlsx') saves the workbook.

2. Using pandas for Excel Files

pandas provides a convenient API for working with both CSV and Excel files. To work with
Excel files, you need to install the openpyxl or xlrd library (for older .xls files):

pip install pandas openpyxl

Reading from an Excel File with pandas

import pandas as pd

# Read Excel file into a DataFrame

df = pd.read_excel('example.xlsx')

# Display the content of the file

print(df)

● pd.read_excel('example.xlsx') reads the Excel file into a DataFrame.

Writing to an Excel File with pandas

import pandas as pd

# Data to write to Excel

data = {
'Name': ['Alice', 'Bob', 'Charlie'],
'Age': [30, 25, 35],
'City': ['New York', 'Los Angeles', 'Chicago']
}

# Create a DataFrame
df = pd.DataFrame(data)

# Write the DataFrame to an Excel file

df.to_excel('output.xlsx', index=False)

● df.to_excel('output.xlsx') writes the DataFrame to an Excel file.

 ● index=False prevents pandas from writing the row indices.

Handling Multiple Sheets in Excel

You can read and write to multiple sheets within an Excel file:

# Read multiple sheets

dfs = pd.read_excel('example.xlsx', sheet_name=None) # Returns
a dictionary of DataFrames
print(dfs.keys()) # Sheet names

# Write to multiple sheets

with pd.ExcelWriter('output_multiple_sheets.xlsx') as writer:
df1.to_excel(writer, sheet_name='Sheet1', index=False)
df2.to_excel(writer, sheet_name='Sheet2', index=False)

● sheet_name=None reads all sheets in the file into a dictionary.

● pd.ExcelWriter is used to write multiple sheets to an Excel file.

Summary of Working with CSV and Excel Files

1. CSV Files:
○ Use the csv module to read and write CSV files.
○ You can use csv.reader and csv.writer for simple operations, or
csv.DictReader and csv.DictWriter for working with data in
dictionary format.
2. Excel Files:
○ Use openpyxl for reading and writing .xlsx files, or pandas for a
higher-level approach that supports both .xlsx and .csv files.
○ pandas provides efficient methods for working with tabular data, and it supports
reading and writing to multiple sheets in an Excel file.
Both csv and pandas are commonly used for data processing, and openpyxl is useful when
working with Excel files when more control over formatting is needed.

10. Working with Image

Working with images in Python is often done using libraries like Pillow (PIL Fork), which
provides a simple and powerful way to open, manipulate, and save image files in various
formats.

Here’s a guide to working with images using the Pillow library.

1. Installing Pillow

First, you'll need to install the Pillow library if you haven't already:

pip install Pillow

2. Opening and Displaying Images

You can open and display an image using the Pillow library. Here's how:

from PIL import Image

# Open an image file

image = Image.open('image.jpg')

# Display the image

image.show()

● Image.open('image.jpg'): Opens the image file.

● image.show(): Displays the image using the default image viewer.
3. Saving Images

Once you've modified an image, you can save it in various formats like PNG, JPEG, etc.

from PIL import Image

# Open an image file

image = Image.open('image.jpg')

# Save the image in a different format

image.save('image.png') # Saves as PNG format

4. Image Size and Dimensions

You can retrieve the dimensions of an image (width and height) using the size attribute:

from PIL import Image

# Open an image file

image = Image.open('image.jpg')

# Get the size of the image

width, height = image.size
print(f"Width: {width}, Height: {height}")

5. Resizing Images

You can resize an image using the resize() method:

from PIL import Image

# Open an image file
image = Image.open('image.jpg')

# Resize the image

resized_image = image.resize((400, 400))

# Save the resized image

resized_image.save('resized_image.jpg')

● resize((400, 400)): Resizes the image to a width and height of 400x400 pixels.

6. Cropping Images

To crop an image, you specify a box with the coordinates (left, upper, right,
lower):

from PIL import Image

# Open an image file

image = Image.open('image.jpg')

# Define the cropping box (left, upper, right, lower)

box = (100, 100, 400, 400)

# Crop the image

cropped_image = image.crop(box)

# Save the cropped image

cropped_image.save('cropped_image.jpg')
7. Rotating and Flipping Images

You can rotate and flip images using the rotate() and transpose() methods:

Rotating:

from PIL import Image

# Open an image file

image = Image.open('image.jpg')

# Rotate the image by 90 degrees

rotated_image = image.rotate(90)

# Save the rotated image

rotated_image.save('rotated_image.jpg')

Flipping (Transpose):

from PIL import Image

# Open an image file

image = Image.open('image.jpg')

# Flip the image horizontally (left-right flip)

flipped_image = image.transpose(Image.FLIP_LEFT_RIGHT)

# Save the flipped image

flipped_image.save('flipped_image.jpg')

8. Converting Between Different Image Modes

You can convert between different color modes, such as RGB and grayscale:

from PIL import Image

# Open an image file

image = Image.open('image.jpg')

# Convert the image to grayscale

grayscale_image = image.convert('L')

# Save the grayscale image

grayscale_image.save('grayscale_image.jpg')

● convert('L') converts the image to grayscale. Other modes include 'RGB',

'RGBA', and 'CMYK'.

9. Applying Filters to Images

The ImageFilter module provides several image filters such as blur, sharpen, and more.

Applying a Blur Filter:

from PIL import Image, ImageFilter

# Open an image file

image = Image.open('image.jpg')

# Apply a blur filter

blurred_image = image.filter(ImageFilter.BLUR)

# Save the blurred image

blurred_image.save('blurred_image.jpg')

Other Filters:

● ImageFilter.CONTOUR: Applies contour filtering.

● ImageFilter.SHARPEN: Sharpen the image.
● ImageFilter.EDGE_ENHANCE: Enhances the edges of the image.

10. Drawing on Images

You can use the ImageDraw module to draw shapes, text, etc., on an image.

Drawing Text on an Image:

from PIL import Image, ImageDraw, ImageFont

# Open an image file

image = Image.open('image.jpg')

# Initialize ImageDraw object

draw = ImageDraw.Draw(image)

# Define font and text color

font = ImageFont.load_default()
text = "Hello, World!"
text_color = (255, 0, 0) # Red

# Draw text on the image at position (50, 50)

draw.text((50, 50), text, font=font, fill=text_color)
# Save the image with text
image.save('image_with_text.jpg')

11. Working with Transparency (Alpha Channel)

If you're working with images that have transparency (e.g., PNG), you can manipulate the alpha
channel (transparency) as well:

from PIL import Image

# Open an image with transparency

image = Image.open('image_with_alpha.png')

# Convert image to RGBA mode (if not already)

image = image.convert('RGBA')

# Get the alpha channel (transparency)

r, g, b, a = image.split()

# Modify the alpha channel (make image more transparent)

a = a.point(lambda i: i * 0.5) # Reduce opacity by 50%

# Merge the channels back

image = Image.merge('RGBA', (r, g, b, a))

# Save the modified image

image.save('image_with_modified_alpha.png')

12. Example of Combining Multiple Operations

Here's an example combining multiple image operations:

from PIL import Image, ImageFilter

# Open an image file

image = Image.open('image.jpg')

# Convert to grayscale
image = image.convert('L')

# Resize the image

image = image.resize((300, 300))

# Apply a blur filter

image = image.filter(ImageFilter.GaussianBlur(5))

# Save the modified image

image.save('final_image.jpg')

Summary of Common Pillow Operations

1. Open and Display: Image.open(), image.show()

2. Save: image.save('filename')
3. Resize: image.resize((width, height))
4. Crop: image.crop((left, upper, right, lower))
5. Rotate/Flip: image.rotate(angle), image.transpose()
6. Convert: image.convert(mode)
7. Apply Filters: image.filter(filter)
8. Draw Text/Shapes: ImageDraw.Draw()
 9. Transparency: image.split() and Image.merge()

These are just a few of the basic operations you can perform with images using Python. The
Pillow library is very powerful and allows for more complex image processing tasks,
including advanced filters, transformations, and effects.

11. Working with Directories

Working with directories in Python can be done using the built-in os and pathlib modules.
Both modules provide functions for creating, navigating, and managing directories.

1. Using the os Module for Directory Operations

The os module provides a set of methods to interact with the file system, including functions for
working with directories.

Creating a Directory

You can create a directory using os.mkdir() for a single directory or os.makedirs() for
creating intermediate directories as well.

import os

# Creating a single directory

os.mkdir('new_directory')

# Creating intermediate directories if they do not exist

os.makedirs('parent_directory/child_directory')

● os.mkdir() creates a single directory.

● os.makedirs() creates all intermediate directories if they don’t exist.
Listing Contents of a Directory

You can list the files and subdirectories inside a directory using os.listdir():

import os

# List files and directories in the current directory

files_and_dirs = os.listdir('.')
print(files_and_dirs)

● os.listdir() returns a list of the names of the entries in the directory given by the
path.

Changing the Current Working Directory

To change the current working directory, use os.chdir():

import os

# Change to a specific directory

os.chdir('new_directory')

# Verify the current working directory

print(os.getcwd())

● os.getcwd() returns the current working directory.

● os.chdir(path) changes the current working directory to the specified path.

Checking if a Directory Exists

You can check if a directory exists using os.path.exists() or os.path.isdir():

import os

# Check if the directory exists

if os.path.exists('new_directory'):
print('Directory exists')
else:
print('Directory does not exist')

# Check if it is a directory
if os.path.isdir('new_directory'):
print('It is a directory')

● os.path.exists(path) checks if the path exists.

● os.path.isdir(path) checks if the path is a directory.

Removing a Directory

You can remove a directory using os.rmdir() (for empty directories) or

os.removedirs() (for removing empty directories along a path).

import os

# Remove an empty directory

os.rmdir('new_directory')

# Remove directories along a path

os.removedirs('parent_directory/child_directory')

● os.rmdir() removes an empty directory.

 ● os.removedirs() removes intermediate empty directories.

2. Using the pathlib Module for Directory Operations

pathlib provides an object-oriented approach to handle file system paths and directories.

Creating a Directory

Use the Path.mkdir() method to create directories:

from pathlib import Path

# Creating a directory
path = Path('new_directory')
path.mkdir(parents=True, exist_ok=True)

● parents=True: Allows creating intermediate directories if they don't exist.

● exist_ok=True: Prevents raising an error if the directory already exists.

Listing Contents of a Directory

You can list the files and directories inside a directory using Path.iterdir():

from pathlib import Path

# List files and directories in the current directory

path = Path('.')
for item in path.iterdir():
print(item)

● path.iterdir() returns an iterator over the entries in the directory.

Checking if a Directory Exists

You can check if a directory exists using the exists() and is_dir() methods:

from pathlib import Path

# Check if the directory exists

path = Path('new_directory')
if path.exists():
print('Directory exists')

# Check if it is a directory
if path.is_dir():
print('It is a directory')

● path.exists() checks if the path exists.

● path.is_dir() checks if the path is a directory.

Changing the Current Working Directory

You can change the current working directory using os.chdir() (as there’s no direct method
in pathlib):

from pathlib import Path

import os

# Change to a specific directory

path = Path('new_directory')
os.chdir(path)

# Verify the current working directory

print(Path.cwd()) # Get current working directory using pathlib

● Path.cwd() returns the current working directory using pathlib.

Removing a Directory

Use Path.rmdir() to remove an empty directory:

from pathlib import Path

# Remove an empty directory

path = Path('new_directory')
path.rmdir()

● path.rmdir() removes an empty directory.

3. Example of Working with Directories

Here’s an example that demonstrates various directory operations:

import os
from pathlib import Path

# Create a directory using os

os.mkdir('test_directory')

# Create a directory using pathlib

path = Path('test_directory_pathlib')
path.mkdir(parents=True, exist_ok=True)

# List contents of the current directory using os

print('Using os.listdir():', os.listdir('.'))

# List contents using pathlib

print('Using pathlib:', [item for item in Path('.').iterdir()])

# Check if directories exist using os

print('Directory exists using os:',
os.path.exists('test_directory'))

# Check if directories exist using pathlib

print('Directory exists using pathlib:', path.exists())

# Change current working directory using os

os.chdir('test_directory')
print('Current working directory using os:', os.getcwd())

# Change using pathlib (with os for compatibility)

os.chdir(path)
print('Current working directory using pathlib:', Path.cwd())

# Remove the directories

os.rmdir('test_directory')
path.rmdir()

Summary of Directory Operations

● Creating Directories: Use os.mkdir() or Path.mkdir().

 ● Listing Contents: Use os.listdir() or Path.iterdir().
● Checking if a Directory Exists: Use os.path.exists() / os.path.isdir() or
Path.exists() / Path.is_dir().
● Changing the Working Directory: Use os.chdir().
● Removing Directories: Use os.rmdir() or Path.rmdir().

Both the os and pathlib modules provide ways to interact with the file system, with
pathlib offering a more modern and object-oriented approach. For newer codebases, it's
recommended to use pathlib for its cleaner syntax and better integration with other Python
libraries.

Module 17 - Regular Expressions in Python

1. Sequence Characters in Regular Expressions
In Python, regular expressions (regex) allow you to work with sequences of characters using the
re module. Here’s how to handle and match sequences of characters effectively:

Basic Syntax for Sequences

Literal Sequences
Match exact sequences of characters:
import re
pattern = r"hello"
match = re.search(pattern, "hello world")
if match:
print("Match found!")

1. Character Classes
Define a set of characters that a single position in the string can match:
○ [abc]: Matches a, b, or c.
○ [a-z]: Matches any lowercase letter.
○ [0-9]: Matches any digit.
pattern = r"[a-z][0-9]"

match = re.search(pattern, "a1")

if match:
print("Match found!")

2. Quantifiers
Match sequences of characters with specific lengths:
○ *: Matches 0 or more occurrences.
○ +: Matches 1 or more occurrences.
○ ?: Matches 0 or 1 occurrence.
○ {n}: Matches exactly n occurrences.
○ {n,}: Matches n or more occurrences.
 ○ {n,m}: Matches between n and m occurrences.

pattern = r"\d{3}-\d{2}-\d{4}"
match = re.search(pattern, "123-45-6789")
if match:
print("Match found!")

3. Escape Sequences
Special sequences to match predefined sets:
○ \d: Matches a digit ([0-9]).
○ \D: Matches a non-digit.
○ \w: Matches an alphanumeric character ([a-zA-Z0-9_]).
○ \W: Matches a non-alphanumeric character.
○ \s: Matches a whitespace character.
○ \S: Matches a non-whitespace character.

pattern = r"\w+\s\w+"
match = re.search(pattern, "hello world")
if match:
print("Match found!")

Examples of Complex Sequences

Email Address Validation
pattern = r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}"
match = re.search(pattern, "[email protected]")
if match:
print("Valid email!")
Extracting Specific Patterns
text = "Order ID: 12345, Customer ID: 67890"
pattern = r"Order ID: (\d+), Customer ID: (\d+)"
match = re.search(pattern, text)
if match:
print("Order ID:", match.group(1))
print("Customer ID:", match.group(2))
Phone Number Validation
pattern = r"\(\d{3}\) \d{3}-\d{4}"
match = re.search(pattern, "(123) 456-7890")
if match:
print("Valid phone number!")

Useful Functions in re Module

● re.match(): Matches from the beginning of the string.

● re.search(): Searches for the pattern anywhere in the string.
● re.findall(): Returns a list of all matches.
● re.sub(): Replaces occurrences of a pattern with a replacement string.

Example:

import re

text = "Price: $50, Discount: $10"

new_text = re.sub(r"\$\d+", "<amount>", text)
print(new_text) # Output: Price: <amount>, Discount: <amount>

2. Quantifiers in Regular Expressions

Quantifiers in regular expressions (regex) define how many times a specific pattern must occur
for it to match. Here's an overview of the quantifiers available in Python's re module:
Common Quantifiers

1. * (Zero or more)
○ Matches 0 or more occurrences of the preceding pattern.

import re
pattern = r"ab*"
match = re.search(pattern, "a") # Matches "a"
match = re.search(pattern, "abbb") # Matches "abbb"

2. + (One or more)
○ Matches 1 or more occurrences of the preceding pattern.

pattern = r"ab+"
match = re.search(pattern, "ab") # Matches "ab"
match = re.search(pattern, "abbb") # Matches "abbb"
match = re.search(pattern, "a") # No match

3. ? (Zero or one)
○ Matches 0 or 1 occurrence of the preceding pattern.

pattern = r"ab?"
match = re.search(pattern, "a") # Matches "a"
match = re.search(pattern, "ab") # Matches "ab"
match = re.search(pattern, "abb") # Matches "ab"

4. {n} (Exactly n occurrences)

○ Matches exactly n occurrences of the preceding pattern.

pattern = r"a{3}"
match = re.search(pattern, "aaa") # Matches "aaa"
match = re.search(pattern, "aa") # No match
 5. {n,} (At least n occurrences)
○ Matches n or more occurrences of the preceding pattern.

pattern = r"a{2,}"
match = re.search(pattern, "aaa") # Matches "aaa"
match = re.search(pattern, "aa") # Matches "aa"
match = re.search(pattern, "a") # No match

6. {n,m} (Between n and m occurrences)

○ Matches between n and m occurrences of the preceding pattern.

pattern = r"a{2,4}"
match = re.search(pattern, "aa") # Matches "aa"
match = re.search(pattern, "aaa") # Matches "aaa"
match = re.search(pattern, "aaaa") # Matches "aaaa"
match = re.search(pattern, "aaaaa") # Matches "aaaa"

Greedy vs. Non-Greedy Quantifiers

Quantifiers are greedy by default, meaning they match as many occurrences as possible. To make
them non-greedy, append a ? after the quantifier.

Greedy Example:

pattern = r"a.*b"
text = "acbacb"
match = re.search(pattern, text) # Matches "acbacb"

Non-Greedy Example:

pattern = r"a.*?b"
text = "acbacb"
match = re.search(pattern, text) # Matches "acb"
Practical Examples
Matching Phone Numbers
pattern = r"\d{3}-\d{3}-\d{4}"
match = re.search(pattern, "123-456-7890")
if match:
print("Valid phone number!")
Extracting Words with Optional Suffix
pattern = r"\bcat(s)?\b"
text = "I have a cat and two cats."
matches = re.findall(pattern, text)
print(matches) # Matches "cat" and "cats"
Validating Password Length
pattern = r".{8,}"
match = re.search(pattern, "securePass")
if match:
print("Valid password!")

3. Methods in Regular Expression (findall, finditr, search, sub, compile)

The re module in Python provides several methods to work with regular expressions. Here's an
overview of the commonly used methods:

1. findall()

● Purpose: Returns a list of all non-overlapping matches of the pattern in the string.
● Usage: Ideal for extracting multiple occurrences of a pattern.

Syntax:
re.findall(pattern, string, flags=0)
Example:

import re
text = "The price is $5, and the discount is $2."
pattern = r"\$\d+"
matches = re.findall(pattern, text)
print(matches) # Output: ['$5', '$2']

2. finditer()

● Purpose: Returns an iterator yielding match objects for all non-overlapping matches in
the string.
● Usage: Useful when you need detailed match information (like start and end positions).

Syntax:
re.finditer(pattern, string, flags=0)

Example:

import re
text = "The price is $5, and the discount is $2."
pattern = r"\$\d+"
matches = re.finditer(pattern, text)
for match in matches:
print(f"Match: {match.group()}, Start: {match.start()}, End:
{match.end()}")
# Output:
# Match: $5, Start: 13, End: 15
# Match: $2, Start: 35, End: 37

3. search()
 ● Purpose: Searches the string for the first occurrence of the pattern and returns a match
object if found.
● Usage: Best when you want to find the first match or verify if a pattern exists.

Syntax:
re.search(pattern, string, flags=0)

Example:

import re
text = "Order ID: 12345"
pattern = r"\d+"
match = re.search(pattern, text)
if match:
print(f"Match: {match.group()}, Start: {match.start()}, End:
{match.end()}")
# Output: Match: 12345, Start: 10, End: 15

4. sub()

● Purpose: Replaces all matches of the pattern with a specified replacement string.
● Usage: Useful for substitutions like masking sensitive information or formatting strings.

Syntax:
re.sub(pattern, replacement, string, count=0, flags=0)

Example:

import re
text = "My phone number is 123-456-7890."
pattern = r"\d{3}-\d{3}-\d{4}"
masked = re.sub(pattern, "[REDACTED]", text)
print(masked) # Output: My phone number is [REDACTED].

5. compile()

● Purpose: Compiles a regex pattern into a regex object for repeated use, improving
performance.
● Usage: Use when the same pattern will be applied multiple times.

Syntax:
regex = re.compile(pattern, flags=0)

Example:

import re
pattern = re.compile(r"\d{3}-\d{3}-\d{4}")
text1 = "My phone number is 123-456-7890."
text2 = "Emergency contact: 987-654-3210."
# Use the compiled pattern
print(pattern.search(text1).group()) # Output: 123-456-7890
print(pattern.search(text2).group()) # Output: 987-654-3210

Comparison of Methods

Method Returns Use Case

findall List of matches Extract all matching patterns.

finditer Iterator of match objects Extract matches with positions/details.

search First match object Check or extract the first match.

sub Modified string Replace occurrences of a pattern.

compile Regex object Reuse the same pattern efficiently.

Putting It All Together

import re
text = "Emails: [email protected], [email protected]"

# Compile a pattern
email_pattern =
re.compile(r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")

# Find all emails

emails = email_pattern.findall(text)
print("Emails:", emails)

# Mask emails
masked_text = email_pattern.sub("[EMAIL]", text)
print("Masked Text:", masked_text)

# Iterate through emails with details

for match in email_pattern.finditer(text):
print(f"Email: {match.group()}, Start: {match.start()}, End:
{match.end()}")

Output:

Emails: ['[email protected]', '[email protected]']

Masked Text: Emails: [EMAIL], [EMAIL]
Email: [email protected], Start: 8, End: 24
Email: [email protected], Start: 26, End: 46
4. Special Characters in Regular Expressions

Special characters in regular expressions have specific meanings and allow you to build complex
patterns. Here's a breakdown of these characters and their purposes:

1. Special Characters

Character Description Example

. Matches any single character a.c matches abc, a1c, a-c, but not ac.
except a newline (\n).

^ Matches the start of a string. ^hello matches hello world but not world
hello.

$ Matches the end of a string. world$ matches hello world but not world
hello.

* Matches 0 or more repetitions a* matches aaa, a, or an empty string.

of the preceding character or
group.

+ Matches 1 or more repetitions a+ matches a, aaa, but not an empty string.

of the preceding character or
group.

? Matches 0 or 1 occurrence of colou?r matches color or colour.

the preceding character or
group.

{n} Matches exactly n occurrences a{3} matches aaa but not aa or aaaa.
of the preceding character or
group.

{n,} Matches at least n occurrences. a{2,} matches aa, aaa, or more.

{n,m} Matches between n and m a{2,4} matches aa, aaa, or aaaa.
occurrences.

` ` Acts as an OR operator between patterns.

() Groups patterns together and (abc)+ matches abc, abcabc, etc.

captures the matched text.

[] Defines a character set or range [a-z] matches any lowercase letter.

to match.

\ Escapes a special character or \. matches a literal ..

introduces a special sequence
(e.g., \d, \w).

2. Special Sequences

Sequence Description Example

\d Matches any digit ([0-9]). \d+ matches 123, 45, etc.

\D Matches any non-digit character. \D+ matches abc, xyz, etc.

\w Matches any word character (alphanumeric + \w+ matches word,

underscore, [a-zA-Z0-9_]). hello_123.

\W Matches any non-word character. \W+ matches !@#, spaces,

etc.

\s Matches any whitespace character (spaces, tabs, \s+ matches spaces or other
newlines). whitespace.

\S Matches any non-whitespace character. \S+ matches text,

word123.
 \b Matches a word boundary. \bword\b matches word
but not word123.

\B Matches a non-word boundary. \Bword matches sword or

worded.

\\ Matches a literal backslash. \\ matches \.

3. Escaping Special Characters

When you want to match a literal special character (e.g., *, +, .), you must escape it using \.

Example:

import re
text = "3+5=8"
pattern = r"\+"
match = re.search(pattern, text)
if match:
print("Matched a literal +")

4. Practical Examples

Match a Valid Email

import re
pattern = r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}"
email = "[email protected]"
if re.match(pattern, email):
print("Valid email!")
Extract Digits from a String

import re
text = "Order ID: 12345"
pattern = r"\d+"
numbers = re.findall(pattern, text)
print(numbers) # Output: ['12345']

Match a URL

import re
pattern = r"https?://[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}"
url = "Visit https://example.com for details."
match = re.search(pattern, url)
if match:
print("Found URL:", match.group())

Replace Special Characters

import re
text = "Hello, @world! #Python is amazing."
pattern = r"[^\w\s]"
cleaned = re.sub(pattern, "", text)
print(cleaned) # Output: Hello world Python is amazing

Tips for Using Special Characters

1. Escape only when necessary:

○ re.escape() can help if you're dealing with user-generated input containing
special characters.

import re
text = "Price is $5.00."
escaped = re.escape(text)
print(escaped) # Output: Price\ is\ \$5\.00\.

2. Combine groups for complex patterns:

○ Use parentheses for grouping and alternation for flexibility.
pattern = r"(cat|dog)s?"

5. Using Regular Expressions on Files

Using regular expressions to process files in Python is a common and powerful technique for
tasks like extracting data, validating content, or performing substitutions. Below are key steps
and examples for applying regular expressions to files.

Steps to Use Regular Expressions on Files

1. Open and Read the File:

○ Use Python's open() function to read the file content.
○ Read either line by line or the entire file at once.
2. Apply Regular Expressions:
○ Use the re module's functions like search(), findall(), or sub() to
process the content.
3. Handle the Results:
○ Extract matched data, count occurrences, replace text, etc.

Examples

1. Extract All Email Addresses from a File

import re

# Open the file and read its contents

with open("sample.txt", "r") as file:
content = file.read()

# Define the email pattern

email_pattern =
r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}"

# Find all email addresses

emails = re.findall(email_pattern, content)

print("Extracted Emails:")
print(emails)

2. Count the Occurrences of a Word in a File

import re

# Open the file and read its contents

with open("sample.txt", "r") as file:
content = file.read()

# Define the word to search (case-insensitive)

word = "python"
pattern = rf"\b{word}\b"

# Find all occurrences

matches = re.findall(pattern, content, flags=re.IGNORECASE)

print(f"The word '{word}' appears {len(matches)} times.")

3. Replace Dates in a File

Suppose a file contains dates in the format MM/DD/YYYY, and you want to change them to
YYYY-MM-DD.

import re

# Open the file and read its contents

with open("sample.txt", "r") as file:
content = file.read()

# Define the date pattern

date_pattern = r"(\d{2})/(\d{2})/(\d{4})"

# Replace with the new format

updated_content = re.sub(date_pattern, r"\3-\1-\2", content)

# Save the modified content back to the file

with open("sample_updated.txt", "w") as file:
file.write(updated_content)

print("Dates have been reformatted.")

4. Validate File Content

Check if a file contains only valid phone numbers (e.g., 123-456-7890).

import re

# Open the file and read its contents line by line

with open("phone_numbers.txt", "r") as file:
valid_pattern = r"^\d{3}-\d{3}-\d{4}$"
for line in file:
line = line.strip()
if re.match(valid_pattern, line):
print(f"Valid: {line}")
else:
print(f"Invalid: {line}")

5. Extract Specific Data Using finditer()

Find all prices (e.g., $10, $5.99) and their positions.

import re

# Open the file and read its contents

with open("sample.txt", "r") as file:
content = file.read()

# Define the price pattern

price_pattern = r"\$\d+(\.\d{2})?"

# Use finditer to get matches with positions

matches = re.finditer(price_pattern, content)

print("Prices found:")
for match in matches:
print(f"Price: {match.group()}, Start: {match.start()}, End:
{match.end()}")
6. Extract Data from a Log File

Extract all IP addresses from a log file.

import re

# Open the log file and read its contents

with open("server.log", "r") as file:
content = file.read()

# Define the IP address pattern

ip_pattern = r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b"

# Find all IP addresses

ips = re.findall(ip_pattern, content)

print("Extracted IP Addresses:")
print(ips)

Best Practices

1. Use with Statement:

○ Ensures the file is closed properly after processing.
2. Optimize Patterns:
○ Test and refine your regular expressions for efficiency, especially with large files.
3. Use Flags for Flexibility:
○ Example: Use re.IGNORECASE for case-insensitive matches or
re.MULTILINE for multi-line processing.
4. Read in Chunks:
○ For large files, process the file in chunks or line by line to avoid memory issues.
 5. Test Patterns with Sample Data:
○ Use tools like regex testers or small test files to validate patterns before applying
them to large datasets.

These examples should cover a wide range of real-world scenarios for using regular expressions
with files in Python!