DATA ENGINEERING:
Data Modeling with Postgres
READ OPERATIONS
● Methods: fetchall(), fetchmany(), and fetchone()
`psycopg2` MODULE ● Example:
select_query = "SELECT * FROM songs"
● psycopg2 is a PostgreSQL database adapter for
cur.execute(select_query)
the Python programming language
records = cur.fetchmany(5)
● Installation: pip install psycopg2
● Usage: import psycopg2 for record in records:
print(record)
INSERT OPERATIONS
CONNECTING TO A DATABASE
insert_query = "INSERT INTO customer
● conn = psycopg2.connect("host=127.0.0.1 (customer_id, name, rewards) VALUES (%s, %s, %s)"
dbname=studentdb user=student data = (1, "Amanda", True)
password=student")
cur.execute(insert_query, data)
CREATE A CURSOR OBJECT ON CONFLICT
● Possible actions: DO NOTHING or DO UPDATE
cur = conn.cursor()
Note: a cursor object allows execution of ● Example:
PostgreSQL command through Python. INSERT INTO users (id, level) VALUES (1, 0)
ON CONFLICT (id) DO UPDATE
SET level = users.level + 1;
CREATE TABLE UPDATE OPERATIONS
create_table = “CREATE TABLE IF NOT EXISTS update_query = "UPDATE vendors SET
songs (song_title varchar, artist_name varchar, year vendor_name = %s WHERE vendor_id = %s"
int, album_name varchar, single Boolean);” new_data = ("Walmart", 2)
cur.execute(update_query, new_data)
cur.execute(create_table)
conn.commit()
print("Table created successfully in PostgreSQL")
DELETE OPERATIONS
delete_query = "DELETE FROM vendors WHERE id
= %s"
cur.execute(delete_query, (5))
Page 1 of 2
SQL MODULE & JUPYTER NOTEBOOK REFERENCES
● Load SQL module: %load_ext sql ● psycopg documentation
● Connect to a db: %sql ● PostgreSQL documentation
postgresql://localhost:5432/<db> ● Python Database API summary.
● Execute and return a SQL query in a list of ● Python errors and exceptions documentation.
tuples: %sql SELECT * FROM vendors;
● Return SQL query in a table with header using
Jupyter cell magic: %%sql
HANDLING EXCEPTION IN PYTHON
try:
cur = conn.cursor()
except psycopg2.Error as e:
print("Error: Could not get cursor to the DB")
print(e)
How it works:
● try statement will be executed first.
● If there is no exception, except statement will
be skipped.
● Otherwise, execute the codes in the exception.
TIPS & TRICKS
● Naming convention:
○ SQL keywords: UPPER CASE
○ names (identifiers):
lower_case_with_underscores
○ Example: UPDATE table SET name = 10;
● Use triple quotes (“”” “””) or backslash (\) to pass
a multi-line query.
● Close db connection as soon as completing a
task because connections are limited resources:
conn.close()
● Set automatic commit to be true so that each
action is committed without having to call
conn.commit() after each command:
conn.set_session(autocommit=True)
Page 2 of 2