PYTHON ADVANCED
PROGRAMMING
Chitra G M and P Rama Devi
Department of Computer Science Engineering
PYTHON ADVANCED
PROGRAMMING
Inter Process
Communication
Chitra G M and P Rama Devi
Department of Computer Science and Engineering
PYTHON ADVANCED PROGRAMMING
IPC
A process can be of two types:
Independent process.
Co-operating process.
An independent process is not affected by the execution of
other processes while a co-operating process can be
affected by other executing processes
PYTHON ADVANCED PROGRAMMING
Though one can think that those processes, which are running independently,
will execute very efficiently, in reality, there are many situations when co-
operative nature can be utilised for increasing computational speed,
convenience and modularity.
Inter process communication (IPC) is a mechanism which allows processes to
communicate with each other and synchronize their actions. The
communication between these processes can be seen as a method of co-
operation between them.
Processes can communicate with each other through both:
Shared Memory
Message passing
PYTHON ADVANCED PROGRAMMING
Consider the program
import multiprocessing
import time
result = []
def square_list(mylist):
global result
# append squares of mylist to global list result
for num in mylist:
result.append(num * num)
# print global list result
print("Result(in process p1)",result)
time.sleep(1)
PYTHON ADVANCED PROGRAMMING
if __name__ == "__main__":
# input list
mylist = [1,2,3,4]
# creating new process
p1 = multiprocessing.Process(target=square_list,
args=(mylist,))
# starting process
p1.start()
# wait until process is finished
p1.join()
# print global result list
print("Result(in main program)",result)
When the above program is executed the output is
available only in the childs address space.
So modify the above program such that the output is
available in both child and parents address space
PYTHON ADVANCED PROGRAMMING
import multiprocessing
def square_list(mylist, result, square_sum):
# append squares of mylist to result array
for idx, num in enumerate(mylist):
result[idx] = num * num
# square_sum value
square_sum.value = sum(result)
# print result Array
print("Result(in process p1):",result[:])
# print square_sum Value
print("Sum of squares(in process
p1):",square_sum.value)
PYTHON ADVANCED PROGRAMMING
if __name__ == "__main__":
# input list
mylist = [1,2,3,4]
# creating Array of int data type with space for 4 integers
result = multiprocessing.Array('i', 4)
# creating Value of int data type
square_sum = multiprocessing.Value('i')
# print(square_sum)
# creating new process
p1 = multiprocessing.Process(target=square_list, args=(mylist, result,
square_sum))
# starting process
p1.start()
# wait until process is finished
p1.join()
# print result array
print("Result(in main program):",result[:])
# print square_sum Value
print("Sum of squares(in main program):",square_sum.value)
THANK YOU
Chitra G M and P Rama Devi
Department of CSE
[email protected][email protected] PYTHON ADVANCED
PROGRAMMING
Chitra G M and P Rama Devi
Department of Computer Science Engineering
PYTHON ADVANCED
PROGRAMMING
Parallel Processes
Chitra G M and P Rama Devi
Department of Computer Science and Engineering
PYTHON ADVANCED PROGRAMMING
Parellel Processes
What is multiprocessing?
Multiprocessing refers to the ability of a system to support more
than one processor at the same time. Applications in a
multiprocessing system are broken to smaller routines that run
independently. The operating system allocates these threads to the
processors improving performance of the system.
PYTHON ADVANCED PROGRAMMING
Why multiprocessing?
Consider a computer system with a single processor. If it is assigned several
processes at the same time, it will have to interrupt each task and switch
briefly to another, to keep all of the processes going.
This situation is just like a chef working in a kitchen alone. He has to do
several tasks like baking, stirring, kneading dough, etc.
Multiprocessing: Running more than one process on a single processor
parallel processing: running a process on more than one processor.
PYTHON ADVANCED PROGRAMMING
Simple example for MultiProcessing
import multiprocessing
def print_cube(num):
print("Cube of a number is",num * num * num)
def print_square(num):
print("squre of a number is",num * num)
if __name__ == "__main__":
# creating processes
p1 = multiprocessing.Process(target=print_square, args=(10, ))
p2 = multiprocessing.Process(target=print_cube, args=(10, ))
# starting process 1
p1.start()
# starting process 2
p2.start()
# wait until process 1 is finished
p1.join()
# wait until process 2 is finished
p2.join()
# both processes finished
print("Done!")
PYTHON ADVANCED PROGRAMMING
# importing the multiprocessing module
import multiprocessing
import os
def f1():
print("p1_id: ",os.getpid())
def f2():
print("p2_id: ",os.getpid())
if __name__ == "__main__":
# printing main program process id
print("main process id",os.getpid())
# creating processes
p1 = multiprocessing.Process(target=f1)
p2 = multiprocessing.Process(target=f2)
PYTHON ADVANCED PROGRAMMING
# starting processes
p1.start()
p2.start()
# wait until processes are finished
p1.join()
p2.join()
# both processes finished
print("Both processes finished execution!")
# check if processes are alive
print("p1 status is alive?:",p1.is_alive())
print("p2 status is alive?:",p2.is_alive())
THANK YOU
Chitra G M and P Rama Devi
Department of CSE
[email protected][email protected] PYTHON ADVANCED
PROGRAMMING
Chitra G M and P Rama Devi
Department of Computer Science Engineering
PYTHON ADVANCED
PROGRAMMING
Parallel Processes
Chitra G M and P Rama Devi
Department of Computer Science and Engineering
PYTHON ADVANCED PROGRAMMING
Parellel Processes
What is multiprocessing?
Multiprocessing refers to the ability of a system to support more
than one processor at the same time. Applications in a
multiprocessing system are broken to smaller routines that run
independently. The operating system allocates these threads to the
processors improving performance of the system.
PYTHON ADVANCED PROGRAMMING
Why multiprocessing?
Consider a computer system with a single processor. If it is assigned several
processes at the same time, it will have to interrupt each task and switch
briefly to another, to keep all of the processes going.
This situation is just like a chef working in a kitchen alone. He has to do
several tasks like baking, stirring, kneading dough, etc.
Multiprocessing: Running more than one process on a single processor
parallel processing: running a process on more than one processor.
PYTHON ADVANCED PROGRAMMING
Simple example for MultiProcessing
import multiprocessing
def print_cube(num):
print("Cube of a number is",num * num * num)
def print_square(num):
print("squre of a number is",num * num)
if __name__ == "__main__":
# creating processes
p1 = multiprocessing.Process(target=print_square, args=(10, ))
p2 = multiprocessing.Process(target=print_cube, args=(10, ))
# starting process 1
p1.start()
# starting process 2
p2.start()
# wait until process 1 is finished
p1.join()
# wait until process 2 is finished
p2.join()
# both processes finished
print("Done!")
PYTHON ADVANCED PROGRAMMING
# importing the multiprocessing module
import multiprocessing
import os
def f1():
print("p1_id: ",os.getpid())
def f2():
print("p2_id: ",os.getpid())
if __name__ == "__main__":
# printing main program process id
print("main process id",os.getpid())
# creating processes
p1 = multiprocessing.Process(target=f1)
p2 = multiprocessing.Process(target=f2)
PYTHON ADVANCED PROGRAMMING
# starting processes
p1.start()
p2.start()
# wait until processes are finished
p1.join()
p2.join()
# both processes finished
print("Both processes finished execution!")
# check if processes are alive
print("p1 status is alive?:",p1.is_alive())
print("p2 status is alive?:",p2.is_alive())
THANK YOU
Chitra G M and P Rama Devi
Department of CSE
[email protected][email protected] PYTHON ADVANCED
PROGRAMMING
Chitra G M and P Rama Devi
Department of Computer Science Engineering
PYTHON ADVANCED
PROGRAMMING
Locks
Chitra G M and P Rama Devi
Department of Computer Science and Engineering
PYTHON ADVANCED PROGRAMMING
Race Condition
Race conditions and Locks
Multiprogramming and concurrency bring in the problem of
race conditions, where multiple processes executing
concurrently on shared data may leave the shared data in an
undesirable, inconsistent state due to concurrent execution.
Note that race conditions ca n happen even on a single
processor system, if processes are context switched out by
the scheduler, or are interrupted otherwise, while updating
shared data structures.
PYTHON ADVANCED PROGRAMMING
Consider a simple example of two processes of a process incrementing a
shared variable. Now, if the increments happen in parallel, it is possible that
the processes will overwrite each other’s result, and the counter will not be
incremented twice as expected. That is, a line of code that increments a
variable is not atomic, and can be executed concurrently by different threads.
ex. bank applications
• Pieces of code that must be accessed in a mutually exclusive atomic
manner by the contending processes are referred to as critical sections.
Critical sections must be protected with locks to guarantee the property of
mutual exclusion.
The code to update a shared counter is a simple example of a critical section.
PYTHON ADVANCED PROGRAMMING
Code that adds a new node to a linked list is another
example.
A critical section performs a certain operation on a shared
data structure, that may temporarily leave the data structure
in an inconsistent state in the middle of the operation.
Therefore, in order to maintain consistency and preserve the
invariants of shared data structures, critical sections must
always execute in a mutually exclusive fashion.
Locks guarantee that only one contending thread resides in
the critical section at any time, ensuring that the shared data
is not left in an inconsistent state.
PYTHON ADVANCED PROGRAMMING
#program to demonstrate race condition.
#Program to get letters in pairs
#remove Locks (demo race condition)
# execute the below program.
from threading import *
import random
import sys
import time
lock=Lock()
# get letters in pairs
PYTHON ADVANCED PROGRAMMING
def fn1() :
s = 'ABCDEFGH'
for i in range(0, len(s)) :
lock.acquire()
print(s[i], end=' ')
sys.stdout.flush()
time.sleep(int(random.random() * 3))
print(s[i], end=' ')
sys.stdout.flush()
lock.release()
time.sleep(int(random.random() * 3))
PYTHON ADVANCED PROGRAMMING
def fn2() :
s = 'abcdefgh'
for i in range(0, len(s)) :
lock.acquire()
print(s[i], end=' ')
sys.stdout.flush()
time.sleep(int(random.random() * 3))
print(s[i], end=' ')
sys.stdout.flush()
lock.release()
time.sleep(int(random.random() * 3))
THANK YOU
Chitra G M and P Rama Devi
Department of CSE
[email protected][email protected] PYTHON ADVANCED
PROGRAMMING
Chitra G M and P Rama Devi
Department of Computer Science Engineering
PYTHON ADVANCED
PROGRAMMING
Locks
Chitra G M and P Rama Devi
Department of Computer Science and Engineering
PYTHON ADVANCED PROGRAMMING
Race Condition
Race conditions and Locks
Multiprogramming and concurrency bring in the problem of
race conditions, where multiple processes executing
concurrently on shared data may leave the shared data in an
undesirable, inconsistent state due to concurrent execution.
Note that race conditions ca n happen even on a single
processor system, if processes are context switched out by
the scheduler, or are interrupted otherwise, while updating
shared data structures.
PYTHON ADVANCED PROGRAMMING
Consider a simple example of two processes of a process incrementing a
shared variable. Now, if the increments happen in parallel, it is possible that
the processes will overwrite each other’s result, and the counter will not be
incremented twice as expected. That is, a line of code that increments a
variable is not atomic, and can be executed concurrently by different threads.
ex. bank applications
• Pieces of code that must be accessed in a mutually exclusive atomic
manner by the contending processes are referred to as critical sections.
Critical sections must be protected with locks to guarantee the property of
mutual exclusion.
The code to update a shared counter is a simple example of a critical section.
PYTHON ADVANCED PROGRAMMING
Code that adds a new node to a linked list is another
example.
A critical section performs a certain operation on a shared
data structure, that may temporarily leave the data structure
in an inconsistent state in the middle of the operation.
Therefore, in order to maintain consistency and preserve the
invariants of shared data structures, critical sections must
always execute in a mutually exclusive fashion.
Locks guarantee that only one contending thread resides in
the critical section at any time, ensuring that the shared data
is not left in an inconsistent state.
PYTHON ADVANCED PROGRAMMING
#program to demonstrate race condition.
#Program to get letters in pairs
#remove Locks (demo race condition)
# execute the below program.
from threading import *
import random
import sys
import time
lock=Lock()
# get letters in pairs
PYTHON ADVANCED PROGRAMMING
def fn1() :
s = 'ABCDEFGH'
for i in range(0, len(s)) :
lock.acquire()
print(s[i], end=' ')
sys.stdout.flush()
time.sleep(int(random.random() * 3))
print(s[i], end=' ')
sys.stdout.flush()
lock.release()
time.sleep(int(random.random() * 3))
PYTHON ADVANCED PROGRAMMING
def fn2() :
s = 'abcdefgh'
for i in range(0, len(s)) :
lock.acquire()
print(s[i], end=' ')
sys.stdout.flush()
time.sleep(int(random.random() * 3))
print(s[i], end=' ')
sys.stdout.flush()
lock.release()
time.sleep(int(random.random() * 3))
THANK YOU
Chitra G M and P Rama Devi
Department of CSE
[email protected][email protected] PYTHON APPLICATION
PROGRAMMING
Chitra G M ,P Rama Devi
Computer Science and
Engineering
PYTHON APPLICATION PROGRAMMING
SQLite3
Chitra G M, P Rama Devi
Department of Computer Science and Engineering
PYTHON APPLICATION PROGRAMMING
Database Programming
SQLite3
• SQLite is a software library that implements a self-contained,
serverless, zero configuration, transactional SQL database
engine.
• SQLite is the most widely deployed SQL database engine . The
source code for SQLite is in the public domain.
PYTHON APPLICATION PROGRAMMING
Database Programming
SQLite3
SQLite is not directly comparable to client/server SQL database
engines such as MySQL, Oracle, PostgreSQL, or SQL Server since
SQLite is trying to solve a different problem.
PYTHON APPLICATION PROGRAMMING
Database Programming
Client Server DB Engine VS SQLite3
• Client/server SQL database engines strive to implement a
shared repository of enterprise data. They emphasize scalability,
concurrency, centralization, and control.
• SQLite strives to provide local data storage for individual
applications and devices.
• SQLite emphasizes economy, efficiency, reliability,
independence, and simplicity.
PYTHON APPLICATION PROGRAMMING
Database Programming
When not to Use SQLITE ?
Client/Server Applications
If there are many client programs sending SQL to the same
database over a network, then use a client/server database
engine instead of SQLite.
High-volume Websites
SQLite will normally work fine as the database backend to a
website. But if the website is write-intensive or is so busy that it
requires multiple servers, then consider using an enterprise-class
client/server database engine instead of SQLite.
PYTHON APPLICATION PROGRAMMING
Database Programming
When not to Use SQLITE ?
Very large datasets
An SQLite database is limited in size to 140 terabytes (247 bytes,
128 tibibytes).
High Concurrency
SQLite supports an unlimited number of simultaneous readers,
but it will only allow one writer at any instant in time.
PYTHON APPLICATION PROGRAMMING
Database Programming
When not to Use SQLITE ?
Very large datasets
An SQLite database is limited in size to 140 terabytes (247 bytes,
128 tibibytes).
High Concurrency
SQLite supports an unlimited number of simultaneous readers,
but it will only allow one writer at any instant in time.
PYTHON APPLICATION PROGRAMMING
Database Programming
Connection objects
Connection objects create a connection with the database and
these are further used for different transactions. These connection
objects are also used as representatives of the database session.
You can use a connection object for calling methods like commit(),
rollback() and close()
Cursor objects
Cursor is one of the powerful features of SQL. These are objects
that are responsible for submitting various SQL statements to a
database server.
PYTHON APPLICATION PROGRAMMING
Database Programming
Sample Program:
import sqlite3
cxn=sqlite3.connect("example.db")
cur=cxn.cursor()
''' #execute first and then comment
cur.execute('create table tab1 (name char(20), age integer)')
cur.execute('create table tab2 (name char(20), age integer)')
'''
PYTHON APPLICATION PROGRAMMING
Database Programming
Sample Program(continued):
cur.execute('insert into tab1 values ("chitra",34)')
cur.execute('insert into tab1 values ("rama",34)')
cxn.execute('insert into tab2 values ("Ganesh",21)')
cxn.execute('insert into tab2 values ("Ggg",21)')
#usage of place holder.
info=('aa',30)
cur.execute('insert into tab1 (name,age) values (?,?)',info)
x=cur.execute('select * from tab1')
for i in x:
print(i)
print("done")
THANK YOU
Chitra G M, Rama Devi
Department of Computer Science and Engineering
[email protected]
+91 9900300411
[email protected]
+91 9620072112
PYTHON ADVANCED
PROGRAMMING
Chitra G M and P Rama Devi
Department of Computer Science Engineering
PYTHON ADVANCED
PROGRAMMING
Semaphores
Chitra G M and P Rama Devi
Department of Computer Science and Engineering
PYTHON ADVANCED PROGRAMMING
Semaphores
The Semaphore class of the Python threading module
implements the concept of semaphore.
It has a constructor and two methods acquire() and release().
The acquire() method decreases the semaphore count if the
count is greater than zero. Else it blocks till the count is
greater than zero.
The release() method increases the semaphore count and
wakes up one of the threads waiting on the semaphore.
Class semaphore([value])
The optional argument gives the initial value for the internal
counter; it defaults to 1.
PYTHON ADVANCED PROGRAMMING
acquire([blocking])
Acquire a semaphore.
When invoked without arguments: if the internal counter is larger than
zero on entry, decrement it by one and return immediately. If it is zero on
entry, block, waiting until some other thread has called release() to
make it larger than zero. This is done with proper interlocking so that if
multiple acquire() calls are blocked, release() will wake exactly one
of them up. The implementation may pick one at random, so the order in
which blocked threads are awakened should not be relied on. There is
no return value in this case.
PYTHON ADVANCED PROGRAMMING
When invoked with blocking set to true, do the same
thing as when called without arguments, and return
true. When invoked with blocking set to false, do not
block. If a call without an argument would block, return
false immediately; otherwise, do the same thing as
when called without arguments, and return true.
release()
Release a semaphore, incrementing the internal counter
by one. When it was zero on entry and another thread is
waiting for it to become larger than zero again, wake up
that thread.
PYTHON ADVANCED PROGRAMMING
import threading
from time import sleep
sem = threading.Semaphore()
def fun1():
print("fun1 starting")
sem.acquire()
for loop in range(1,5):
print("Fun1 Working in loop")
sleep(1)
sem.release()
print("fun1 finished")
PYTHON ADVANCED PROGRAMMING
def fun2():
print("fun2 starting")
while not sem.acquire(blocking=False):
print("Fun2 No Semaphore available")
sleep(1)
else:
print("Got Semphore")
for loop in range(1, 5):
print("Fun2 Working loop too")
sleep(1)
sem.release()
PYTHON ADVANCED PROGRAMMING
t1 = threading.Thread(target = fun1)
t2 = threading.Thread(target = fun2)
t1.start()
t2.start()
t1.join()
t2.join()
print("All Threads done Exiting")
PYTHON ADVANCED PROGRAMMING
# Then you do whatever sensitive thing needed to be
restricted to five threads.
# When you're finished, you release the semaphore, and it
goes back to 5.
s1.release()
# That's all fine, but you can also release it without acquiring
it first.
s1.release()
# The counter is now 6! That might make sense in some
situations, but not in most.
print(s1._value) # => 6
# If that doesn't make sense in your situation, use a
BoundedSemaphore.
PYTHON ADVANCED PROGRAMMING
s2 = BoundedSemaphore(5) # Start at 5.
s2.acquire() # Lower to 4.
s2.release() # Go back to 5.
print(s2._value)
try:
s2.release() # Try to raise to 6, above
starting value.
print(s2._value)
except ValueError:
print('As expected, it complained.')
THANK YOU
Chitra G M and P Rama Devi
Department of CSE
[email protected][email protected]
