0 ML Tools Part1 Python
0 ML Tools Part1 Python
Prof. S. Areibi
School of Engineering
University of Guelph
Downloading Python
Downloading/Running
Python
Download Python from
– https://www.python.org/downloads/
– This will create an IDLE Shell and a Python Terminal
– With the IDLE Shell you can create a new file and write your Python Code.
– You can then choose “Run” to execute the program
Downloading and Installing Anaconda:
Steps: https://www.youtube.com/watch?v=HW29067qVWk
https://www.youtube.com/watch?v=aN6OVm0mTHo
3
Jupyter Notebook
Jupyter Notebooks
Notebooks are an excellent way to document an engineering
process because they allow you to combine nicely formatted text
with code and results.
The Jupyter Notebook is a web application that allows you to
create and share documents that contain live code, equations,
visualizations and explanatory text.
Jupyter notebooks use a serialized data storage format, JSON, to
store the document.
They allow you to integrate many different text processing
formats, including HTML, Markdown and LaTex (a popular word
processor in STEM).
Notebooks are integrated into Anaconda.
https://www.youtube.com/watch?v=q_BzsPxwLOE&list=PLeo1K3hjS3uuZPwzACannnFSn9qHn8to8
5
Jupyter Notebooks:
•
Resources
Jupyter Notebook: An Introduction
– https://realpython.com/jupyter-notebook-introduction/#creating-a-notebook
• How to Use Jupyter Notebook in 2020: A Beginner’s Tutorial
– https://www.dataquest.io/blog/jupyter-notebook-tutorial/
• Learn Jupyter (Tutorial)
– https://www.tutorialspoint.com/jupyter/index.htm
• What is Jupyter Notebook (YouTube)
– https://www.youtube.com/watch?v=q_BzsPxwLOE&list=PLeo1K3hjS3uuZPwzACannnFSn9qHn8to8
• Introduction to machine learning with Jupyter notebooks:
– https://developers.redhat.com/articles/2021/05/21/introduction-machine-learning-jupyter-notebooks#
• Jupyter Notebook Tutorial : The Definitive Guide:
– https://www.datacamp.com/community/tutorials/tutorial-jupyter-notebook
6
Python Basics
Introduction to Python
Python is a high-level programming language
Python is an Object Oriented Programming(OOP) language
Practically everything can be treated as an object
Strings, as objects, have methods that return the result of a
function on the string
Open source and community driven
“Batteries Included”
A standard distribution includes many modules
Source can be compiled or run just-in-time
Similar to perl, tcl, ruby
https://www.youtube.com/watch?v=_xQNeOTRyig
8
Features of Python
9
Uses of Python
10
Python
Frameworks/Libs
11
Best Python IDEs
12
Companies using
Python
13
Python: Preliminary
Example Python
“”” 3
Declaring a float 4.5
“”” hello world
myNumber2 = 4.5
print(myNumber2)
# Declaring a string
myString ="hello world"
print(myString)
16
White Space
Whitespace is meaningful in Python: especially
indentation and placement of newlines
Use a newline to end a line of code
Use \ when must go to next line prematurely
No braces {} to mark blocks of code, use
consistent indentation instead
• First line with less indentation is outside of the block
• First line with more indentation starts a nested block
Colons start of a new block in many constructs,
e.g. function definitions, then clauses
17
Naming Rules
· Names are case sensitive and cannot start
with a number. They can contain letters,
numbers, and underscores.
bob Bob _bob _2_bob_ bob_2 BoB
· There are some reserved words:
and, assert, break, class, continue,
def, del, elif, else, except, exec,
finally, for, from, global, if,
import, in, is, lambda, not, or,
pass, print, raise, return, try,
while
18
Naming Conventions
The Python community has these recommend-ed
naming conventions
joined_lower for functions, methods and, attributes
joined_lower or ALL_CAPS for constants
StudlyCaps for classes
camelCase only to conform to pre-existing
conventions
Attributes: interface, _internal, __private
19
Python: Variables
Variables
In other programming languages like C, C++, and Java, you will
need to declare the type of variables but in Python you don’t
need to do that.
Just type in the variable and when values will be given to it, then
it will automatically know whether the value given would be an
int, float, or char or even a String.
Need to assign (initialize)
• use of uninitialized variable raises exception
Everything is a "variable":
• Even functions, classes, modules
21
Variables in Python
>>> x, y = 2, 3
>>> x
2
>>> y
3
This makes it easy to swap values
>>> x, y = y, x
Assignments can be chained
>>> a = b = x = 2
23
Basic Data Types
Integers (default for numbers)
z = 5 / 2 # Answer 2, integer division
Floats
x = 3.456
Strings
• Can use “” or ‘’ to specify with “abc” == ‘abc’
• Unmatched can occur within the string:
“matt’s”
• Use triple double-quotes for multi-line strings or
strings than contain both ‘ and “ inside of them:
“““a‘b“c”””
24
Basic Data Types
# Python has several data types including int, float, string, complex, bool
a=3
type(a)
int
b = 3.14
type(b)
float
c = 14 + 3j
type(c)
complex
d = True
type(d)
bool
e = “Hello”
type(e)
str
25
Python: Input
Python I/O
• We can take input from the user via the keyboard and hence
manipulate it for simply display it.
• The function ``input()” is used to take input from the user.
27
Python I/O
28
Python Math
• You can prompt a user to enter a value via screen using “input”
• In this example two variables “x” and “y” are entered by the
user and several math operations are performed on these
variables and printed to the screen.
29
Python: Lists, Tuples,
Strings
Python Objects
1. Tuple: (‘john’, 32, [CMSC])
• A simple immutable ordered sequence of items
• Items can be of mixed types, including collection
types
2. Strings: “John Smith”
• Immutable
• Conceptually very much like a tuple
3. List: [1, 2, ‘john’, (‘up’, ‘down’)]
• Mutable ordered sequence of items of mixed
types
31
Python Objects
All three sequence types (tuples, strings, and
lists) share much of the same syntax and
functionality.
Key difference:
Tuples and Strings are immutable
Lists are mutable
The operations shown in this section can be
applied to all sequence types
most examples will just show the operation
performed on one
32
Objects: Definition
• Define tuples using parentheses and commas
>>> tu = (23, ‘abc’, 4.56, (2,3), ‘def’)
33
Objects: Accessing
Members
• Access individual members of a tuple, list, or string
using square bracket “array” notation
• Note that all are 0 based…
>>> tu = (23, ‘abc’, 4.56, (2,3), ‘def’)
>>> tu[1] # Second item in the tuple.
‘abc’
>>> li = [“abc”, 34, 4.34, 23]
>>> li[1] # Second item in the list.
34
>>> st = “Hello World”
>>> st[1] # Second character in string.
‘e’
34
Objects: “+” Operator
The + operator produces a new tuple, list, or string
whose value is the concatenation of its arguments.
35
Objects: “*” Operator
The * operator produces a new tuple, list, or string
that “repeats” the original content.
>>> (1, 2, 3) * 3
(1, 2, 3, 1, 2, 3, 1, 2, 3)
>>> [1, 2, 3] * 3
[1, 2, 3, 1, 2, 3, 1, 2, 3]
>>> “Hello” * 3
‘HelloHelloHello’
Objects: Lists
String
A string is a sequence of characters.
In Python, a string is a sequence of Unicode characters. Unicode
was introduced to include every character in all languages and
bring uniformity in encoding.
Python has a great support for strings.
Strings can be created by enclosing characters inside a single
quote or double-quotes. Even triple quotes can be used in
Python but generally used to represent multiline strings and
docstrings.
38
Strings
# defining strings in Python
# all of the following are equivalent
my_string = 'Hello'
print(my_string)
my_string = "Hello"
print(my_string)
my_string = '''Hello'''
print(my_string)
39
Strings
hello = 'hello' # String literals can use single quotes
world = "world" # or double quotes; it does not matter.
print(hello) # Prints "hello"
print(len(hello)) # String length; prints "5"
hw = hello + ' ' + world # String concatenation
print(hw) # prints "hello world"
hw12 = '%s %s %d' % (hello, world, 12) # sprintf style string formatting
print(hw12) # prints "hello world 12"
s = "hello"
print(s.capitalize()) # Capitalize a string; prints "Hello"
print(s.upper()) # Convert a string to uppercase; prints "HELLO"
print(s.rjust(7)) # Right-justify a string, padding with spaces; prints " hello"
print(s.center(7)) # Center a string, padding with spaces; prints " hello "
print(s.replace('l', '(ell)')) # Replace all instances of one substring with another;
# prints "he(ell)(ell)o"
print(' world '.strip()) # Strip leading and trailing whitespace; prints "world"
40
Objects: Lists
Lists
A List is the Python equivalent of an array (but resizeable)
A List is the most basic Data Structure in python.
List is a mutable data structure i.e items can be added to list
later after the list creation.
The append() function is used to add data to the list.
print(nums) list
type(nums)
42
Lists
xs = [3, 1, 2] # Create a list
print(xs, xs[2]) # Prints "[3, 1, 2] 2"
print(xs[-1]) # Negative indices count from the end of the list; prints "2"
xs[2] = 'foo' # Lists can contain elements of different types
print(xs) # Prints "[3, 1, 'foo']"
xs.append('bar’) # Add a new element to the end of the list
print(xs) # Prints "[3, 1, 'foo', 'bar']"
x = xs.pop() # Remove and return the last element of the list
print(x, xs) # Prints "bar [3, 1, 'foo']"
43
Lists: append/insert
44
Lists: append vs
extend
· + creates a fresh list with a new memory ref
· extend operates on list li in place.
>>> li.extend([9, 8, 7])
>>> li
[1, 2, ‘i’, 3, 4, 5, ‘a’, 9, 8, 7]
· Potentially confusing:
• extend takes a list as an argument.
• append takes a singleton as an argument.
>>> li.append([10, 11, 12])
>>> li
[1, 2,‘i’, 3, 4, 5,‘a’, 9, 8, 7,[10, 11, 12]]
45
Lists:
index/count/remove
Lists have many methods, including index, count,
remove, reverse, sort
>>> li = [‘a’, ‘b’, ‘c’, ‘b’]
>>> li.index(‘b’) # index of 1st occurrence
1
>>> li.count(‘b’) # number of occurrences
2
>>> li.remove(‘b’) # remove 1st occurrence
>>> li
[‘a’, ‘c’, ‘b’]
Lists: reverse/sort
>>> li = [5, 2, 6, 8]
>>> li.sort(some_function)
# sort in place using user-defined comparison
Lists: Change Value
48
Lists: Access index
>>> t = [23, “abc”, 4.56, (2,3), “def”]
Positive index: count from the left, starting with 0
>>> t[1]
‘abc’
49
Lists: Access using “:”
>>> t = [23, “abc”, 4.56, (2,3), “def”]
Return a copy of the container with a subset of the
original members. Start copying at the first index,
and stop copying before second.
>>> t[1:4]
[‘abc’, 4.56, (2,3)]
50
Lists: Access using “:”
>>> t = [23, “abc”, 4.56, (2,3),“def”]
51
Lists: Access using “:”
· [ : ] makes a copy of an entire sequence
>>> t[:]
(23, ‘abc’, 4.56, (2,3), ‘def’)
52
Lists: “in” Operator
· Boolean test whether a value is inside a container:
>>> t = [1, 2, 4, 5]
>>> 3 in t
False
>>> 4 in t
True
>>> 4 not in t
False
55
Tuples vs. Lists
Lists slower but more powerful than tuples
Lists can be modified, and they have lots of
handy operations and mehtods
Tuples are immutable and have fewer features
To convert between tuples and lists use the list()
and tuple() functions:
li = list(tu)
tu = tuple(li)
56
Objects: Dictionary
More Python Objects
• Lists (mutable sets of strings)
– var = [] # create list
– var = [‘one’, 2, ‘three’, ‘banana’]
– You can add items to the List created
• Tuples (immutable sets)
– var = (‘one’, 2, ‘three’, ‘banana’)
– You cannot add any new elements
• Dictionaries (associative arrays or ‘hashes’)
– var = {} # create dictionary
– var = {‘lat’: 40.20547, ‘lon’: -74.76322}
– var[‘lat’] = 40.2054
• Each has its own set of methods
58
Dictionary
A Dictionary is similar to a list.
Dictionary is a mutable data structure i.e items can be added to
list later after the Dictionary creation.
Output:
‘John Adams’
Output:
59
Dictionary: Access
Method
• Dictionary.keys() lists all keys of a dictionary
• Dictionary.values() list all values
• Dictionary.items() List of item Tuples
60
Dictionary: Access/Modify
• Dictionary.get() get the key value
• Dictionary.update() change the value of the key
61
Objects: Set
Sets
A Set is similar to a List.
It displays only unique non-repeated items
{'a', 's', 'h', 'y', 'i', 'c', 'r', 'o', 'u', 't'}
63
Python: Control Structures
Control Structures
if condition: while condition:
statements statements
[elif condition:
statements] ... for var in sequence:
else: statements
statements
break
continue
65
Grouping Indentation
In Python: In C:
66
Selection in Python
Selection in Python is made using the two keywords
‘if’ and ‘elif’ (else if) and else
# If-elif-else construct
A = 400
B = 250 Output:
If B > A: A is Greater than B
# Notice indentation of statement
print(“B is Greater than A”)
elif A == B:
print (“A is Equal to B”
Else:
print(“A is Greater than B”)
68
Looping with For
• For allows you to loop over a block of code a set number of times
69
Looping with For
• For is great for manipulating lists:
70
Looping with While
• Similarly, while allows you to loop over a block of code a set
number of times
71
Python: Functions
Functions, Procedures
• A Python Function is a block of related statements designed to
perform a computational, logical, or evaluative task.
Syntax:
Syntax:
def function_name(arguments):
def function-name():
statement(s)
#function body
return expression
73
Creating/Calling a
Function
Creating a Function:
• We can create a Python function using the def keyword.
def fun():
print("Welcome to ENG6500!!")
Calling a Function
• We can call a function by using the name of the function followed by
parenthesis containing parameters of that particular function.
74
Example Function
# Python program to illustrate functions
def hello():
print("hello")
print("hello again")
# calling function
hello()
# calling function
hello()
Output:
hello
hello again
hello
hello again
75
Function Arguments
Arguments of a Function
• Arguments are the values passed inside the parenthesis of the function.
• A function can have any number of arguments separated by a comma.
def evenOdd(x):
if (x % 2 == 0):
print("even")
else: Output
print("odd") even
odd
# Driver code to call the function
evenOdd(2)
evenOdd(3)
76
Example Function
def gcd(a, b):
"greatest common divisor"
while a != 0:
a, b = b%a, a # parallel assignment
return b
>>> gcd.__doc__
'greatest common divisor'
>>> gcd(12, 20)
4
77
Example Function
• Now as we know any program starts from a ‘main’ function…
• Let’s create a main function like in many other programming languages
def Main():
print("Startin @ Main") Output:
# calling the getInteger function and
Starting @ Main
# storing its returned value in the output variable
output = getInteger() Enter integer: 5
print(output)
78
Python: Modules
Modules
• Python has a very rich module library that has several functions to do
many tasks.
• The ‘import’ keyword is used to import a particular module into your
python code.
• For instance, consider the following program.
if __name__=="__main__":
Main()
80
Python: Files
Files
• Files are manipulated by creating a file object
– f = open("points.txt", "r")
• The file object then has new methods
– print f.readline() # prints line from file
• Files can be accessed to read or write
– f = open("output.txt", "w")
– f.write("Important Output!")
• Files are iterable objects, like lists
82
Python: Keywords
Keywords
Keywords in Python are reserved words that can not be used as a
variable name, function name, or any other identifier.
List of all keywords in
Python
https://www.geeksforgeeks.org/python-keywords/
84
Keywords
We can also get all the keyword names using the below code.
# Python code to demonstrate working of iskeyword()
Output:
[‘False’, ‘None’, ‘True’, ‘and’, ‘as’, ‘assert’, ‘async’, ‘await’, ‘break’, ‘class’, ‘continue’, ‘def’,
‘del’, ‘elif’, ‘else’, ‘except’, ‘finally’, ‘for’, ‘from’, ‘global’, ‘if’, ‘import’, ‘in’, ‘is’, ‘lambda’,
‘nonlocal’, ‘not’, ‘or’, ‘pass’, ‘raise’, ‘return’, ‘try’, ‘while’, ‘with’, ‘yield’]
85
Platforms: Linux
Running on Linux
On Unix…
% python
>>> 3+3
6
· Python prompts with ‘>>>’.
· To exit Python (not Idle):
• In Unix, type CONTROL-D
• In Windows, type CONTROL-Z + <Enter>
• Evaluate exit()
87
Running on Linux
· Call python program via the python interpreter
% python fact.py
· Make a python file directly executable by
• Adding the appropriate path to your python
interpreter as the first line of your file
#!/usr/bin/python
• Making the file executable
% chmod a+x fact.py
• Invoking file from Unix command line
% fact.py
88
Example `script’:
fact.py
#! /usr/bin/python
def fact(x):
"""Returns the factorial of its argument, assumed to be a posint"""
if x == 0:
return 1
return x * fact(x - 1)
print
print ’N fact(N)’
print "---------"
for n in range(10):
print n, fact(n)
89
Platforms: Notebooks
Jupyter/Colab
Notebooks
· A Jupyter notebook lets you write and execute Python code locally in
your web browser.
· Jupyter notebooks make it very easy to tinker with code and execute it
in bits and pieces; for this reason they are widely used in scientific
computing.
· Colab on the other hand is Google’s flavor of Jupyter notebooks that is
particularly suited for machine learning and data analysis and that runs
entirely in the cloud.
· Colab is basically Jupyter notebook on steroids: it’s free, requires no
setup, comes preinstalled with many packages, is easy to share with the
world, and benefits from free access to hardware accelerators like
GPUs and TPUs (with some caveats).
https://cs231n.github.io/python-numpy-tutorial/
91
Summary
Summary
o Python is a high-level, general-purpose and a very popular
programming language that was developed by Guido van
Rossum in the early 1990s.
o Python is an interpreted language i.e. it’s not compiled and
the interpreter will check the code line by line.
o Python programming language (latest Python 3) is being
used in web development, Machine Learning applications,
along with all cutting-edge technology in Software
Industry.
o Python Programming Language is very well suited for
Beginners, also for experienced programmers with other
programming languages like C++ and Java.
93
Python: Resources
Additional Python
Resources
• Python Homepage
http://www.python.org/
• Dive Into Python
http://www.diveintopython.org/
• Learning Python, 3rd Edition
http://www.oreilly.com/catalog/9780596513986/
• Getting Started Writing Geoprocessing Scripts
Available on ESRI's support page
95
Resources
Misc. Resources:
YouTube
o Introduction to Python Basics:
https://open.ed.ac.uk/5861-2/
o Python for Beginners:
https://www.youtube.com/watch?v=kqtD5dpn9C8
o How to install Anaconda
https://www.youtube.com/watch?v=71DMPhmxOS4
o Introduction to Python-1 (Simplilearn)
https://www.youtube.com/watch?v=_xQNeOTRyig
o Learn Python – Full Course
https://www.youtube.com/watch?v=rfscVS0vtbw
97
Misc. Resources:
Documents
o Python Basics:
• https://www.learnpython.org/
• https://www.w3schools.com/python/default.asp
• https://www.geeksforgeeks.org/python-programming-language/?ref=shm
• https://machinelearningmastery.com/logging-in-python/?utm_source=drip&utm_medium=email&utm
_campaign=An+intro+to+decorators+in+Python&utm_content=An+intro+to+decorators+in+Python
o Python Numpy:
• https://cs231n.github.io/python-numpy-tutorial/
98