What are scripting languages?
Scripting Languages
Unix shells: sh, ksh, bash
job control
Perl
Slashdot, bioinformatics, financial data processing, cgi
Python
Kathleen Fisher
System administration at Google BitTorrent file sharing system
Ruby
Various blogs, data processing applications
QuickTime and a TIFF (Uncompressed) decompresso are needed to see this picture.
PHP
Yahoo web site
JavaScript
Google maps
The glue that holds the web together
Characteristics
Interpreted (no compilation step) Dynamically typed High-level model of underlying machine Garbage collected Dont have to declare variables
Design philosophy
Often people, especially computer engineers, focus on the machines. They think, "By doing this, the machine will run faster. By doing this, the machine will run more effectively. By doing this, the machine will something something something." They are focusing on the machines. But in fact we need to focus on humans, on how humans care about doing programming or operating the application of the machines. We are the masters. They are the slaves. Yukihiro Matz Matsumoto Creator of Ruby
QuickTime and a TIFF (Uncompressed) decompresso are needed to see this picture.
Designed to support quick programming
Demo: Getting the homework
What if I dont want to go to the web site to see if I have cs242 homework? Write a script to check for me!
> hwk http://www.stanford.edu/class/cs242/handouts/index.html Hwk 1 was due on Wednesday, October 05. Hwk 2 was due on Wednesday, October 12. Hwk 3 is due on Wednesday, October 19.
#!/sw/bin/ruby require 'uri'; require 'net/http' uri= URI.parse(ARGV[0]) h=Net::HTTP.new(uri.host,80) resp,data = h.get(uri.path) hwk = {} if resp.message == "OK" data.scan(/Homework (\d*) \(due (\d*)\/(\d*)\)/)\ {|x,y,z| hwk[x] = Time.local(2005,y,z)} end hwk.each{| assignment, duedate| if duedate < (Time.now - 60 * 60 * 24) puts "Hwk #{assignment} was due on #{duedate.strftime("%A, %B %d")}." else puts "Hwk #{assignment} is due on #{duedate.strftime("%A, %B %d")}." end }
#!/sw/bin/ruby require 'uri'; require 'net/http'
#!/sw/bin/ruby require 'uri'; require 'net/http'
Shebang
uri= URI.parse(ARGV[0]) h=Net::HTTP.new(uri.host,80) resp,data = h.get(uri.path) hwk = {} if resp.message == "OK" data.scan(/Homework (\d*) \(due (\d*)\/(\d*)\)/)\ {|x,y,z| hwk[x] = Time.local(2005,y,z)} end hwk.each{| assignment, duedate| if duedate < (Time.now - 60 * 60 * 24) puts "Hwk #{assignment} was due on #{duedate.strftime("%A, %B %d")}." else puts "Hwk #{assignment} is due on #{duedate.strftime("%A, %B %d")}." end } uri = URI.parse(ARGV[0]) h = Net::HTTP.new(uri.host,80)
Many useful libraries
resp,data = h.get(uri.path) hwk = {} if resp.message == "OK" data.scan(/Homework (\d*) \(due (\d*)\/(\d*)\)/)\ {|x,y,z| hwk[x] = Time.local(2005,y,z)} end hwk.each{| assignment, duedate| if duedate < (Time.now - 60 * 60 * 24) puts "Hwk #{assignment} was due on #{duedate.strftime("%A, %B %d")}." else puts "Hwk #{assignment} is due on #{duedate.strftime("%A, %B %d")}." end }
#!/sw/bin/ruby require 'uri'; require 'net/http' uri= URI.parse(ARGV[0]) h=Net::HTTP.new(uri.host,80)
#!/sw/bin/ruby require 'uri'; require 'net/http'
Powerful regular expression support
uri= URI.parse(ARGV[0]) h=Net::HTTP.new(uri.host,80)
Associative arrays
resp,data = h.get(uri.path) hwk = {} if resp.message == "OK" data.scan(/Homework (\d*) \(due (\d*)\/(\d*)\)/)\ {|x,y,z| hwk[x] = Time.local(2005,y,z)} end hwk.each{| assignment, duedate| if duedate < (Time.now - 60 * 60 * 24) puts "Hwk #{assignment} was due on #{duedate.strftime("%A, %B %d")}." else puts "Hwk #{assignment} is due on #{duedate.strftime("%A, %B %d")}." end }
resp,data = h.get(uri.path) hwk = {} if resp.message == "OK" data.scan(/Homework (\d*) \(due (\d*)\/(\d*)\)/)\ {|x,y,z| hwk[x] = Time.local(2005,y,z) } end hwk.each{| assignment, duedate| if duedate < (Time.now - 60 * 60 * 24) puts "Hwk #{assignment} was due on #{duedate.strftime("%A, %B %d")}." else puts "Hwk #{assignment} is due on #{duedate.strftime("%A, %B %d")}." end }
#!/sw/bin/ruby require 'uri'; require 'net/http' uri= URI.parse(ARGV[0]) h=Net::HTTP.new(uri.host,80)
String processing
Shebang
In Unix systems, shebang tells the O/S how to evaluate an executable text file.
doit: #! interp-path prog-text > interp-path doit args > ./doit args
resp,data = h.get(uri.path) hwk = {} if resp.message == "OK" data.scan(/Homework (\d*) \(due (\d*)\/(\d*)\)/)\ {|x,y,z| hwk[x] = Time.local(2005,y,z)} end hwk.each{| assignment, duedate| if duedate < (Time.now - 60 * 60 * 24) puts "Hwk #{assignment} was due on #{duedate.strftime("%A, %B %d")}." else puts "Hwk #{assignment} is due on #{duedate.strftime("%A, %B %d")}." end }
Advantages: Dont need file extensions, program looks built-in, and can change implementation transparently.
Large standard library
Date, ParseDate File, Tempfile GetoptLong: processing command line switches profile: automatic performance profiling Pstore: automatic persistence BasicSocket, IPSocket, TCPSocket, TCPServer, UDPSocket, Socket Net::FTP, Net::HTTP, Net::HTTPResponse, Net::POPMail, Net::SMTP, Net::Telnet CGI: cookies, session mngt.
Contributing users
Ruby Application Archive (RAA)
http://raa.ruby-lang.org/ 144 library categories, 833 libraries available eg: URI library, database access
Comprehensive Perl Archive Network (CPAN)
http://www.cpan.org/ 8853 Perl modules from 4655 authors With Perl, you usually dont have to write much code: just find the code that somebody else has already written to solve your problem.
QuickTime and a TIFF (Uncompressed) decompressor are needed to see this picture.
Example: URI and HTTP libs
require 'uri'; require 'net/http' uri = URI.parse(ARGV[0]) h = Net::HTTP.new(uri.host,80) resp,data = h.get(uri.path)
Example: URI and HTTP libs
require 'uri'; require 'net/http' uri = URI.parse(ARGV[0]) h = Net::HTTP.new(uri.host,80) resp,data = h.get(uri.path)
Require clauses cause Ruby to load named libraries.
URI.parse converts argument string into a uri object, with host and path components (among other things).
Example: URI and HTTP libs
require 'uri'; require 'net/http' uri = URI.parse(ARGV[0]) h = Net::HTTP.new(uri.host,80) resp,data = h.get(uri.path)
Example: URI and HTTP libs
require 'uri'; require 'net/http' uri = URI.parse(ARGV[0]) h = Net::HTTP.new(uri.host,80) resp,data = h.get(uri.path)
Net::HTTP.new creates an http connection object, ready to converse with the specified host on the indicated port.
h.get asks to retrieve the headers and content of the given path from the site associated with h. It returns a pair of the response code and the payload data.
Strings
Strings are just objects: hermione.length yields 8 Strings can include expressions with # operator: 3 + 4 = #{3+4} yields 3 + 4 = 7 Plus operator concatenates strings: Hermione + Granger yields Hermione Granger Many more operations (more than 75!).
Powerful regular expressions
Regular expressions are patterns that match against strings, possibly creating bindings in the process. Uses greedy matching. In Ruby, regular expressions are objects created with special literal forms:
/reg-exp/ or %r{reg-exp}
Examples:
/arr/ matches strings containing arr /\s*\|\s*/ matches a | with optional white space
Simple matches
All characters except .|()[\^{+$*? match themselves .|()[\^{+$*? . Precede by \ to match directly Matches any character
Compound matches
re* re+ re{m,n} re? re1 | re2 () Matches 0 or more occurrences of re. Matches 1 or more occurrences of re. Matches at least m and no more than n occurrences of re. Matches zero or one occurrence of re. Matches either re1 or re2 Groups regular expressions and directs interpretor to introduce bindings for intermediate results.
[characters] Matches any single character in [] May include ranges; Initial ^ negates \d Matches any digit \w \s ^ $ Matches any word character Matches any whitespace Matches the beginning of a line Matches the end of a line
Introducing bindings
Matching a string against a regular expression causes interpretor to introduce bindings:
$` $& $ $1, $2, Portion of string that preceded match. Portion of string that matched. Portion of string after match. Portion of match within ith set of parentheses.
Using regular expressions
We can use these bindings to write functions to display the results of a match:
def showre(str,regexp) if str =~ regexp "#{$`}--->#{$&}<---#{$'}" else "match failed" end end def showone(str,regexp) if str =~ regexp "#{$1}" else "match failed" end end
showre(hello, /l+/) yields he--->ll<---o showone(hello, /(l+)/) yields ll
Example: Finding homework
To match the homework assignment portion of the course website, we can use the regular expression:
/Homework (\d*) \(due (\d*)\/(\d*)\)/
Example: Finding homework
To match the homework assignment portion of the course website, we can use the regular expression:
/Homework (\d*) \(due (\d*)\/(\d*)\)/
<TH>3</TH> <TD>Homework 3 (due 10/19)</TD> <!--<TD><a href="hw1.ps">PS</a></TD>--> <TD><a href="hw3.pdf">PDF</a></TD> </TR>
<TH>3</TH> <TD>Homework 3 (due 10/19)</TD> <!--<TD><a href="hw1.ps">PS</a></TD>--> <TD><a href="hw3.pdf">PDF</a></TD> </TR>
Associative Arrays
Like arrays, indexed collection of objects Unlike arrays, index can be any kind of object
aa = {'severus' => 'snape', 'albus' => 'dumbledore'} aa['harry'] = 'potter' aa['hermione'] = 'granger' aa['ron'] = 'weasley' def putaa(aa) aa.each{|first,last| puts first + " " + last} end puts aa['ginny']
#!/sw/bin/ruby require 'uri'; require 'net/http' uri= URI.parse(ARGV[0]) h=Net::HTTP.new(uri.host,80) resp,data = h.get(uri.path) hwk = {} if resp.message == "OK" data.scan(/Homework (\d*) \(due (\d*)\/(\d*)\)/)\ {|x,y,z| hwk[x] = Time.local(2005,y,z)} end hwk.each{| assignment, duedate| if duedate < (Time.now - 60 * 60 * 24) puts "Hwk #{assignment} was due on #{duedate.strftime("%A, %B %d")}." else puts "Hwk #{assignment} is due on #{duedate.strftime("%A, %B %d")}." end }
Other features
Reflection allows querying an object for its capabilities at run-time
obj.class returns the class of an object obj.methods returns its methods
Tainting
Problem: How to ensure untrusted input data does not corrupt ones system? Solution:
Track the influence of input data, marking dependent data as tainted. Disallow risky actions based on tainted data depending upon a programmer-specified safety level. In Ruby, the default safety level (0) permits everything. Levels 1 to 4 add various restrictions; The demo program fails to run at level 1.
Native modules
Relatively easy to implement Ruby modules in C for better performance. Provides APIs to access Ruby objects as C data structures
Swig allows wrapping of existing C/C++ libraries to import into various scripting languages.
Design Slogans
Optimize for people, not machines Principle of Least Surprise (after you know the language well) Theres more than one way to do it (TMTOWTDI, pronounced Tim Toady) No built-in limits Make common things short Make easy tasks easy and hard tasks possible Executable pseudo-code
Some downsides
Write once, read never
Perl in particular seems to facilitate writing difficult to read programs. A consequence of TMTOWDI?
Performance can be difficult to predict
Fast: regular expression processing Slow: threads Shell calls?
Errors are detected dynamically
The past
Unix shells: sh, ksh, bash (1971) Perl (Larry Wall, 1987) Python (Guido van Rossum, 1990) Ruby (Yukihiro Matz Matsumoto, 1995) PHP (Rasmus Lerdorf, 1995) JavaScript (Brendan Eich, 1995)
The present
Ruby, Perl, Python, etc., are all open source. Rely on volunteers to
QuickTime and a TIFF (Uncompressed) decompresso are needed to see this picture.
Two things to note: Each language was driven by one person 1995 was a big year
Write documentation Write test cases Maintain the systems Port to new platforms Fix bugs Implement libraries Implement new features and more
Sprit of fun
The joy of Ruby Golfing
Competitions in which each entrant endeavors to solve some problem with the minimum of keystrokes.
Obfuscation contests
Any guesses as to what this Perl program does?
@ P =splt/" i/,.URRU U\c8R" ;@d=sp i/, rekcah x lt/ \n inU /l reh reP tona t suJ" ;sub p{ @ p{" r$p" "u$p" , }=(P,P) ipe" ;p r$p" " , u$p" ;++$p; ($q*=2)+=$f= fo ;map{$P=$P[$ ! rk f^ord ($p{$_} )&6] ;$p{$_}= ^$P/ / ix?$P:c ose$_}keys%p}p ;p ;p l ;p ;p ;map{$p{$_}=~/^ . /&& [P ] c lose$_}%p;wa t un i ;map{ r &&<$_>}%p;$_=$d[$q ;s i tl$? /^ / ] leep rand(2)f\S ;p in i/ / r t
Poetry
About the language In the language Generated by the language Original and transliterations
pr t STD O UT q in Jus another Per hacker t l , un less $spr ing Lar Wal ry l
Obfuscation competitions
Obfuscation contests
Any guesses as to what this Perl program does?
@ P =splt/" i/,.URRU U\c8R" ;@d=sp i/, rekcah x lt/ \n inU /l reh reP tona t suJ" ;sub p{ @ p{" r$p" "u$p" , }=(P,P) ipe" ;p r$p" " , u$p" ;++$p; ($q*=2)+=$f= fo ;map{$P=$P[$ ! rk f^ord ($p{$_} )&6] ;$p{$_}= ^$P/ / ix?$P:c ose$_}keys%p}p ;p ;p l ;p ;p ;map{$p{$_}=~/^ . /&& [P ] c lose$_}%p;wa t un i ;map{ r &&<$_>}%p;$_=$d[$q ;s i tl$? /^ / ] leep rand(2)f\S ;p in i/ / r t
On to the future
Ruby 2, Python 3000, Perl 6, all in the works.
User communities working with language originator to plan the future. Projects to revise languages without worry about backwards compatibility.
It slowly prints: Just another Perl / Unix hacker It works by forking 32 processes, each of which prints one letter in the message. It uses pipes for coordination. http://perl.plover.com/obfuscated/ describes how it works.
Perl 6 (a very dynamic language)
QuickTime and a TIFF (Uncompressed) decompressor are needed to see this picture.
Parrot runtime system, designed to be used by other scripting languages as well. Will they? Pugs implementation of Perl 6 completed (in Haskell).
"We're really serious about reinventing everything that needs reinventing." --Larry Wall
QuickTime and a TIFF (Uncompressed) decompressor are needed to see this picture.
