Showing posts with label ODBC. Show all posts
Showing posts with label ODBC. Show all posts

Wednesday, April 10, 2013

Using xtopdf and pypyodbc to publish MS Access database data to PDF


By Vasudev Ram

I had blogged about pypyodbc, a pure-Python ODBC library, recently.

Using pypyodbc with my xtopdf toolkit for PDF creation, you can publish your MS Access database data to PDF.

Here is some example code to publish MS Access data to PDF:

First, the program create_ppo_mdb.py, shown below, creates an MS Access database called fruits.mdb, then creates a table called fruits in it, and inserts 3 records into the table:

# create_ppo_mdb.py

import pypyodbc 
             
pypyodbc.win_create_mdb('.\\fruits.mdb')
connection_string = 'Driver={Microsoft Access Driver (*.mdb)};DBQ=.\\fruits.mdb'
connection = pypyodbc.connect(connection_string)

SQL = 'CREATE TABLE fruits (id COUNTER PRIMARY KEY, fruit_name VARCHAR(25));'
connection.cursor().execute(SQL).commit()

SQL = "INSERT INTO fruits values (1, 'apple');"
connection.cursor().execute(SQL).commit()

SQL = "INSERT INTO fruits values (2, 'banana');"
connection.cursor().execute(SQL).commit()

SQL = "INSERT INTO fruits values (3, 'orange');"
connection.cursor().execute(SQL).commit()

# Uncomment the 5 lines below make the program also display the data after creating it.

#SQL = 'SELECT * FROM fruits;'
#cursor = connection.cursor().execute(SQL)
#for row in cursor:
#    for col in row:
#        print col,
#    print

cursor.close()
connection.close()

Next, the program MDBtoPDF.py, shown below, reads the data from the fruits table in the MDB database just created above, and publishes the selected records to PDF:

#-------------------------------------------------------------------

# MDBtoPDF.py
# Description: A program to convert MS Access .MDB data to PDF format.
# Author: Vasudev Ram - http://www.dancingbison.com

#-------------------------------------------------------------------

# imports

import sys 
import os
import time
import string
import pypyodbc 
from PDFWriter import PDFWriter
             
#-------------------------------------------------------------------

# globals

##------------------------ usage ---------------------------------------

def usage():

 sys.stderr.write("Usage: python " + sys.argv[0] + " MDB_DSN table_name pdf_file\n")
 sys.stderr.write("where MDB_DSN is the ODBC DSN (Data Source Name) for the\n")
 sys.stderr.write("MDB file, table_name is the name of the table in that MDB,\n")
 sys.stderr.write("whose data you want to convert to PDF, and pdf_file is the\n")
 sys.stderr.write("output PDF filename.\n")
 sys.stderr.write(sys.argv[0] + " reads the table data from the MDB and\n")
 sys.stderr.write("writes it to pdf_file.\n")

##------------------------ main ------------------------------------------

def main():

 '''Main program to convert MDB data to PDF.
 '''

 # check for right num. of args
 if (len(sys.argv) != 4):
  usage()
  sys.exit(1)

 # extract MDB DSN, table name and pdf filename from args
 mdb_dsn = sys.argv[1]
 table_name = sys.argv[2]
 pdf_fn = sys.argv[3]

 print "mdb_dsn =", mdb_dsn
 print "table_name =", table_name
 print "pdf_fn =", pdf_fn

    # build connection string
 connection_string_prefix = 'Driver={Microsoft Access Driver (*.mdb)};DBQ='
 connection_string = connection_string_prefix + mdb_dsn
 print "connection_string =", connection_string
 connection = pypyodbc.connect(connection_string)
 print "connection =", connection

 # create the PDFWriter instance
 pw = PDFWriter(pdf_fn)

 # and set some of its fields

 # set the font
 pw.setFont("Courier", 10)

 # set the page header
 gen_datetime = time.asctime()
 pw.setHeader("Generated by MDBtoPDF: Input: " + mdb_dsn + \
 " At: " + gen_datetime)

 # set the page footer
 pw.setFooter("Generated by MDBtoPDF: Input: " + mdb_dsn + \
 " At: " + gen_datetime)

 # create the separator for logical grouping of output
 sep = "=" * 60

 # print the data records section title
 pw.writeLine("MDB Data Records from MDB: %s, table: %s" % (mdb_dsn, 
  table_name))

 # print a separator line
 pw.writeLine(sep)

 # read the input MDB data and write it to the PDF file

 SQL = 'SELECT * FROM fruits;'

 cursor = connection.cursor().execute(SQL)
 for row in cursor:
  str_row = ""
  for col in row:
   str_row = str_row + str(col) + " "
  pw.writeLine(str_row)

 # close the cursor and connection
 cursor.close()
 connection.close()

 # print a separator line
 pw.writeLine(sep)

 # save current page
 pw.savePage()

 # close the PDFWriter
 pw.close()

##------------------------ Global code -----------------------------------

# invoke main

if __name__ == '__main__':
 main()

##------------------------ EOF - MDBto_PDF.py ---------------

To make the above programs work, you need to have the Reportlab toolkit v1.21 and the xtopdf toolkit installed, in addition to pypyodbc and Python 2.7. (Click on the "Branches" tab on the xtopdf page linked in the previous sentence to download xtopdf.)

I've had an interest in ODBC ever since I first worked, as team leader, on a middleware software product that used ODBC. The middleware was developed at Infosys Technologies, where I worked at the time.

Though ODBC itself had a good architecture, many driver implementations of the time (this was some years ago) were rather slow, so one of the main goals of the product was to improve the performance of client-server or desktop applications (written in Visual Basic or C) that used ODBC for database access.

I remember learning ODBC as part of the project (and teaching it to the team), and reading most of the book "Inside ODBC" by Kyle Geiger, one of the architects of ODBC - it was a fascinating book, that gave a detailed look inside the architecture of ODBC, the reasons for certain design decisions that were made, and so on.

We succeeded in meeting all the goals of the project, and that middleware product was used in many large client-server applications (using VB and Oracle / Sybase) that were developed by Infosys for its clients. I really had a lot of fun working on that project.

Related links:

ODBC entry on Wikipedia

Inside ODBC - the book, on Amazon

eGenix mxODBC Connect, from eGenix, a German Python products company.

eGenix mxODBC

unixODBC

DataDirect ODBC

iODBC

The Microsoft SQL Server ODBC Driver for Linux - it provides native connectivity from Linux to Microsoft SQL Server. (Seems to be 64-bit only).

- Vasudev Ram - Dancing Bison Enterprises

Tuesday, March 26, 2013

pypyodbc, a pure Python ODBC library

By Vasudev Ram

In my recent post, PyODBC for Linux, Windows and Mac OS X, Ben Rousch commented: "I've had better luck with pure-Python PyPyODBC than with PyODBC."

So I downloaded and tried out pypyodbc.

Features of pypyodbc (from its web page):

Design Goal: Small, Maintainable, Cross Platform, Compatible, Portable

Features
One pure Python script, runs on CPython / IronPython / PyPy , Python 3.3 / 2.4 / 2.5 / 2.6 / 2.7 , Win / Linux / Mac , 32 / 64 bit
Very similar usage as pyodbc ( can be seen like a re-implementation of pyodbc in pure Python )
Simple - the whole module is implemented in a single python script with less than 3000 lines
Built-in functions to create and compress Access MDB files on Windows

I first tried the simple test program given on the pypyodbc page, which just creates a new MS Access database using Python. It worked.

Then I added code to that test program to insert three rows to the table and then and then retrieve the rows from the database and print them.

Here is the code:
# test_ppo.py

import pypyodbc 
             
pypyodbc.win_create_mdb('C:\\vr\\pypyodbc\\db2.mdb')
connection_string = 'Driver={Microsoft Access Driver (*.mdb)};DBQ=C:\\vr\\pypyodbc\\db2.mdb'
connection = pypyodbc.connect(connection_string)

SQL = 'CREATE TABLE saleout (id COUNTER PRIMARY KEY,product_name VARCHAR(25));'
connection.cursor().execute(SQL).commit()

SQL = "INSERT INTO saleout values (1, 'apple');"
connection.cursor().execute(SQL).commit()

SQL = "INSERT INTO saleout values (2, 'banana');"
connection.cursor().execute(SQL).commit()

SQL = "INSERT INTO saleout values (3, 'orange');"
connection.cursor().execute(SQL).commit()

SQL = 'SELECT * FROM saleout;'
cursor = connection.cursor().execute(SQL)
for row in cursor:
    print row
And I got the expected output:
C:\vr\pypyodbc>python test_ppo.py
(1, u'apple')
(2, u'banana')
(3, u'orange')
- Vasudev Ram - Dancing Bison Enterprises

Thursday, March 21, 2013

PyODBC for Linux, Windows and Mac OS X

By Vasudev Ram

PyODBC is like the odbc module in the Standard Python library, but claims to work on Linux, Windows and Mac OS X. I've used the odbc module from the stdlib earlier. Using it is straightforward for basic CRUD-type usage.

PyODBC is supposed to "implement the Python Database API Specification v2.0, but additional features have been added to simplify database programming even more." I am trying out PyODBC (have taken an initial look at the examples, seems similar to the stdlib odbc module) and will report on it a little later.

ODBC on Wikipedia.

- Vasudev Ram - Dancing Bison Enterprises


Thursday, October 13, 2011

PyAudio and PortAudio - like ODBC for sound

By Vasudev Ram - dancingbison.com | @vasudevram | jugad2.blogspot.com

PyAudio and PortAudio are I/O libraries for sound (i.e. audio).

I'm stretching the analogy a bit here, but they made me think:

"... like ODBC for sound". (*)

PyAudio is a Python interface to PortAudio.

PyAudio:

http://people.csail.mit.edu/hubert/pyaudio/

Excerpt:

[ PyAudio provides Python bindings for PortAudio, the cross-platform audio I/O library. With PyAudio, you can easily use Python to play and record audio on a variety of platforms. ]

PortAudio:

http://www.portaudio.com/

PortAudio apps:

http://www.portaudio.com/apps.html

I installed PyAudio for Windows. Installation was trivial. It also automatically installed the PortAudio DLL (actually, the .pyd file).

I then tried a PyAudio Python program from the docs to play a small .WAV file. It worked.

(*) That's because PyAudio and PortAudio support both:

a) different platforms (Windows, Linux, Mac, UNIX)

b) different "Host APIs" on the same platform, where the different Host API's have, obviously, different API's, but PortAudio (and hence PyAudio) hide those differences behind a uniform interface (to some extent).


UPDATE: If you interested in checking out other Python multimedia libraries, you may also like to read this earlier post of mine about pyglet:

Playing an MP3 with pyglet and Python is easy

pyglet has some of the same benefits as PyAudio - cross-platform, easy to use, etc.

Posted via email

- Vasudev Ram @ Dancing Bison

Tuesday, September 27, 2011

Microsoft to switch back to ODBC from OLE DB, say reports

By Vasudev Ram - dancingbison.com | @vasudevram | jugad2.blogspot.com

Surprising as it may seem, Microsoft may switch back to ODBC from OLE DB.

I read about this a few days ago on the Net.

Here are some relevant links to the news about Microsoft going back to ODBC.

From the Microsoft SQLNCli team blog (I guess SQLNCli stands for SQL Native Client):

Microsoft is Aligning with ODBC for Native Relational Data Access:

http://blogs.msdn.com/b/sqlnativeclient/archive/2011/08/29/microsoft-is-aligning-with-odbc-for-native-relational-data-access.aspx

From the blog of Hal Berenson, former Distinguished Engineer and General Manager, and who, in his own words, "was both a godfather of the OLE DB strategy and responsible for the SQL Server implementations that have now been deprecated":

OLE DB and SQL Server: History, End-Game, and some Microsoft "dirt":

http://hal2020.com/2011/09/25/ole-db-and-sql-server-history-end-game-and-some-microsoft-dirt/

Interesting stuff. I had worked some years ago on a middleware product that involved ODBC - that sat on top of ODBC, actually (*). One of its main goals was to improve the performance of ODBC-based client-server applications. (Another goal was a more programmer-friendly API for application programmers working on client-server projects that used ODBC, in Visual Basic as well as C.) The product was a success, and was deployed in several large client-server projects of the company I worked for at the time.

Also, the Java JDBC API and the Perl and Python DBI API's were probably influenced quite a bit by the architecture / design of ODBC. (This is what I think, based on having studied and worked on both ODBC and JDBC a good amount, and some on the Perl and Python DB APIs). It (ODBC) was a pretty good technology for its time, and was very extensively deployed and used (almost universally, in fact, for client-server database applications), during the heyday of the client-server period of computing - though native database drivers were also used a lot then.

Interesting to see that Microsoft is now moving back to it - presumably, to improved versions of it, suited to today's requirements.

(*) If you are wondering how another software layer on top of ODBC could improve performance of ODBC apps, rather than only make it worse (due to more overhead of the extra layer), think a bit more. It may sound counter-intuitive, but is possible - it actually happened.

Posted via email

- Vasudev Ram @ Dancing Bison