How to store data?

Re: How to store data?


"Lars Behrens" <Spam.Buster@we b.de> wrote in message
news:bi8lqg$6qo [email protected]...[color=blue]
> Hi there!
>
> For a web project I need a little expert help. I don't have written much
> code yet, just been fiddling around a bit, testing and planning.
>
> The web site will have a submission page for attendants of a congress.
> In a form the user will submit name, mailadress etc. and I plan to store
> the data in a list of dictionaries:
>
> People = [{'name' : 'Sir Galahad', 'address' : 'Camelot', 'mail' :
> 'Carrierpigeon@ england.com'}]
>
> #new attendant:
> People.append({ 'name' : 'Brian', 'address' : 'Palestine', 'mail' :
> 'fastridingmess [email protected]' })
>
> The 'lines' can be accessed by their index.
>
> Now my question: Where do you suggest to store the data? I know that
> something sql-like would be quite useful but I don't have any knowledge
> of sql. So I thought about putting the data either into a comma
> separated file or pickling it.
>
> I should mention that some other people on different operation systems
> should be able to import the file (Excel, Open Office) so I tend towards
> the csv file.
>
> What would be better?[/color]

Python 2.3 has an excellent CSV module.
[color=blue]
> And another problem: How to lock the file when several users try to
> write to it?
>
> As I'm no very experienced programmer I would appreciate your help,
> thanks in advance[/color]

John Roth[color=blue]
>
> Cheerz Lars
>
>
>
>
>
>
>
>
>
>[/color]

Re: How to store data?


"Lars Behrens" <Spam.Buster@we b.de> wrote in message
news:bi8lqg$6qo [email protected]...[color=blue]
> Hi there!
>
> For a web project I need a little expert help. I don't have written much
> code yet, just been fiddling around a bit, testing and planning.
>
> The web site will have a submission page for attendants of a congress.
> In a form the user will submit name, mailadress etc. and I plan to store
> the data in a list of dictionaries:
>
> People = [{'name' : 'Sir Galahad', 'address' : 'Camelot', 'mail' :
> 'Carrierpigeon@ england.com'}]
>
> #new attendant:
> People.append({ 'name' : 'Brian', 'address' : 'Palestine', 'mail' :
> 'fastridingmess [email protected]' })
>
> The 'lines' can be accessed by their index.
>
> Now my question: Where do you suggest to store the data? I know that
> something sql-like would be quite useful but I don't have any knowledge
> of sql. So I thought about putting the data either into a comma
> separated file or pickling it.
>
> I should mention that some other people on different operation systems
> should be able to import the file (Excel, Open Office) so I tend towards
> the csv file.
>
> What would be better?[/color]

Python 2.3 has an excellent CSV module.
[color=blue]
> And another problem: How to lock the file when several users try to
> write to it?
>
> As I'm no very experienced programmer I would appreciate your help,
> thanks in advance[/color]

John Roth[color=blue]
>
> Cheerz Lars
>
>
>
>
>
>
>
>
>
>[/color]

Re: How to store data?

Lars-

I don't want to see you cut on bleeding edge software, but, I just
released an updated version of the Pointrel Data Repository System

which could in theory handle such a task (subject to bugs etc.), and I
could not resist responding to your post. The Pointrel System can handle
simple locking and simple transactions (although this has only be tested
recently under GNU/Linux and Python2.3). As a caveat, this Pointrel
System database bunny may have long pointy, nasty teeth, etc., and so if
your application is mission critical you should most likely go with a
more proven solution -- like a convetional relational database (and
using SQL to access one is a good skill to learn). And possibly if your
needs are minimal, pickling your dictionaries or using the 'shelve'
module might just be good enough for a simple application.

Mainly just for fun, and also as another example for people who are
interested in playing with the related concepts, rather than to try to
persuade you specifically to try the Pointrel System (since if you are
not an experienced programmer you would have no way to judge what you
were getting into, and so my best advice to you specifically would be to
go with something proven like pickle, shelve, SQL etc.), here is how you
might solve the storage issue you outline using the Pointrel System:

=============== =============== =============== =====
File: "congress.p y"
=============== =============== =============== =====

from pointrel2003081 2 import *

Pointrel_initia lize("archive_c ongress")

# add a first attendant -- uses built in unique ID function
# but could be any unique ID string
# each change will be implicitely a seperate transaction
attendantID = Pointrel_genera teUniqueID()
Pointrel_add("c ongress", attendantID, 'object type', 'user')
Pointrel_add("c ongress", attendantID, 'name', 'Sir Galahad')
Pointrel_add("c ongress", attendantID, 'address', 'Camelot')
Pointrel_add("c ongress", attendantID, 'mail', 'Carrierpigeon@ england.com')

# add a second attendant, this time as an atomic transaction
attendantID = Pointrel_genera teUniqueID()
Pointrel_startT ransaction()
Pointrel_add("c ongress", attendantID, 'object type', 'user')
Pointrel_add("c ongress", attendantID, 'name', 'Brian')
Pointrel_add("c ongress", attendantID, 'address', 'Palestine')
Pointrel_add("c ongress", attendantID, 'mail',
'fastridingmess [email protected]' )
Pointrel_finish Transaction()

# add a third attendant, again as a single transaction
attendantID = Pointrel_genera teUniqueID()
Pointrel_startT ransaction()
Pointrel_add("c ongress", attendantID, 'object type', 'user')
Pointrel_add("c ongress", attendantID, 'name', 'Lars')
Pointrel_add("c ongress", attendantID, 'address', 'Pythonland')
Pointrel_add("c ongress", attendantID, 'mail', '[email protected] om')
Pointrel_finish Transaction()

# update a value, as an implicit transaction
attendantID = Pointrel_lastMa tch("congress", WILD, 'name', 'Sir Galahad')
Pointrel_add("c ongress", attendantID, 'address', "Zoot's castle")

# delete an attendant
attendantID = Pointrel_lastMa tch("congress", WILD, 'name', 'Brian')
Pointrel_add("c ongress", attendantID, 'deleted', 'true')

# get a list of all attendants (deleted or not)
attendants = Pointrel_allMat ches("congress" , WILD, 'object type', 'user')
print '\nAll attendant IDs (including deleted ones):', attendants

# filter out deleted attendants
currentAttendan ts = filter(lambda user: Pointrel_lastMa tch('congress',
user, 'deleted', WILD) <> 'true', attendants)
print '\nCurrent attendants:', currentAttendan ts

print "\nAttendan t names:"
for attendant in currentAttendan ts:
print Pointrel_lastMa tch('congress', attendant, 'name', WILD)

print "\nAttendan t csv info"
for attendant in currentAttendan ts:
name = Pointrel_lastMa tch('congress', attendant, 'name', WILD)
address = Pointrel_lastMa tch('congress', attendant, 'address', WILD)
email = Pointrel_lastMa tch('congress', attendant, 'mail', WILD)
print '"%s","%s","%s" ' %(name, address, email)
# note proper csv output would require escaping embedded quotes etc.

=============== =============== =============== =====
Sample output:
=============== =============== =============== =====
creating archive file archive_congres s.pointrel_data base.poi
archive with only header

All attendant IDs (including deleted ones):
['unique://freevolution:23 :43207645167115 827',
'unique://freevolution:25 :37062969616920 138',
'unique://freevolution:26 :11213192987278 475']

Current attendants: ['unique://freevolution:23 :43207645167115 827',
'unique://freevolution:26 :11213192987278 475']

Attendant names:
Sir Galahad
Lars

Attendant csv info
"Sir Galahad","Zoot' s castle","Carrie rpigeon@england .com"
"Lars","Pythonl and","lars@exam ple.com"

=============== =============== =============== =====
Database file created: 'archive_congre ss.pointrel_dat abase.xml'
(There is also a binary file which is maintained for efficiency).
=============== =============== =============== =====

<?xml version="1.0" encoding="utf-8" ?>
<P:archive xmlns:P="http://www.pointrel.or g/Pointrel2003081 2/NS/"
version="200308 12.1" archiveID="uniq ue://freevolution:24 :35433055818285 208">
<P:transactio n>
<P:triad
r="1584L"
s="congress"
a="unique://freevolution:23 :43207645167115 827"
b="object type"
c="user"
/>
</P:transaction>
<P:transactio n>
<P:triad
r="1984L"
s="congress"
a="unique://freevolution:23 :43207645167115 827"
b="name"
c="Sir Galahad"
/>
</P:transaction>
<P:transactio n>
<P:triad
r="2376L"
s="congress"
a="unique://freevolution:23 :43207645167115 827"
b="address"
c="Camelot"
/>
</P:transaction>
<P:transactio n>
<P:triad
r="2792L"
s="congress"
a="unique://freevolution:23 :43207645167115 827"
b="mail"
c="Carrierpigeo [email protected]"
/>
</P:transaction>
<P:transactio n>
<P:triad
r="3104L"
s="congress"
a="unique://freevolution:25 :37062969616920 138"
b="object type"
c="user"
/>
<P:triad
r="3312L"
s="congress"
a="unique://freevolution:25 :37062969616920 138"
b="name"
c="Brian"
/>
<P:triad
r="3528L"
s="congress"
a="unique://freevolution:25 :37062969616920 138"
b="address"
c="Palestine"
/>
<P:triad
r="3760L"
s="congress"
a="unique://freevolution:25 :37062969616920 138"
b="mail"
c="fastridingme [email protected] m"
/>
</P:transaction>
<P:transactio n>
<P:triad
r="4072L"
s="congress"
a="unique://freevolution:26 :11213192987278 475"
b="object type"
c="user"
/>
<P:triad
r="4280L"
s="congress"
a="unique://freevolution:26 :11213192987278 475"
b="name"
c="Lars"
/>
<P:triad
r="4496L"
s="congress"
a="unique://freevolution:26 :11213192987278 475"
b="address"
c="Pythonland "
/>
<P:triad
r="4712L"
s="congress"
a="unique://freevolution:26 :11213192987278 475"
b="mail"
c="lars@example .com"
/>
</P:transaction>
<P:transactio n>
<P:triad
r="4992L"
s="congress"
a="unique://freevolution:23 :43207645167115 827"
b="address"
c="Zoot's castle"
/>
</P:transaction>
<P:transactio n>
<P:triad
r="5384L"
s="congress"
a="unique://freevolution:25 :37062969616920 138"
b="deleted"
c="true"
/>
</P:transaction>
</P:archive>

[And, as an example of bleeding edge, from testing this, I found a bug
from a last minute change I made to the unique ID system which had been
working well for a long time; it turns out that because of an error in
how I open a support file when it does not exists, the file
"a_pointrel_uni queIDCounter.tx t" now needs to exist in the local
directory with a number (e.g. '10') in it to get sequence numbers in
addition to a random ID component in unique IDs. The system still works
without this file though -- the unique IDs just aren't as collision
resistant. Something for an incremental release ASAP though.]

The one plug I'd put for the Pointrel System is that the simple (yet, I
hope, elegant) data storage approach is designed to be expandable with
changing needs over time, and usually data storage needs do change, and
in unexpected ways. Whether the Pointrel System can really deliver on
that promise remains to be seen. Again, MySql etc. have a proven track
record handling the sorts of issues you have outlined so far, and are
probably your best choice.

Anyway, if all this sounds like gibberish, feel free to ignore it! :-)

Best of luck with your project.

--Paul Fernhout

Lars Behrens wrote:[color=blue]
> Hi there!
>
> For a web project I need a little expert help. I don't have written much
> code yet, just been fiddling around a bit, testing and planning.
>
> The web site will have a submission page for attendants of a congress.
> In a form the user will submit name, mailadress etc. and I plan to store
> the data in a list of dictionaries:
>
> People = [{'name' : 'Sir Galahad', 'address' : 'Camelot', 'mail' :
> 'Carrierpigeon@ england.com'}]
>
> #new attendant:
> People.append({ 'name' : 'Brian', 'address' : 'Palestine', 'mail' :
> 'fastridingmess [email protected]' })
>
> The 'lines' can be accessed by their index.
>
> Now my question: Where do you suggest to store the data? I know that
> something sql-like would be quite useful but I don't have any knowledge
> of sql. So I thought about putting the data either into a comma
> separated file or pickling it.
>
> I should mention that some other people on different operation systems
> should be able to import the file (Excel, Open Office) so I tend towards
> the csv file.
>
> What would be better?
>
> And another problem: How to lock the file when several users try to
> write to it?
>
> As I'm no very experienced programmer I would appreciate your help,
> thanks in advance
>
> Cheerz Lars[/color]



-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 100,000 Newsgroups - 19 Different Servers! =-----

Re: How to store data?

John Roth wrote:[color=blue]
> Python 2.3 has an excellent CSV module.[/color]

Thx, I'll talk to my admin if upgrading would be possible.

Cheerz Lars

Re: How to store data?


Thx for your help and hints and examples. But as you assume, my
application *is* mission critical. And as I read Pointres is only tested
under GNU/Linux but the server I have to use runs under Solaris, so I
have to find another solution.

Anyways, what I've read so far about Pointres sounds quite interesting
to me (especially because I'm a real database hater). So I will
certainly take a look at Pointres sometime...

Cheerz Lars