Accessing c objects in python

Re: Accessing c objects in python

Gary wrote:
[color=blue]
> Hi-
>
> I've been searching the web for a while and I've been unable to
> find a
> way to access c data objects in python without using SWIG. I can
> do
> methods just fine but I can't access variables. It seems like
> PyModule_AddObj ect should work but it segfaults my program.
> There's a good chance I'm just using it wrong, or I should be
> using something
> else. Could someone give me a simple, complete example pretty
> please?
> Thanks![/color]


One way to solve this problem is to implement a new type for
Python that "wraps" your C object. Then your Python code can
create and manipulate instances of this new Python type. These
Python objects in turn manipulate your C object/type.

If that sounds like what you want to do, then read:



In particular, look at:



It contains an example that you can start with. Beginning by
replacing "Noddy" with whatever you want to call your new data type.

Then look at the struct at the top of that example. Put (a
pointer to) you C object in that struct.

Now modify the functions Noddy_new, Noddy_dealloc, and Noddy_init
(which you have re-named, whatever).

Next, add functions to get and set things in your C type.

.... and keep on following the examples.

Is there an easier way? I'm not sure, but I'd certainly read the
following section of the Pyrex documentation. It's about how to
wrap C structures:



And, the part in the following about external extension types in
Pyrex seems particularly appropriate:



Any Pyrex experts listening in? Am I on the right track here?

Dave

--
Dave Kuhlman

[email protected] om

Re: Accessing c objects in python

Alex Martelli <[email protected] > writes:
[color=blue]
> Dave Kuhlman wrote:
> ...
> [snipped: C API usage directions]
>[color=green]
>> Is there an easier way? I'm not sure, but I'd certainly read the
>> following section of the Pyrex documentation. It's about how to
>> wrap C structures:
>>
>> http://ldots.org/pyrex-guide/3-structures.html[/color]
> ...[color=green]
>> Any Pyrex experts listening in? Am I on the right track here?[/color]
>
> I'm no pyrex expert, but I recently re-evaluated various options
> for wrapping large existing C libraries: C API, Boost Python,
> Pyste, pyrex, ctypes, and Swig. Somewhat to my surprise (I had
> bad memories about it from the past...), Swig came up on top in
> terms of simplicity. Pyste might easily be best for _C++_
> libraries, already organized into classes and possibly using
> templates &c -- it, in turn, "rests" om Boost Python. Boost
> itself -- or pyrex if you'd rather not use C++ for the purpose --
> might likely be best if you want to code some significant amount
> of fast logic, not just wrappers. But for "just wrappers", and
> C libraries as opposed to C++ ones, Swig rules.
>
> I'd use the C API directly only if the other tools were ruled
> out for some reason (e.g., licensing); it's by far the largest
> amount of work -- all other tools rest on top of it and can be
> seen as "code generators" for it that save you aggravation.[/color]

Would you care to write a more detailed summary of your findings? This
would probably make a good page in the main Python wiki...

Thomas

Re: Accessing c objects in python

Thomas Heller <theller@python .net> wrote in news:smocwafb.f [email protected]:
[color=blue]
> Would you care to write a more detailed summary of your findings? This
> would probably make a good page in the main Python wiki...
>[/color]

I did an evaluation of the various ways to link to Python for the ACCU
conference this year, although I must say my conclusion was slightly
different than Alex's: I would use Pyrex in preference to SWIG even if I
didn't want to embed much code in the interface.

You can find a paper describing my views at:



The slides and source code for the talks are also on the same site. See


--
Duncan Booth [email protected] k
int month(char *p){return(1248 64/((p[0]+p[1]-p[2]&0x1f)+1)%12 )["\5\x8\3"
"\6\7\xb\1\x9\x a\2\0\4"];} // Who said my code was obscure?

Re: Accessing c objects in python

Duncan Booth <duncan@NOSPAMr cp.co.uk> writes:
[color=blue]
> Thomas Heller <theller@python .net> wrote in news:smocwafb.f [email protected]:
>[color=green]
>> Would you care to write a more detailed summary of your findings? This
>> would probably make a good page in the main Python wiki...
>>[/color]
>
> I did an evaluation of the various ways to link to Python for the ACCU
> conference this year, although I must say my conclusion was slightly
> different than Alex's: I would use Pyrex in preference to SWIG even if I
> didn't want to embed much code in the interface.
>
> You can find a paper describing my views at:
>
> http://dales.rmplc.co.uk/Duncan/accu...ingpython.html
>
> The slides and source code for the talks are also on the same site. See
> http://dales.rmplc.co.uk/Duncan/accu
>[/color]

Interesting. I added these links to the Python wiki.

Thomas

Re: Accessing c objects in python

Duncan Booth wrote:
...[color=blue]
> Thomas Heller <theller@python .net> wrote in news:smocwafb.f [email protected]:
>[color=green]
>> Would you care to write a more detailed summary of your findings? This
>> would probably make a good page in the main Python wiki...
>>[/color]
>
> I did an evaluation of the various ways to link to Python for the ACCU
> conference this year, although I must say my conclusion was slightly
> different than Alex's: I would use Pyrex in preference to SWIG even if I
> didn't want to embed much code in the interface.[/color]

Interesting. I do agree with your observation that, with SWIG,
responsibility for allocating and freeing tends to show up at the
Python level. However, SWIG generates such a Python level itself,
as it wraps 'foo' into a low-level _foo.so and a higher-level foo.py.

Thus, the _USER_ of the overall wrapper need not be concerned with
memory allocation and freeing -- unless he WANTS to be, for performance
reasons, in which case, with SWIG, he gets that *option*. That, in
summary, is the epiphany I had on this latest round of encounter with
SWIG, which made me chose it to complete the wrapping in preference
to Boost and pyrex: the python user of the wrapper could use it in
the simplest and safest of ways _OR_, when performance mattered a
lot (as it might in some cases for these wrappings, which dealt with
multimedia-related libraries), the user could also CHOOSE to take
more direct control -- e.g. explicitly allocate a buffer at the start
of a loop, reuse the buffer N times in the loop, and free it after
the loop, rather than rely on automatic allocation and freeing and
thus pay the resulting performance cost -- in that spot of two in a
huge program where such a strategy pays off.

"Trust the programmer" and "don't stop the programmer from doing
what needs to be done" are two of the "spirit of C" principles (from
the Rationale of the C standard) which Python respects (more so, in
fact, than the other principle from the same source that is so often
quoted, "provide only one way to perform a task"). With SWIG I get
such an arrangement "almost by default" -- and I think that's quite
different from saying that the python user "has" to deal with the
issues of memory management... it is, rather, a _choice_ he or she
gets, just about "for free" from the POV of the guy who is authoring
(and those who in the future will be maintaining) the wrapper.

I could, of course, get the same choice for the user by coding in
Boost -- but then, since I'm wrapping a C library [and not a C++
one with classes already provided and decently well designed], I'd
have a lot more boilerplate-coding to do myself. (I could probably
concoct incredibly subtle C++ templates with some macros in support,
to lighten my own boiler-plate coding task -- but I fear that would
leave far too heavy a maintenance burden on future maintainers of
those wrappers).

I suspect I could most likely get the same result in pyrex, too -- but
then, again, I'd have to code a lot of boilerplate, _without_ even the
help of C++ templates, OR even of C-preprocessor macros... that's too
much raw boilerplate for me;-) Quite possibly I haven't grasped "the
pyrex way to metaprogramming " (just as I guess I hadn't grasped the
SWIG one on earlier encounters with it -- Boost never gave me any
trouble in this regard, as I already understood C++ templates & the
like in sufficient depth before I met it).

I'd better underline yet again that this applies to _wrapping C libraries_!
I do NOT claim I have any real grasp yet about the SWIG way wrt C++
(templates in particular) -- when I next need to wrap a C++ library, I'll
reach for Boost and Pyste, without doubts; and if I ever need to provide
Python users with very speedy code, psyco can't supply it (e.g. because I
need to target non-Intel-like CPU's), and Boost is not applicable (e.g.
because I can't require future maintainers to know C++), I'm most
definitely going to give pyrex another chance -- apart from the boilerplate
issues (==lack of [or insufficient grasp of mine for] metaprogramming and
code generation facilities), I found it just delightful.


Alex