memory mapped tar file contents

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Chris Brooks

    memory mapped tar file contents


    Hi,

    I would like to read directly from a tar file into memory so I can
    manipulate a file (quickly) and write its changes out to another file. I
    thought I could do something like:

    #!/usr/bin/env python

    import tarfile
    import mmap

    fil = tarfile.open( "out.tar.gz " , "r:gz" )
    tarinf = fil.next()
    myfils = {}
    while tarinf != None:
    tarinf = fil.next()
    ref = fil.extractfile ( tarinf )
    myfils[ tarinf.name ] = mmap.mmap( ref.fileno() , 0 )

    But the extractfile() function of TarInfo doesn't seem to give me a fileno,
    so I can't pass this to mmap.

    Thoughts on a way to accomplish this?

    Chris
    --
    View this message in context: http://www.nabble.com/memory-mapped-...p20473925.html
    Sent from the Python - python-list mailing list archive at Nabble.com.

  • Aaron Brady

    #2
    Re: memory mapped tar file contents

    On Nov 12, 8:51 pm, Chris Brooks <[email protected] ask.cawrote:
    Hi,
    >
    I would like to read directly from a tar file into memory so I can
    manipulate a file (quickly) and write its changes out to another file.  I
    thought I could do something like:
    >
    #!/usr/bin/env python
    >
    import tarfile
    import mmap
    >
    fil = tarfile.open( "out.tar.gz " , "r:gz" )
    tarinf = fil.next()
    myfils = {}
    while tarinf != None:
        tarinf = fil.next()
        ref = fil.extractfile ( tarinf )
        myfils[ tarinf.name ] = mmap.mmap( ref.fileno() , 0 )
    >
    But the extractfile() function of TarInfo doesn't seem to give me a fileno,
    so I can't pass this to mmap.
    >
    Thoughts on a way to accomplish this?
    It appears you have to read the contents of the file into the mmap.
    You can create an anonymous map of size tarinfoobj.size , then set
    mapobj[:]= fil.extractfile ( tarinf ).read( ) . Untested.

    Comment

    Working...