s3 – bits.of.info

Copying lots of files into S3 (and within S3) using s3-bucket-loader

Posted on November 10, 2014December 22, 2018 by bitsofinfo

Recently a project I've been working on had the following requirements for a file-set containing roughly a million files varying in individual size from one byte to over a gigabyte; and the file-set size in total being sized between 500gb and one terabyte Store this file-set on Amazon S3 Make this file-set accessible to applications … Continue reading Copying lots of files into S3 (and within S3) using s3-bucket-loader →

Testing yas3fs: a distributed S3 FUSE filesystem

Posted on May 4, 2014 by bitsofinfo

I've recently been doing quite a bit of evaluation of a few S3 filesystems, one in particular is yas3fs which so far is quite impressive. I plan on doing a more detailed post about it later, however for now I'd like to share a little tool I wrote to help me in my evaluation of it. You … Continue reading Testing yas3fs: a distributed S3 FUSE filesystem →