[physfs] PhysicsFS 2.1 roadmap...

Gerry JJ trick at icculus.org
Wed Apr 8 01:01:50 EDT 2009

Related to this (somewhat), and just for ideas..  I made a little
streaming library some time ago (because I also wanted to be able to
access in-memory buffers as streams, which current PhysFS doesn't
provide), tentatively named MIO, which simply treats everything as
streams. Streams could be plugged and chained together, so you could
read from an archive or disk, deflate, patch and calculate md5sum, all
with just one call to the read function (or even write it as well,
using copy), with everything done on the fly, only processing what's
needed.  For a little more efficient reading you could also plug in
buffers, etc.  I even toyed with using it as a primitive sound mixer =)

There was essentially two types of streams: Raw streams from some
specific source (disk, ram, etc), and transformation streams that took
other streams of any type as input and processed them in some way
(deflate, calculate things, etc).  Additionally, streams had
"properties", or specialized actions -- eg, a stream with the
'Directory' property could be used to open new raw streams from inside
that stream with a call to the control function (more on that below).

The API for this was pretty simple. A stream-specific open function
would set up a stream structure with a minimum of three function
pointers, one each for read, write and control. Control was the
catch-all used for seeking, determining size, closing, opening from
directory, etc, usually implemented as a switch block with the
supported actions (perhaps this was a bit dense, but it made supporting
only those actions you wanted very easy).  There was also a system for
querying support for actions (so you didn't have to actually try
calling them to see if the were supported).  Streams also had an id
field and room for stream-specific data and the parent stream if any.
The MIO library was essentially a collection of some standard stream
handlers like this, tied together by a header with some prototypes
and inline functions to call the read, write and control function
pointers in a more user-friendly way.

Anyway, the project I made this *for* never got anywhere, but MIO
itself worked great.  Lightweight, extensible (you could perhaps argue
that the implementation was just some standard extensions..), and did
everything I wanted it to do.  So hey, it'd be nice to be able to do
something like this with PhysFS 3.0 .. =)  (Why I'm not just using MIO
myself?  .. well, I could I guess, but I see no reason to try to
compete with PhysFS if it's heading in that direction anyway.)

Oh also, how likely would you say it'd be for PhysFS to replace RWops
in SDL 1.3 and friends?  Just basic streams, with support for pulling in
the rest of PhysFS if you wanted to handle compressed data and such..
I'd guess extremely unlikely, but thought I'd at least air the idea
anyway =).  It'd be convenient to just have one stream wrapper to worry

Blue sky,
- Gerry

Den Tue, 7 Apr 2009 16:59:15 +1000
skrev Anthony Cassidy <anthonytcassidy at hotmail.com>:
> I don't really fit under the concept "peanut gallery", I am more a
> "friendly lurker", I have been reading the mailing list for long
> enough that I don't remember the original reason I joined.
> But I have some thoughts about one of your ideas "New archive
> types: .tar(.bz2|.gz),...", bzip2 and .gzip only compress a single
> file (stream), which would need to be uncompressed then processed as
> a tar file. libzip2 provides library functions to do that part, where
> the gzip documentation says use zib. Where all this is going is that
> both will need to be uncompressed then processed. Where for rar and
> the like, the uncompression is part of the processing. If you support
> tar.bz2 and tar.gz, then it would be good to be able to provide a
> custom uncompression routine, which could be highjacked to do things
> like unecryption instead.
> tldr; If you are going to support tar.bz2 and tar.gz files, its
> really processed (uncompressed) and the tar file is then read. It
> would be nice to be able to provide custom processing routiens that
> would allow for file encryption or downloading the file from an
> external source.
> Then this process could be applied to any archive type. Not sure how
> useful this would be, but its just a thought.
> -Anthony "Timmy" Cassidy
> _________________________________________________________________
> Looking for a fresh way to share photos? Get the new Windows Live
> Messenger. http://download.live.com/

More information about the physfs mailing list