[physfs] PhysicsFS 2.1 roadmap...

Wed Apr 8 01:59:01 EDT 2009

Den Wed, 8 Apr 2009 01:31:49 -0400
skrev Indy Sams <indy at driftsolutions.com>:
> Hello Gerry,
> 
>       I don't know about replace, but there are wrappers included that
>       allow you to use PhysicsFS handles as RWOps.

Ah yes, I forgot about those.  Also there's that little detail that
PhysFS is a *filesystem* abstraction, not just stream abstraction --
probably not something you want to force on people, so you'd have to
separate the two concepts (kinda like MIO, heh).

That is, essentially the stream part would be separate from PhysFS, so
that PhysFS could open streams from its abstraced filesystem (of
streams? =)) and pass them to other stuff using the stream part, but not
aware of the filesystem abstraction..  So then, in this magic fairytale
land of ours, SDL and other things could then also use the stream part
without depending on PhysFS.  Streams would roam free between the..
okay, I'll stop now =)

- Gerry

> 
> Wednesday, April 8, 2009, 1:01:50 AM, you wrote:
> 
> GJ> Related to this (somewhat), and just for ideas..  I made a little
> GJ> streaming library some time ago (because I also wanted to be able
> GJ> to access in-memory buffers as streams, which current PhysFS
> GJ> doesn't provide), tentatively named MIO, which simply treats
> GJ> everything as streams. Streams could be plugged and chained
> GJ> together, so you could read from an archive or disk, deflate,
> GJ> patch and calculate md5sum, all with just one call to the read
> GJ> function (or even write it as well, using copy), with everything
> GJ> done on the fly, only processing what's needed.  For a little
> GJ> more efficient reading you could also plug in buffers, etc.  I
> GJ> even toyed with using it as a primitive sound mixer =)
> 
> GJ> There was essentially two types of streams: Raw streams from some
> GJ> specific source (disk, ram, etc), and transformation streams that
> GJ> took other streams of any type as input and processed them in
> GJ> some way (deflate, calculate things, etc).  Additionally, streams
> GJ> had "properties", or specialized actions -- eg, a stream with the
> GJ> 'Directory' property could be used to open new raw streams from
> GJ> inside that stream with a call to the control function (more on
> GJ> that below).
> 
> GJ> The API for this was pretty simple. A stream-specific open
> GJ> function would set up a stream structure with a minimum of three
> GJ> function pointers, one each for read, write and control. Control
> GJ> was the catch-all used for seeking, determining size, closing,
> GJ> opening from directory, etc, usually implemented as a switch
> GJ> block with the supported actions (perhaps this was a bit dense,
> GJ> but it made supporting only those actions you wanted very easy).
> GJ> There was also a system for querying support for actions (so you
> GJ> didn't have to actually try calling them to see if the were
> GJ> supported).  Streams also had an id field and room for
> GJ> stream-specific data and the parent stream if any. The MIO
> GJ> library was essentially a collection of some standard stream
> GJ> handlers like this, tied together by a header with some
> GJ> prototypes and inline functions to call the read, write and
> GJ> control function pointers in a more user-friendly way.
> 
> GJ> Anyway, the project I made this *for* never got anywhere, but MIO
> GJ> itself worked great.  Lightweight, extensible (you could perhaps
> GJ> argue that the implementation was just some standard
> GJ> extensions..), and did everything I wanted it to do.  So hey,
> GJ> it'd be nice to be able to do something like this with PhysFS
> GJ> 3.0 .. =)  (Why I'm not just using MIO myself?  .. well, I could
> GJ> I guess, but I see no reason to try to compete with PhysFS if
> GJ> it's heading in that direction anyway.)
> 
> GJ> Oh also, how likely would you say it'd be for PhysFS to replace
> GJ> RWops in SDL 1.3 and friends?  Just basic streams, with support
> GJ> for pulling in the rest of PhysFS if you wanted to handle
> GJ> compressed data and such.. I'd guess extremely unlikely, but
> GJ> thought I'd at least air the idea anyway =).  It'd be convenient
> GJ> to just have one stream wrapper to worry about..
> 
> GJ> Blue sky,
> GJ> - Gerry
> 
> 
> GJ> Den Tue, 7 Apr 2009 16:59:15 +1000
> GJ> skrev Anthony Cassidy <anthonytcassidy at hotmail.com>:
> >> 
> >> I don't really fit under the concept "peanut gallery", I am more a
> >> "friendly lurker", I have been reading the mailing list for long
> >> enough that I don't remember the original reason I joined.
> >> 
> >> But I have some thoughts about one of your ideas "New archive
> >> types: .tar(.bz2|.gz),...", bzip2 and .gzip only compress a single
> >> file (stream), which would need to be uncompressed then processed
> >> as a tar file. libzip2 provides library functions to do that part,
> >> where the gzip documentation says use zib. Where all this is going
> >> is that both will need to be uncompressed then processed. Where
> >> for rar and the like, the uncompression is part of the processing.
> >> If you support tar.bz2 and tar.gz, then it would be good to be
> >> able to provide a custom uncompression routine, which could be
> >> highjacked to do things like unecryption instead.
> >> 
> >> tldr; If you are going to support tar.bz2 and tar.gz files, its
> >> really processed (uncompressed) and the tar file is then read. It
> >> would be nice to be able to provide custom processing routiens that
> >> would allow for file encryption or downloading the file from an
> >> external source.
> >> 
> >> Then this process could be applied to any archive type. Not sure
> >> how useful this would be, but its just a thought.
> >> 
> >> -Anthony "Timmy" Cassidy
> >> 
> >> _________________________________________________________________
> >> Looking for a fresh way to share photos? Get the new Windows Live
> >> Messenger. http://download.live.com/
> GJ> _______________________________________________
> GJ> physfs mailing list
> GJ> physfs at icculus.org
> GJ> http://icculus.org/mailman/listinfo/physfs
> 
> 
> Best regards,
>  Indy Sams
>  mailto:indy at driftsolutions.com
> 
>  P.S. This is what part of the alphabet would look like if Q & R were
> eliminated.
> 
> _______________________________________________
> physfs mailing list
> physfs at icculus.org
> http://icculus.org/mailman/listinfo/physfs