data file location

[Joseph Carter]

I've been considering a problem Twilight has.  No matter what we do,
people in Windows seem to have problems with Twilight if they create a
shortcut without setting its working directory properly.  This causes
Twilight to be run from the wrong directory and not know where its data
files are.  We have suggested people put a config file in C:\WINDOWS for
this, but it's really not the way things are done in Win32.

We also have the problem in Linux that the autoconf setup we have litters
the filesystem with Twilight's files, as mandated by the FHS and other,
older, UNIX conventions.


This leads to a behaviour in Win32 which is contrary to what Win32 users
expect (manually editing config files and things since we do not use the
native system functions which prevent the need), and which also confuses
Linux users who are predominantly newbies who aren't very comfortable with
compiling, let alone configuration.  Note that other games for Linux (most
of which come with some distribution) do not have this problem because
they are Win32 ports which use some hack to locate their data files, or
are made for Linux (or at least POSIX) and were probably a compiled RPM or
Debian package.

The fact that ./configure tells you where to install data files does not
seem to help these people.


The only logical solution I can think of is to basically do it right for
Win32 (read the location of the executable, chop off the executable name,
and there you have your working directory), and to implement POSIX-reliant
hacks to make sure this works on sane OSes.  The Win32 implementation is
simple and clean, so I don't need to go into details.

The POSIX implementation depends on realpath, which has issues:

 - if argv[0] contains no path elements at all, realpath can't resolve it,
   which happens if the binary is in your path and run by name only
 - realpath assumes that you can pass it a string of PATH_MAX.  The docs
   declare this an inherent flaw, but consider that HURD is the only
   system I know of with a literally unbounded PATH_MAX.  Still, we depend
   on an upper limit to the length of a path elsewhere, and while it would
   be nice if we did not, there's nothing we can do about it except check
   for buffer overrun.
 - note that the first point about realpath being unable to resolve
   something in your path is untrue - it is quite able to resolve the path
   to a binary what was in your path is a symlink, rather than the binary
   itself.  This raises the question of whether or not some idiot isn't
   likely to add the package dir to their path and complain when it
   doesn't work...


As I said elsewhere, we already depend on a known limit for path values,
and on Linux machines we know that there are two different limits.  I
think we use the longer, but don't remember.  Either way, if we don't know
the limit and pathconf (a POSIX function) can't tell us, then the best we
can do is treat it as 4k or so and check for overruns.

The best thing then seems to be to attempt the realpath check.  If it
fails, look up the name of argv[0] in the path.  If THAT fails, well, I
don't know how you ran the thing since on UNIX systems one or the other
should have worked.  An environment variable TWILIGHT_DIR can be looked up
to override the check on any system, I suppose, and you may need to use it
if your system does not provide realpath.  But at that point you're
getting into the question of which UNIXes do we care to support, exactly?
Currently we support Linux, FreeBSD, and kinda MacOS X.  Solaris is known
to work for servers.  IRIX might work, but nobody's bothered to try it.
Obviously any client is expected to have working accellerated OpenGL.  Any
server is expected to require manual configuration anyway.


This leads to the question of how exactly we'd set up the build system and
the organisation of the package directory once installed.  For the former,
I suggest:

 - Our primary target in the POSIX world is Linux.  We should therefore
   use the prefix which is most correct on a modern system, /opt.  BSD
   users will certainly not like this, so we'll have to make it either do
   the right thing for them or make it easy for a BSD port to do the right
   thing for you.  MacOS X is in a similar boat.
 - make install will normally be run by a build script which specifies its
   own prefix anyway.  It's expected that the result will be unpacked
   using an installer of some sort.
 - Whatever installer we use in Linux will need to be static linked, and
   work with or without X.  Probably SDL-based on the X-side.  I'd suggest
   Loki-Setup, except that it's a mess, IMO.  If someone else wants to
   make it work using Loki-Setup, go ahead.  If you beat me to a stripped
   down SDL-based installer, good for you!  (That won't be hard if I wind
   up writing the installer by myself, given that I have unbelievable time
   constraints these days..)
 - Hard-assed Linux dists (*cough*Debian*cough*) will insist that non-FHS
   directory structures are evil and want to install things in a confusing
   manner.  Not my problem, we'll give them an autoconf parameter to shoot
   themselves in the foot with.  ;)

As to the actual directory layout, I would suggest something like:

/opt/twilight/
/opt/twilight/twilight-nq
/opt/twilight/twilight-qw
/opt/twilight/twilight-qwsv
/opt/twilight/global.cfg
/opt/twilight/data/
/opt/twilight/data/id1/
/opt/twilight/data/id1/pak0.pak
/opt/twilight/data/id1/pak1.pak
/opt/twilight/data/qw/
/opt/twilight/data/qw/qwprogs.dat
/opt/twilight/profile/
/opt/twilight/profile/default/
/opt/twilight/profile/default/id1/
/opt/twilight/profile/default/id1/config.cfg
/opt/twilight/profile/default/qw/
/opt/twilight/profile/default/qw/config.cfg

Symlinks from /opt/bin (again, by default) will be created, allowing the
realpath thing to work without the path search..

The reason for this layout is two-fold.  First, data being a subdir under
the package dir, rather than following Quake tradition, allows it to be a
symlink to wherever you really keep your Quake data on any OS that
supports it.  The same layout exactly will be used in Win32, except that
the installer should create start menu entries rather than symlinks, and
you won't be able to use symlinks, naturally.  The other reason is that it
for the first time gives us a seperation between configs and gamedata.

This is not a trivial change since many parts of Quake assume that there
is a very simple path order.  The current ~/.twilight directory is merely
a hack into that same system.  For the first time, Twilight will need to
know where it has write permissions (something I don't know how to do in
Linux) as well as whether or not a directory exists.  Obviously, if a
directory does not exist and you can't write to its parent, you won't be
adding it to the path.  And obviously, the configuration path would need
to contain the data path for backward compatibility (though new configs
would be written to the current profile in the first writable path
location..)  For normal Quake 1, Twilight would have the equivalent of:

data_path = ["/home/user/.twilight/data/id1", "/opt/twilight/data/id1"]
conf_path = ["/home/user/.twilight/profile/user",
	"/opt/twilight/profile/user"]

Unless set explicitly, provile would be default.  This would introduce a
new Cvar flag, CVAR_GLOBAL, since such a Cvar would need to be remembered
in a central (even if per-user) config file (replacing .twilightrc which
is currently used..)  The Cvar would likely be profile_name, and would be
a CVAR_ROM, changed with the profile set <name> or profile load <name>
commands.  Writing the current profile would happen on profile save, which
the engine would do in menus when configuring individual users' profiles,
or automatically whenever the engine writes configs to disk today.

Obviously, instead of creating path directories and failing quietly if it
can't, on write it will simply:

  def write_dir (path):
    for directory in path:
      if exists(directory) or recursive_create(directory):
        return directory
    return None

Obviously downloads could only happen in data directories and configs
could only be stored in profile directories, though the config reader will
check data_path if no config was found in conf_path for reading, just in
case.

There you have it, my goal in a nutshell.

-- 
Joseph Carter <knghtbrd at efn.org>              Now I'll take over the world
 
<FrikaC> I should probably reboot...
<FrikaC> ok brb
<FrikaC> So, what apart form avoiding virii, memory leaks, and rampant
         crashing does Linux reallhy offer :)
<LordHavoc> reliable multitasking?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 253 bytes
Desc: not available
URL: <http://icculus.org/pipermail/twilight-devel/attachments/20030207/df74b249/attachment.pgp>