[Gtkradiant] Draft for implementing multiple game support V1.0
Timothee Besset
gtkradiant@zerowing.idsoftware.com
Thu, 4 Oct 2001 15:45:29 +0200
I have read through the document. It is good material for new developers,
we could use it (or a lighter version of it) as a roadmap. But first some
introductory remarks:
- Any change to GtkRadiant must be motivated at it's basis by a
functionality change or improvement. Changing something just because it
will be cleaner is not a good enough reason. Each task should begin by a
clear listing of it's objectives and constraints. Then description of a
solution and a conclusion showing that our initial objectives and
constraints are met. (NOTE: "design by contract")
- Radiant evolves step by step. Changes must be self contained and
isolated, so that the editor functionality is maintained through the
developement. Of course it is not always possible, and we can expect a
great deal of b0rkage when isolating the module code into a .lib for
instance.
This is for two reasons: allow several developers to work at once, and
allow ongoing testing while the changes are made.
Now some remarks about the changes themselves:
- VFS:
I think there are some details about the "major" / "minor" stuff in module
APIs that are not very clear. The "major" name (i.e. "VFS" or "shader")
refers to an API, that is function tables, structs and pure virtual
classes. Those are used to abstract a given functionality.
Then we have a "minor" which gives a "type" to the above functionality.
The best example is probably image module, where the same API is used to
load various file formats.
Writing several "minor" for a given "major" (or API) doesn't mean we can
use all of them at the same time in the editor. Right now we can have
several "minor" for the image API. But we use a single "minor" for the
VFS. Dealing with a single minor for some APIs is an important design
choice.
In the case of VFS, we are using a single API and we should continue to do
so: it's no only a matter of ready various filesystems (the OS, .pk3's,
.pak's and .wad's), it's a matter of having a search order configured. We
probably need the "VFSSTDIO" functionality in several VFS implementations,
but it's way easier to use it as a .lib inside the vfs modules.
- Build module:
The main interest I see in the build module is the ability to close the
editor (or if it crashes..) and not loose the compilation. That's what
Q3Build does, and it should be extended to add the watching capabilities +
portability. G_Dewan had started some work towards this some time ago.
Moving q3map in a module that's dependent on the editor (i.e. same
process) is a BAD thing. Robert Duffy tried to do that for Q2, and all
qeradiant users will agree that it was bad.
- Game configuration files:
The basic idea is to move the g_RequiredModules table to a file. This
mostly relies on me writing a seperate .lib for module code, which I
should get started with fairly soon. Meanwhile it's not a problem to have
a bunch of #define and to start working on multiple games support that
way.
- Items stored in game configuration file:
Each module can store his configuration chunk in those files. The code
doesn't have to know about them, it just needs to know to which module to
send the nodes.
Ok I think that's about all I've written down so far.
TTimo
On Thu, 27 Sep 2001 14:24:45 +0100
"Hydra" <hydra@hydras-world.com> wrote:
> Version 1.0 - 2000-09-27
>
> This document currently a draft for the basis of implementing multiple game
> support into GTKRadiant 1.2.
>
> - Hydra (contact me via the gtkradiant mailing list or IRC)
>
> First, before we get onto game configurations, there's a couple of things I
> think we could do with in Radiant before we get into it, I'm not attaching
> any kind of priorities to these ideas at the moment, obviously there are
> alot of low priority things here, but as long as I show you guys what I'd
> like to do, you'll go ahead and do your stuff and will know what I'm working
> towards.
>
> VFSSTDIO
> ========
>
> VFS:
>
> Currently, radiant uses VFS for loading maps, textures, shader scripts,
> and shaderlist.txt (if it is used for anything other than that, let me know)
>
> VFS lets us read files from the computers native filesystems as well as
> reading files stored in compressed files
>
> VFSSTDIO: Why ?
>
> If I'm using Quake3 and I have a pk3 file with a map (.bsp) in it, and have
> the same file in "maps/". Quake3 will use the files stored in pk3's first,
> before it uses files that are not contained within pk3 files.
> Even with /sv_pure 0
>
> According to SPoG (not verified this) Quake 2 does the opposite, in that
> it will use files outside PAK files before it uses files inside them.
>
> What we want is a way to emulate the game engine's method of loading files
> so that the files used by the editor are the same files that will be used
> by the game engine itself.
>
> We need to extend VFS system (or rather, the plugin system itself) so that
> more than one VFS module can be loaded and extend the functions in
> pluginmanager.cpp like QERApp_LoadFile() so they behave like
> QERApp_LoadImage() (which uses CPlugInManager::LoadImage() to try
> each image module loaded)
>
> When this is done, we can move the code in vfspk3 that deals with loading
> files via the OS into a new filesystem module called vfsstdio (or whatever)
>
> Then, we can let the user change the order that the VFS systems are used so
> that the editor emulates the way the engine works!
>
> Now, in doing that we also open up the possibility of having other VFS
> modules
> loaded at the same time, and the possibilities there are endless. (I'm not
> saying all the possibilites will be useful, just that extra functionality
> will then be possible. you could for example have a VFSCVS module)
>
> And, I'm not sure this would work yet, i need to check the code a bit
> further, if VFS used itself to open files then you could open PK3 files
> from inside PK3 files, or you could open PK3 files on a CVS server.
> OMG! Now *that* would be cool.
>
> Entity Modules
> ==============
>
> Instead of having the code for loading .DEF files in the core, we move it
> out
> into a seperate module and define an interface for such modules.
>
> This gives us the ability to expand the editor so that it can read entity
> information from other games and new games as they appear by just creating
> a new plugin.
>
> Build Modules
> =============
>
> Instead of having the code for kicking off a BSP compile built into the
> core we move it into a seperate module (kinda like the exisiting Q3Build
> module that exists for 1.1). We would also define an interface for these
> special kind of plugins.
>
> Thus, the code for BSP Monitoring would be moved into that.
>
> This also gives us the opportunity to actually make Q3MAP itself into a
> module.
>
> It also gives us greater flexibility than having the project file storing
> the commands used to build the BSP file from the map file.
>
> A universal module could do something like this:
> make Radiant save the map to a file that the compiler tool understands
> run pre-build commands
> run build tools
> run post-build commands
>
> Game configuration files
> ========================
>
> Game configuration files will REPLACE .qe4 project files (and will
> probably have an option to import one too).
>
> The idea with a game config is to keep the list of things that Radiant
> itself
> needs to know down to a minimum. And if you think of a modularised system
> this list quickly becomes quite small.
>
> Game config also stores plugin configs in the same file, so if you are using
> the BUILDALL plugin to compile maps for Q3, then configuration for Q3MAP.EXE
> command lines would be in here (thus making stuff like "rshcmd" in the main
> config irrelvevant). Plugins should still be able to have their own GLOBAL
> config files, preferably stored as something like plugins/<myplugin>.cfg.
>
> The format for the game configuration files will be XML, the file extension
> for them will be .xgc (XML Game Config) (unless anyone knows of a common
> extension that conflicts with that idea)
>
> They will be saved in a configs/ subdirectory of Radiant itself (so you
> don't
> loose them when you add/remove games to and from your system)
>
> Once we've decided on what information goes into a game configuration, I'll
> draft up the XML game configuration config file format itself.
>
> Items stored in game configuration file:
> ----------------------------------------
> note: all paths and filenames in config are absolute, unless specified below
> note: Should support for UNC paths on WIN32 should be allowed for some
> paths?
>
> Profile File Name
> The name of this game configuration profile
>
> I envisage that people will want more than one configuration
> profile for each game that Radiant supports, much like we have
> different project files at the moment.
>
> This gets round the problem of GTKRadiant creating lots
> of .qe4 files without the user knowing which one they're
> currently using.
> Comment
> A comment for the profile
>
> This is for display only, and has NOTHING to do with
> anything else at all.
>
> String can be upto 100 chars or so long, so it can be
> reasonably descriptive.
>
> Possible uses:
> It could used when displaying a list of profiles to the user
> Displayed in the title bar after the map filename.
>
> e.g.
> Quake III Arena
> Quake III: Team Arena
> Quake III Arena test mod 1
>
> Base path of game
> The path the user installed the game to.
>
> Game Executable
> The executable file of the game itself. e.g. quake3.exe or
> wolf.exe
>
> Base path(s) of game data
> Multiple paths for this allows support for a) common shared
> areas and b) cd-rom/network drives.
>
> Searched IN ORDER when looking for files.
>
> (e.g, basepath1, basepath2...)
>
> Paths seperated by ,'s or ;' in the view presented to the
> user, or as a list that they can add to/remove from.
>
> Base path(s) of mod data
> For support of game mods, like missionpack or cstrike
>
> Multiple paths for this allows support for a) common shared
> areas and b) cd-rom/network drives.
>
> Searched IN ORDER when looking for files.
>
> MOD Dirs are searched for files BEFORE game data paths so that
> things like shader scripts / shaderlist.txt / textures are picked
> up from the MOD dir, rather than the game dir, thus allowing
> mod developers/mappers to use updated.
>
> (e.g, modpath1, modpath2, then basepath1, basepath2...)
>
> Paths seperated by ,'s or ;' in the view presented to the
> user, or as a list that they can add to/remove from.
>
> Map Source Path
> The path that GTKRadiant loads and saves map source data to
> and from.
>
> Not necessarily used by the map compiler tools.
>
> Prefab path
> The path that GTKRadiant loads and saves map prefabs to and
> from.
>
> MAP Module(s)
> Plugin(s) that this game/mod uses for loading and saving map files.
>
> Needs to be a list of modules, and the one at the top of the
> list should be the primary module (which would be the default
> file/load, file/save) file/export would let you use the
> other modules.
>
> We can't just load one MAP module, or we won't be able to
> import prefabs and or maps of other kinds or convert between them.
>
> VFS Module(s)
> Module(s) that this game/mod uses for loading files (of any kind)
> from PK3/PAK/etc files.
>
> Needs to be a list of modules, files are located using the VFS
> module list in order (e.g. VFSSTDIO then VFSPK3 then VFSetc...)
>
> Shader Module
> Module that this game/mod uses for loading files (of any kind)
> from PK3/PAK/etc files.
>
> Image Module(s)
> Module(s) that this game/mod uses for loading image files.
>
> Needs to be a list of modules, no specific order as images
> are loaded on demand, there might be a performance increase
> if the order can be prioritized however.
> (e.g. looking for PCX files first when editing Q3 maps would
> not be a great idea)
>
> This list can also be empty, as an image module is NOT required
> to EDIT a map. (It sure won't look pretty though!)
>
> Having no image module loading should have the same effect as
> a brush that the texture cannot be found for.
>
> Entity Module(s)
> Module(s) that this game/mod uses for loading entity defintion
> files.
>
> Needs to be a list of modules, no specific order as entity
> definitions are loaded at startup. Though having said that
> letting the user change the order might be useful, but
> certainly not required.
>
> This list can also be empty, as an entity module is NOT required
> to EDIT a map.
>
> Build Module(s)
> Module(s) that this game/mod uses for to call the map compiler tools
>
> (Because Q1/Q2/Q3/HL can all use different compiler tools
> and some of the tools provide additional features like
> monitoring)
>
> Needs to be a list of plugins, and the one at the top of the
> list should be the primary plugin (which a hotkey might call)
>
> This list can also be empty, as a build module is NOT required
> to EDIT a map.
>
> Additional map compiler plugins may be useful, rather than
> just one. (Don't know what for yet, but... :)
>
> Plugins
> List of additional plugin(s) that this game/mod uses for to
> do other stuff.
>
> This list can also be empty, as additional plugins are NOT
> required to EDIT maps.
>
> The idea here is that you can have a bunch of plugins sitting
> in plugins/ but some are only useful to you if you're editing
> Q3 maps (autocaulker..), and some only useful if you're editing
> HL maps (a .RAD file creator/editor for example). So you don't
> want to load HL plugins if you're editing Q3 and vice-versa,
> and this takes care of that.
>
> Module/Plugins:
> ===============
> version_name's currently tend to be either the extension of the filename
> that the plugin supports, or game engine names but are not limited to this.
>
> VFS (interface_name: "filesystem module")
> ---
> VFSSTDIO loads files using normal OS calls (see extra topic below)
> proposed version_name's:
> "stdio"
>
> VFSPK3 loads files from PK3 files
> handled by vfspk3.dll
> current version_name's:
> "quake3", proposed: "pk3"
>
> VFSPAK loads files from PAK files
> proposed version_name's:
> "pak"
>
> VFSWAD loads files from WAD/WAD3 files
> proposed version_name's:
> "wad"
>
>
> MAP (interface_name: "map module")
> ---
> XMAP loads and saves XMAP files
> proposed version_name's:
> "xmap"
>
> MAP loads and saves Q1,Q2,Q3 and Q3BP files
> proposed version_name's:
> "quake3bp"
> "quake3"
> "quake2"
> "quake1"
> "wolf" ? unless Wolf map format == quake3 map format
> "stvef" ? unless STV:EF map format == quake3 map format
>
> HL loads and saves HalfLife .MAP files
> (probably be incoporated to MAP module, above)
> proposed version_name's:
> "hl"
>
> RMF loads and saves Worldcraft .RMF files
> proposed version_name's:
> "rmf"
>
> BUILD (interface_name: "build module")
> -----
> BUILDQ3 shows a GUI and lets you compile Q3 maps with q3map monitoring
> OR it could be the actual BSP compiler itself. (i.e. instead of
> having Q3MAP.EXE it could be built into BUILDQ3.DLL)
> proposed version_name's:
> "quake3"
> BUILDALL shows a GUI that lets you run various programs in sequence
> (which can be used for Q1,Q2,etc..)
> proposed version_name's:
> "all"
>
> ENTITY (interface_name: "entity module")
> ------
> ENTITYDEF loads entities from Q*.DEF files
> proposed version_name's:
> "def"
> ENTITYFGD loads entities from HalfLife FDG files
> "fgd"
>
> IMAGE (interface_name: "image module")
> -----
> IMAGE loads TGA/JPG images
> handled by image.dll
> current version_name's:
> "tga"
> "jpg"
> IMAGEPCX loads PCX images
> proposed version_name's:
> "pcx"
>
> SHADER (interface_name: "shader module")
> ------
> SHADER handles shaders and shader scrips for q3
> currently handled by shader.dll
> proposed new name of shader.dll: "shaderq3.dll"
> current version_name's:
> NULL, proposed: "quake3"
>
>
> Overview of which which interfaces need to be able to support more that one
> loaded module/plugin, which ones are REQUIRED by radiant and which
> ones are prioritizied
>
> Interface Multiple?, Prioritized?, Required?, Example List
> --------------------------------------------------------------
> VFS YES YES YES VFSSTDIO, VFSPK3
> SHADER NO N/A YES SHADERQ3
> IMAGE YES NO YES IMAGE, IMAGEPCX
> MAP YES YES YES XMAP,MAP
> BUILD YES YES NO BUILDQ3, BUILDALL
> ENTITY YES NO NO? ENTITYDEF, ENTITYFGD
>
> Changes to GUI
> ==============
> Have an amalgamated GUI for editing preferences, with a tree view on the
> left
> and a blank area on the right that is filled by whatever item in the tree
> view is being configured. Much in the same way that the Gnome control
> center does.
>
> The core sets up the actual list of items on the tree view
>
> The idea being that plugins/modules add items to the preferences tree view
> on the left and then display their preferences in the blank area on the
> right
>
> Here's a list of things that could be in the tree view.
>
> + User Input
> | |- Mouse (2/3 button, invert in camera, etc)
> | |- Keyboard Shortcuts (provide gui to change them)
> | |- Toolbars (patch toolbar)
> |
> + Editor
> | |- Layout (views, floating z)
> | |- Rendering (Open GL this that and the other)
> | |- Camera (Slow <-> fast, update XY)
> | |- Texturing (Quality, subset, scrollbar, tex increment
> | | matches grid)
> | |- Options (Alt + multidrag, vertex editing splits face,
> | | display size info)
> | |- Misc (Use Win 32 file dialog, autosave every, load
> | last config/map, etc)
> + Modules
> | <interface name>
> | |-<module name> <module prefs>
> | Shader Systems
> | |- ShaderQ3 (Startup Shaders)
> | File System
> | |-VFSPK3
> | |-VFSSTDIO
> | Image Loaders
> | |-Image
> | Build / Compilers
> | |-BuildQ3 (BSP Monitoring config)
> |
> + Plugins
> <plugin name> about info and plugin prefs
>
> Not looked too much at GTK stuff yet, so don't know how hard
> this would be, but it makes sense for everything to be configured in the
> same
> place.
>
> Module priority must be configurable by clicking on the <interface name>
> in the tree view, and then re-arranging a list of modules with the same
> <interface name> via way of UP and DOWN buttons.
>
> Each plugin and module should be able to enabled and disabled by ticking a
> checkbox either a) next to the plugin name in the treeview (dunno if GTK can
> do that) or b) via a checkbox on the right hand side when that plugin
> is selected. Whether or not this requires a restart of Radiant or not
> would be up the the plugin iteself, but the user should be informed if that
> is the case.
> The plugin's enabled/diabled state would have to be stored in the game
> config.
> And, obviously you wouldn't be allowed to disable all required modules (so
> you
> could say, disable XMAP and MAP by themselves, but not have them BOTH
> disabled)
>
> Not sure wether the game configuration dialog should be included in this
> dialog
> yet, it'd be better if it was, but making changes to the game config would
> possible change the list of plugins that are uses, and then the tree view
> list
> would change. So, thinking about it'd be easier if it was a seperate dialog
> box
> much like project preferences and editor preferences are now.
>
> Operation of Radiant
> ====================
>
> Radiant startup
> ---------------
> Radiant loads up, and as part of it's initialization process it does this:
>
> Radiant scans modules/ dir for modules
>
> Loads each module, and builds a list in memory of what each module does but
> DOES NOT activate the modules YET!
>
> After it has scanned the modules and verified all the .dll files are
> actually all valid modules (correct for this radiant version, etc..) it
> will end up with a list like this:
>
> module filename, **interface_table (GUID, interface_name, version_name)
> ---------------------------------------------------------------------
> shaderq3.dll, (* , "shader module" , "quake3")
> image.dll, (* , "image module" , "tga")
> (* , "image module" , "jpg")
> buildq3.dll (* , "build module" , "quake3")
> map.dll (* , "map module" , "quake3bp")
> (* , "map module" , "quake3")
> (* , "map module" , "quake2")
> (* , "map module" , "quake1")
> xmap.dll (* , "map module" , "xmap")
> entity.dll (* , "entity module" , "def")
> vfspk3.dll (* , "filesystem module" , "pk3")
> vfsstdio.dll (* , "filesystem module" , "stdio")
>
> Radiant scans plugins/ dir for plugins and makes a list in memory, plugins
> are NOT TO BE ACTIVATED YET!
>
> Searches configs/ subdirectory for .xgc files (XML Game Config)
> Checks saved settings (radiant.ini) for last .xgc file used and loads
> it if:
> a) the user told us to "load last config on open"
> and
> b) the file exists
>
> If we've not loaded a game config, then prompt the user to
> a) create one (see below)
> b) load one
> c) quit
>
> Game config now in memory
>
> The game config then determines which modules are to be used and activates
> the ones to be used and unloads the ones that are not to be used.
>
> Radiant activates all plugins that are ENABLED in the users config.
> (meaning users would have to actually activate all plugins manually,
> and for each game configuration file)
> OR
> Radiant activates all plugins that are NOT DISABLED in the users config.
> (meaning that new plugins placed in the plugins/ dir are enabled for
> each game configuration until manually disabled)
>
> Note: I'm not sure which one to go for here, as there implications either
> way. I think, for ease of use I'd go with the second option, but if the
> user was required to enable each new plugin we'd have less problems, and
> also configurations would say the same if you copied the game config file
> to another machine that had more plugins than the machine the config came
> from. So thinking about it like that I think I'd go for the first
> option: "activate plugins that are ENABLED", comments ?)
>
> The intention here is not to load plugins/modules that don't need to be
> loaded. For instance, if you're doing Quake3 mapping then it makes no sense
> to activate the shader and vfs modules for Quake1, and vice versa.
>
> There must also be at least one plugin for each core type of plugin for each
> interface listed in the REQUIRED interfaces table (above), radiant cannot
> start if there is not.
>
> If any plugin requires configuration then the plugin calls a function
> in radiant that shows a preferences dialog and allows the user to configure
> the plugin and any other plugins or preferences.
>
> Check that all plugins are actually configured, if not exit.
>
> If any changes to a plugin's config are made then they are saved into the
> game configuration file now.
>
> If changing a plugin's config requires a restart then exit now.
>
> DO NOT free the list of available plugins and modules so that new game
> configs can be created after Radiant has finished loading up.
>
> Carry on with GTKRadiant initialization.
>
> Creating a game config
> ----------------------
> A dialog will pop up asking the user to create a new game config.
> This dialog should let the user import example game configs as well
> as import .QE* files or other editor config files (as we see fit)
>
> The user will then be presented with a list of available modules from which
> they can chose which ones to use and will be able to set the order those
> modules are loaded via way of lists and up down buttons just like in the
> main
> prefs (see above).
>
> The user will also be also be enable (or disable, see above) plugins
> that are available to them.
>
> The user will then be able to configure the items that are stored in the
> main game configuration (filename, comments, etc..)
>
> Information required for configs
> ================================
>
> Example config for Team Arena (A Quake 3 mod)
> ---------------------------------------------
> Hard Drive = C:, CDRom's on drives D: and E:
>
> ProfileFilename
> GameName Quake III: Team Arena
> BasePath C:\Games\Quake3
> GameExecutable C:\Games\Quake3\Quake3.EXE
> DataBasePath C:\Games\Quake3\BaseQ3,D:\Quake3\BaseQ3
> DataModPath C:\Games\Quake3\MissionPack,E:\Quake3\MissionPack
> MapSourcePath \\MyServer\MapDevelopment\Quake3
> PrefabPath \\MyServer\MapDevelopment\Quake3\Prefabs
> MapModules XMAP,MAP
> VFSModules VFSSTDIO,VFSPK3
> ShaderModule SHADERQ3
> ImageModules IMAGE
> EntityModules ENTITYDEF
> BuildModules BUILDQ3,BUILDALL
> Plugins BOBTOOLZ,GENSURF,PRTVIEW,TEXTOOL
>
> Example Config for CounterStrike (a Half-life mod)
> --------------------------------------------------
> Hard Drive = C:, CDRom's on drives D: and E:
>
> GameName Counter-Strike
> BasePath C:\Games\HalfLife
> GameExecutable C:\Games\HalfLife\HL.EXE
> DataBasePath C:\Games\HalfLife\Valve,D:\HalfLife\Valve
> DataModPath C:\Games\HalfLife\CStrike,D:\HalfLife\CStrike
> MapSourcePath \\MyServer\MapDevelopment\CStrike
> PrefabPath \\MyServer\MapDevelopment\CStrike\Prefabs
> MapModules XMAP,MAP,RMF
> VFSModules VFSSTDIO,VFSWAD
> ShaderModule SHADERHL
> ImageModules IMAGEPCX
> EntityModules ENTITYFGD,ENTITYDEF
> BuildModules BUILDALL
> Plugins BOBTOOLZ,GENSURF
>
> Planned list of support for games:
> ==================================
> Quake3 and it's mods
> STV:EF and it's mods
> WOLF and it's mods
> Quake2 and it's mods
> Quake1 and it's mods
> Halflife and it's mods
>
>
> _______________________________________________
> Gtkradiant mailing list
> Gtkradiant@zerowing.idsoftware.com
> http://zerowing.idsoftware.com/mailman/listinfo/gtkradiant