From hook_l at pyrogon.com Thu Aug 7 22:37:26 2003 From: hook_l at pyrogon.com (Brian Hook) Date: Thu, 7 Aug 2003 22:37:26 -0400 Subject: [q2x] Alive In-Reply-To: <20030601062335.GA24687@pc045106> Message-ID: <200387223726.241053@centrino> Okay, I think we're going to get this going again. The biggest issue I'm dealing with now is structure -- Q2X requires quite a few third party open source libraries, and I'm not sure the best way to integrate them. 1. Force the user to download them and either A.) put them in blessed locations or B.) modify the make/project files to point to them correctly 2. Stick them into the cvs, ensure blessed copies but also duplicate code. This also has some crappy side effects for people trying to work on both q2x and one of the dependent libraries (e.g. I'm also a contributor to physfs, so I'd need to have two separate checkouts, one with CVS/ and another without). Ideas? I'm also looking at a new scripting language, Io (www.iolanguage.com), that has some attributes that are a bit nicer than Lua's. Brian From nick at rockstarvancouver.com Fri Aug 8 00:21:20 2003 From: nick at rockstarvancouver.com (Nick Trout) Date: Thu, 7 Aug 2003 21:21:20 -0700 Subject: [q2x] Alive Message-ID: <911F8C8EB7A8084AAEDD55CEDC54D8F80CB558@iggy.rockstarvancouver.com> > From: Brian Hook [mailto:hook_l at pyrogon.com] > Sent: Thursday, August 07, 2003 7:37 PM > To: q2x at icculus.org > Subject: Re: [q2x] Alive > > Okay, I think we're going to get this going again. The biggest issue > I'm dealing with now is structure -- Q2X requires quite a few third > party open source libraries, and I'm not sure the best way to > integrate them. > > 1. Force the user to download them and either A.) put them in blessed > locations or B.) modify the make/project files to point to them > correctly > > 2. Stick them into the cvs, ensure blessed copies but also duplicate > code. This also has some crappy side effects for people trying to > work on both q2x and one of the dependent libraries (e.g. I'm also a > contributor to physfs, so I'd need to have two separate checkouts, one > with CVS/ and another without). > > Ideas? How about supplying tarballs of the versions of the libraries that you are developing with, which expand into q2x/libs (using a supplied batch file?). The tgzs could go into cvs and the q2x makefiles could all call the lib makefiles. This saves having the PITA situation where the source tree changes and you have to duplicate it in your cvs. Duplicating these libraries means you can have different versions of the libraries for different projects and the source for q2x is all off one directory tree (handy for tar-ing if necessary). Where you are working on one of these libraries, requiring CVS, couldn't you just copy the source tree for the library back and forth (i.e. when you wanted to check it in)? I guess this assumes that the libraries being used are in a fairly stable state as you don't really want to update the tarballs daily. > I'm also looking at a new scripting language, Io (www.iolanguage.com), > that has some attributes that are a bit nicer than Lua's. Mmmm, that does look quite interesting. Would you support Lua as well? I was hoping that q2x would develop into a Lua tutorial type project. What do you see as the main advantages of using Io over Lua? Regards, Nick From hook_l at pyrogon.com Fri Aug 8 00:25:01 2003 From: hook_l at pyrogon.com (Brian Hook) Date: Fri, 8 Aug 2003 00:25:01 -0400 Subject: [q2x] Alive In-Reply-To: <911F8C8EB7A8084AAEDD55CEDC54D8F80CB558@iggy.rockstarvancouver.com> Message-ID: <2003880251.101187@centrino> > How about supplying tarballs of the versions of the libraries that > you are developing with, which expand into q2x/libs (using a > supplied batch file?). The tgzs could go into cvs and the q2x > makefiles could all call the lib makefiles. Good idea. That way I can have the blessed versions, and it would be up to the individual user to either extract those files or point the makefiles/project files to their own local copies. > guess this assumes that the libraries being used are in a fairly > stable state as you don't really want to update the tarballs daily. Right. > Mmmm, that does look quite interesting. Would you support Lua as > well? I'm not sure. I like Lua, in theory, but in practice it always seems to irritate me with something =) > I was hoping that q2x would develop into a Lua tutorial type > project. What do you see as the main advantages of using Io over > Lua? The primary advantages are stated on their Web page, but the major things (for me) are: - an object model that I'm more comfortable with (it directly handles OO messaging instead of emulating it as in Lua) - tricolor incremental GC instead of the hiccup prone GC that Lua has That's really the major stuff. It does have several downsides: - very poor documentation (not that Lua is that much better, but Lua is much more mature) - relatively new, which means it can be a moving target somewhat (same thing with Lua), and it's not fully debugged I'm not making any decisions just yet, I still need to get the primary tree cleaned up massively before I do the first public check-in. I really want the code to be relatively clean and solid before we start monkeying around with it. Brian From theoddone33 at linuxgames.com Fri Aug 8 00:30:50 2003 From: theoddone33 at linuxgames.com (Dan Olson) Date: Thu, 7 Aug 2003 23:30:50 -0500 Subject: [q2x] Alive In-Reply-To: <911F8C8EB7A8084AAEDD55CEDC54D8F80CB558@iggy.rockstarvancouver.com> References: <911F8C8EB7A8084AAEDD55CEDC54D8F80CB558@iggy.rockstarvancouver.com> Message-ID: <20030808043050.GA1235@localhost> On Thu, Aug 07, 2003 at 09:21:20PM -0700, Nick Trout wrote: > > I'm also looking at a new scripting language, Io (www.iolanguage.com), > > that has some attributes that are a bit nicer than Lua's. > > Mmmm, that does look quite interesting. Would you support Lua as well? I > was hoping that q2x would develop into a Lua tutorial type project. What > do you see as the main advantages of using Io over Lua? > Object orientation and exception handling seem advantageous. Lua is a little more mature and stable. I think it will be a while until Io is stable enough to use in any project. Io is also more of a brain shift than Lua to a modder coming at this from a QuakeC or C++ background. I'd like to stick with Lua for now, but keep a close eye on Io as it progresses. Io has a lot of little things (like built-in concurrency with auto deadlock detection) that could turn out to be very helpful once it becomes stable enough to run its own example code. -- Dan Olson From nick at rockstarvancouver.com Fri Aug 8 00:40:04 2003 From: nick at rockstarvancouver.com (Nick Trout) Date: Thu, 7 Aug 2003 21:40:04 -0700 Subject: [q2x] Alive Message-ID: <911F8C8EB7A8084AAEDD55CEDC54D8F80CB559@iggy.rockstarvancouver.com> > > Mmmm, that does look quite interesting. Would you support Lua as > > well? > > I'm not sure. I like Lua, in theory, but in practice it always seems > to irritate me with something =) How so? I realise it has its quirks, just like anything else, but slowly it's evolving. > > I was hoping that q2x would develop into a Lua tutorial type > > project. What do you see as the main advantages of using Io over > > Lua? > > The primary advantages are stated on their Web page, but the major > things (for me) are: > > - an object model that I'm more comfortable with (it directly handles > OO messaging instead of emulating it as in Lua) It could also complicate matters for functional programmers! :) > - tricolor incremental GC instead of the hiccup prone GC that Lua has This does sound interesting. Lua has IGC on the way but it does seem to be slow coming. Still, when the Lua authors do deliver it's usually pretty well thought out and very stable. > That's really the major stuff. It does have several downsides: > > - very poor documentation (not that Lua is that much better, but Lua > is much more mature) > > - relatively new, which means it can be a moving target somewhat (same > thing with Lua), and it's not fully debugged Mmmm this would put me off. I think that has been one of the pluses for Lua for me is that there a so few bug reports once it's released. > I'm not making any decisions just yet, I still need to get the primary > tree cleaned up massively before I do the first public check-in. I > really want the code to be relatively clean and solid before we start > monkeying around with it. Righto. I'm pretty snowed under with work but I'll try and find time to help out if you go with Lua as I like the cut of this projects jib. Nick From hook_l at pyrogon.com Fri Aug 8 10:16:30 2003 From: hook_l at pyrogon.com (Brian Hook) Date: Fri, 8 Aug 2003 10:16:30 -0400 Subject: [q2x] Alive In-Reply-To: <911F8C8EB7A8084AAEDD55CEDC54D8F80CB559@iggy.rockstarvancouver.com> Message-ID: <200388101630.849165@centrino> Fine fine fine, Lua it is. =) If the GC becomes an issue, we'll work around it. I'm still digging on Io, and may give it a whirl for a MUD-like project I'm dorking with. Brian From nick at rockstarvancouver.com Fri Aug 8 14:13:33 2003 From: nick at rockstarvancouver.com (Nick Trout) Date: Fri, 8 Aug 2003 11:13:33 -0700 Subject: [q2x] Alive Message-ID: <911F8C8EB7A8084AAEDD55CEDC54D8F80CB55A@iggy.rockstarvancouver.com> > From: Brian Hook [mailto:hook_l at pyrogon.com] > Sent: Friday, August 08, 2003 7:17 AM > To: q2x at icculus.org > Subject: RE: [q2x] Alive > > Fine fine fine, Lua it is. =) If the GC becomes an issue, we'll work > around it. I'm still digging on Io, and may give it a whirl for a > MUD-like project I'm dorking with. Io does look quite interesting and two of the principles, Steve Dekorte and Edgar Toernig were regular contributors to the Lua list a couple of years ago. From the look of the newsgroup/mailing list it looks like the syntax and feature set are still in flux. Whilst it may be an interesting future scripting language Lua may be more practical for the purpose intended here currently. WRT Lua GC, the trick is to try and avoid creating objects at runtime to avoid forcing a collection. This is not vastly different from trying to avoid dynamic memory allocation in C during frame time. http://lua-users.org/wiki/OptimisingGarbageCollection It would still be wise to do this even if you had an incremental garbage collector as work is done in the allocation, free and clean up. Nick From hook_l at pyrogon.com Fri Aug 8 15:39:17 2003 From: hook_l at pyrogon.com (Brian Hook) Date: Fri, 8 Aug 2003 15:39:17 -0400 Subject: [q2x] Alive In-Reply-To: <911F8C8EB7A8084AAEDD55CEDC54D8F80CB55A@iggy.rockstarvancouver.com> Message-ID: <200388153917.948284@centrino> > Io does look quite interesting and two of the principles, Steve > Dekorte and Edgar Toernig were regular contributors to the Lua list > a couple of years ago. Right. In fact, I learned about it because Steve posted to the Lua list about it. > it looks like the syntax and feature set are still in flux. Whilst Definitely, so I'm going to go with Lua 5.x. How familiar are you with the Q2 code base/way of doing things? > WRT Lua GC, the trick is to try and avoid creating objects at > runtime to avoid forcing a collection. Yeah, this is what some game developers have done, but that somewhat begs the point of automatic memory management -- for me, I use GC when I don't want to _think_ about memory management, period. It's still easier to use Lua-style GC than it is to use C-style malloc/free, but it's still not the holy grail of "just do it". Brian From nick at rockstarvancouver.com Fri Aug 8 15:54:41 2003 From: nick at rockstarvancouver.com (Nick Trout) Date: Fri, 8 Aug 2003 12:54:41 -0700 Subject: [q2x] Alive Message-ID: <911F8C8EB7A8084AAEDD55CEDC54D8F80CB55D@iggy.rockstarvancouver.com> > How familiar are you with the Q2 code base/way of doing things? I've had a look at it but not really played around with it much. I understand you are changing it fairly extensively though so would a good knowledge of Q2 help?! In what sense do you mean "way of doing things"? Do you mean pipeline, structure, object management, coding style, etc., or all of the above? > > WRT Lua GC, the trick is to try and avoid creating objects at > > runtime to avoid forcing a collection. > > Yeah, this is what some game developers have done, but that somewhat > begs the point of automatic memory management -- for me, I use GC when > I don't want to _think_ about memory management, period. It's still > easier to use Lua-style GC than it is to use C-style malloc/free, but > it's still not the holy grail of "just do it". I guess I'm stuck in the "old school" programming rut in this sense where I'm conscience of inefficiency whilst programming :) (and perhaps because Lua doesn't yet have IGC) Nick From hook_l at pyrogon.com Fri Aug 8 16:17:45 2003 From: hook_l at pyrogon.com (Brian Hook) Date: Fri, 8 Aug 2003 16:17:45 -0400 Subject: [q2x] Alive In-Reply-To: <911F8C8EB7A8084AAEDD55CEDC54D8F80CB55D@iggy.rockstarvancouver.com> Message-ID: <200388161745.418081@centrino> >In what sense do you mean "way of doing things"? Specifically, how Carmack did a faux-Obj-C style of programming with the AI routines, where each file encapsulates a specific set of AI routines, etc. No big deal, but that's where the Lua changes will occur. Brian From nick at rockstarvancouver.com Fri Aug 8 22:26:09 2003 From: nick at rockstarvancouver.com (Nick Trout) Date: Fri, 8 Aug 2003 19:26:09 -0700 Subject: [q2x] Alive Message-ID: <911F8C8EB7A8084AAEDD55CEDC54D8F80CB563@iggy.rockstarvancouver.com> > >In what sense do you mean "way of doing things"? > > Specifically, how Carmack did a faux-Obj-C style of programming with > the AI routines, where each file encapsulates a specific set of AI > routines, etc. No big deal, but that's where the Lua changes will > occur. I assume you mean in src/game, the m_* (monster?) and g_* (game?) files. I can see an object oriented style of C there. So the plan is to make all of the behaviour data (script) driven rather than hard coded. So you don't necessarily have to copy the style that's there either. So all of the "service objects" that allow script to control the game will need to be exposed. How are you planning on doing this? Are you hand coding or using the newly released tolua5? Nick From hook_l at pyrogon.com Fri Aug 8 22:31:54 2003 From: hook_l at pyrogon.com (Brian Hook) Date: Fri, 8 Aug 2003 22:31:54 -0400 Subject: [q2x] Alive In-Reply-To: <911F8C8EB7A8084AAEDD55CEDC54D8F80CB563@iggy.rockstarvancouver.com> Message-ID: <200388223154.209543@centrino> > I assume you mean in src/game, the m_* (monster?) and g_* (game?) Right. > I can see an object oriented style of C there. Sadly, if I recall, most of that was a straight translation of the QC stuff from Q1 then updated for the new monster types. > is to make all of the behaviour data (script) driven rather than > hard coded. Bingo. In theory, Q2 is architected such that the entire game should be loadable from a DLL/script. Unfortunately the reality so far has been that significant modifications still require changes on the client, which is just so wrong it boggles me. But that's what happens when you have deadlines I guess (or when John doesn't write all the code =) ). So my ideal universe consists of the following components: raw client (no game specific knowledge) raw server (no game specific knowledge) client side game logic (cgame.dll in Q3 parlance) server side game logic (gamex86.dll) ref_gl (pure rendering) ref_soft (possibly punting this) Q3 achieved much of this via cgame.dll, but John opted to use a VM instead of a scripting language (not that I can fault him too much for that). The downside is that making mods became much less accessible and portable. Mods for Quake were immediately portable to all architectures because of QC. Mods for Q2 and Q3 required someone recompiling for each target architecture. Using a scripting language like Lua, you once again get portable mods, AND you get the beauty of using a language that is friendlier than C or C++ for newbies. So all the m_* and g_* will eventually become Lua code. There'll be a gamex86_lua.dll bridge that exports the same interface as the C DLL, but it then thunks/dispatches to the Lua scripting system for everything else. I haven't worked out the specifics of how the bridging should work, but overall it should be fairly simple. > there either. So all of the "service objects" that allow script to > control the game will need to be exposed. How are you planning on > doing this? Are you hand coding or using the newly released tolua5? I think hand coding is best, if for no other reason than to make me acutely aware of what is being transported back and forth between Lua and the game. Brian From nick at rockstarvancouver.com Sun Aug 10 20:27:44 2003 From: nick at rockstarvancouver.com (Nick Trout) Date: Sun, 10 Aug 2003 17:27:44 -0700 Subject: [q2x] Alive Message-ID: <911F8C8EB7A8084AAEDD55CEDC54D8F80CB564@iggy.rockstarvancouver.com> > > I can see an object oriented style of C there. > Sadly, if I recall, most of that was a straight translation of the QC > stuff from Q1 then updated for the new monster types. I haven't really used QuakeC. I remember having a look at it a few years ago and not really liking it a great deal. I can't remember why I didn't like it, I think it seemed a bit esoteric. I remember thinking if you're going to all this effort to write a compiler then why not invent something that is easier to use and nicer to read than C - otherwise you might as well use C. > > is to make all of the behaviour data (script) driven rather than > > hard coded. > > Bingo. In theory, Q2 is architected such that the entire game should > be loadable from a DLL/script. Unfortunately the reality so far has > been that significant modifications still require changes on the > client, which is just so wrong it boggles me. Was src/game included in the client and server? I guess the client needs to know about the objects because of animation and prediction? So I assume that your plan is to turn this generic? i.e. so the client handles objects but the server decides how they behave? Clients can start a game based on a common script, with a common object resource database, but thereafter the server is running the show based on client input (with possible network/latency problems). > So my ideal universe consists of the following components: > > raw client (no game specific knowledge) > raw server (no game specific knowledge) > client side game logic (cgame.dll in Q3 parlance) > server side game logic (gamex86.dll) So based on my assumptions, you could have a q2xgame module which contains common functionality for client and server, e.g. object manager, memory manager, math functions etc. > ref_gl (pure rendering) > ref_soft (possibly punting this) Mmmm doesn't seem like an awful lot of sense in supporting software rendering anymore does there? Can't people who don't have a 3D video card (are the using abacuses?!) use Mesa GL? Does that have a software renderer still? > Q3 achieved much of this via cgame.dll, but John opted to use a VM > instead of a scripting language (not that I can fault him too much for > that). The downside is that making mods became much less accessible > and portable. Mods for Quake were immediately portable to all > architectures because of QC. Mods for Q2 and Q3 required someone > recompiling for each target architecture. Not sure what you mean by "VM instead of a scripting language"; most scripting languages have a compiler with some form of intermediate code and a VM to run it on. > Using a scripting language like Lua, you once again get portable mods, > AND you get the beauty of using a language that is friendlier than C > or C++ for newbies. I think a lot of it depends on how friendly the architecture and documentation is rather than the syntax, but I agree it would be easier than C. > So all the m_* and g_* will eventually become Lua code. There'll be a > gamex86_lua.dll bridge that exports the same interface as the C DLL, > but it then thunks/dispatches to the Lua scripting system for > everything else. I haven't worked out the specifics of how the > bridging should work, but overall it should be fairly simple. You'll probably want a lualib.dll which contains Lua and its default libraries. This way other modules could use Lua as well (for example for configuration). The bridge, or binding module, will publish a Lua-esque API for receiving user input and manipulating game objects as you say. > > there either. So all of the "service objects" that allow script to > > control the game will need to be exposed. How are you planning on > > doing this? Are you hand coding or using the newly released tolua5? > > I think hand coding is best, if for no other reason than to make me > acutely aware of what is being transported back and forth between Lua > and the game. I'm just porting my Doris OpenGL tool thing over to Lua 5 and toLua 5 so I'll see how that goes (still have a couple of things to iron out with GLUT). Using an auto binding generator can save an awful lot of effort when writing a large binding and maintaining it. For example it took me less than an hour to move from Lua 4 to Lua 5 in Doris (another hour of messing with CVS :( ). You specify exactly what you want to expose in a package file for toLua. The error handling and binding can be more efficient and flexible if done by hand though. Nick From hook_l at pyrogon.com Sun Aug 10 20:52:10 2003 From: hook_l at pyrogon.com (Brian Hook) Date: Sun, 10 Aug 2003 20:52:10 -0400 Subject: [q2x] Alive In-Reply-To: <911F8C8EB7A8084AAEDD55CEDC54D8F80CB564@iggy.rockstarvancouver.com> Message-ID: <2003810205210.634744@centrino> > then why not invent something that is easier to use and nicer to > read than C - otherwise you might as well use C. The goal of QuakeC was actually to target C programms but in a safe, sandboxed environment with managed memory. I think it achieved its goals quite well (and, in fact, it got quite a few programmers off to a start). > Was src/game included in the client and server? I guess the client > needs to know about the objects because of animation and prediction? The animations, yes, but there were some grosser things that had to be shared for no apparent reason -- like muzzle flash offsets. A lot of the shared information was simply because the programmers were lazy and didn't want to bother broadcasting/architecting a shared data structure -- even though the game is actually setup to do just that! > So I assume that your plan is to turn this generic? I want things to be generic, but not to the point that it stops being interesting or good. =) Basically I don't mind it operating under the "presume we're an FPS with these types of common properties" model, I just get irked when you have to distribute a brand new client just to get some animation data. > So based on my assumptions, you could have a q2xgame module which > contains common functionality for client and server, e.g. object > manager, memory manager, math functions etc. The project is already set up that way, although in shared file form, nothing as elaborate as a DLL. > Mmmm doesn't seem like an awful lot of sense in supporting software > rendering anymore does there? Very low end systems. The issue isn't hardware capability, it's more along the lines of driver stability. If you were doing a budget title aimed at the $20 SKU downloadable games market, the software renderer would be very appealing. But I agree, I'm not quite sure it's relevant these days. > Not sure what you mean by "VM instead of a scripting language"; > most scripting languages have a compiler with some form of > intermediate code and a VM to run it on. i.e. he didn't invent a new scripting language, he actually used the C language and just wrote a VM for it. This was more portable than what happened with Q2, but still not as accessible as a higher level scripting language. > I think a lot of it depends on how friendly the architecture and > documentation is rather than the syntax, but I agree it would be > easier than C. Syntax isn't the concern as much as more basic things like pointers and memory management, traditionally the things that new programmers bump into right away. Brian