[Gtkradiant] gtkradiant crash on single component jpeg

[reed mideke]

Hello radiant developers,

I've been messing around with using gtkradiant for
wolfenstein, and ran into this problem. I know wolf isn't
supported yet, but this seems to be a general problem. If
not, feel free to ignore me ;-)

The symptom is that gtkradiant (current stable) crashes with
an invalid page fault when selecting 'common' from the textures
menu.(having rigged gtkradiant to load the wolf pk3 files)

To investigate, I downloaded the source (stable/alpha version)
and built it (by building the q3radiant project in the
radiant.dsw workspace)
Then I ran the built gtkradiant exe, and did attach to process
in msdev. The crash happens at libs\jpeg6\jpgload.cpp, function
LoadJPGBuff, line 122.
The image in question is a single component 64x64 jpeg
Looking at the code of LoadJPGBuff, it allocates it's buffer
based on height*width*components, but assumes 4 components later.

Allocation in LoadJPGBuff, starting at line 89:
   nSize = cinfo.output_width*cinfo.output_height*cinfo.output_components;
   out = reinterpret_cast<unsigned char*>(malloc(nSize+1));

The part that crashes, starting at line 113:
   // clear all the alphas to 255
   {
     int i, j;
     unsigned char *buf;

     buf = *pic;

     j = cinfo.output_width * cinfo.output_height * 4;
     for ( i = 3 ; i < j ; i+=4 ) {
       buf[i] = 255; // <--crashes here, i = some value greater than 4096
     }
   }

*pic ends up being the out allocated in the previous sippet.
For a single component .jpeg, j will be 4 times larger than
the allocated buffer, hence the crash.
You can see the image that leads to the crash at
http://www.redshift.com/~rfm/wolf/blackish.jpg
This comes from the wolf pack file:
{wolfdir}/main/pak0.pk3:/textures/common/blackish.jpg

MSDEV stack trace:
LoadJPGBuff(unsigned char * 0x01552360, int 380, unsigned char * * 0x00a6de7c, int * 0x00a6de78, int * 0x00a6de74) line 122 + 12 bytes
LoadJPG(const char * 0x00a6de80, unsigned char * * 0x00a6de7c, int * 0x00a6de78, int * 0x00a6de74) line 1344 + 25 bytes
LoadImageA(const char * 0x00a6de80, unsigned char * * 0x00a6de7c, int * 0x00a6de78, int * 0x00a6de74) line 1390 + 21 bytes
QERApp_Try_Texture_ForName(const char * 0x01552670) line 668 + 33 bytes
CShader::Try_Activate() line 491 + 20 bytes
CShader::Activate() line 499
Texture_ShowDirectory() line 1233
Texture_ShowDirectory(int 60019, unsigned char 0) line 1277
MainFrame::OnTextureWad(unsigned int 60019) line 5433 + 11 bytes
HandleCommand(_GtkWidget * 0x00acf43c, void * 0x0000ea73) line 345
GTK-1.3! 0079362e()

Info from the windows crash dialog:
Q3RADIANT caused an invalid page fault in
module Q3RADIANT.EXE at 015f:004cc8b1.
Registers:
EAX=018a0003 CS=015f EIP=004cc8b1 EFLGS=00010216
EBX=00a9055c SS=0167 ESP=00a6dadc EBP=00a6dd2c
ECX=0000144f DS=0167 ESI=00ab43f0 FS=4faf
EDX=0000144f ES=0167 EDI=00a6f54c GS=0000
Bytes at CS:EIP:
c6 00 ff eb d2 8d 8d 54 fe ff ff 51 e8 de 06 00
Stack dump:
00004000 0000144f 0189ebb4 00000000 004cd62c 004cd696 004cd65e 004cd6f4 004cd837 00000067 00000000 0000003f 00000000 00000000 00000000 00000000

I can submit this to the bugzilla if that is prefered
(I did search for "jpeg crash" and got nothing).
I can also try to fix or work around this, but it seems that
is the kind of thing that should be a no-brainer to someone
more familar with the code than me.
I have not tried this in the unstable branch, but the
code viewable with with the web CVS interface seems to be the
same as what I have.

Best regards and seasons greetings,
-reed (AKA SCDS_reyalP)
--
Email:               rfm(at)redshift.com or rfm(at)portalofevil.com
Home page:                             http://www.redshift.com/~rfm