[quake3-bugzilla] [Bug 5702] QVM execution crashes when compiled with -fstack-protector-all

[bugzilla-daemon at icculus.org]

https://bugzilla.icculus.org/show_bug.cgi?id=5702

--- Comment #5 from Thilo Schulz <arny at ats.s.bawue.de> 2012-07-07 17:29:40 EDT ---
Yes. I can reproduce the bug now. And I know the reason why this is happening.
The DoSyscall() function is the main entry point for when the VM calls outside
the VM. To work around different calling conventions, I added some inline ASM
that immediately at function start retrieves the arguments from the VM from the
right registers.

This assumes that the compiler doesn't add too much boilerplate that fucks with
the registers. And exactly this is happening with the stack protector. It uses
EAX as temp register to write to the stack:

   0x000000000051a4ca <+12>:    mov    %fs:0x28,%rax
   0x000000000051a4d3 <+21>:    mov    %rax,-0x18(%rbp)
   0x000000000051a4d7 <+25>:    xor    %eax,%eax

However, EAX is used to specify the syscall number. That's why it won't work.
This could be fixed if I stored these variables in memory instead of just
registers. I could also get rid of a whole lot of platform specific code. So I
probably am gonna do this some time in the future.

-- 
Configure bugmail: https://bugzilla.icculus.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.