I would like to write a small thunk which will call an underlying function and then compare the value of rsp
before and after the call. Critically, this code shouldn't clobber any registers.
The obvious implementation is to simply push rsp
before the call, and then compare after. The problem is that push rsp
itself subtracts 8 from the stack, so you should actually compare the saved value of rsp
with the value after the call plus 8.
Here's one way to do it:
foo_thunk:
push rsp ; save the value of rsp
call foo ; call the underlying function
add rsp, 8 ; adjust rsp by 8, popping the saved value
cmp rsp, [rsp - 8]
jne bad_rsp ; bad foo!
ret
The problem is this accesses a value [rsp - 8]
which is above rsp
- i.e., not on the stack but the nebulous region above the stack. This should be where you have a red-zone, but it isn't when you don't.
What are the alternatives? Performance and code-size is important.