The Ptrace Anti-RE Trick

[aetherspawn]

We used to place an INT3 vectored exception handler on function entry points and do everything interesting inside the exception handler. This made the execution stack basically invisible to every debugger since it doesn't debug exception handlers. You can enable/disable interrupts and tracing and whatever you need to do inside the exception handler to guarantee that nobody can see what you are doing and/or verify that no other program has registered another exception handler before doing anything interesting.

If you need to hook functions in third party software, this trick can be used to hook the function without modifying any of the functions code. All you need to do is modify some pointer used by the function to zero, and it will raise an exception as soon as something like p-> is executed on that pointer, then your exception handler can execute whatever code you need (i.e. write over stack, write to memory, exfiltrate handles) and on exit all you need to do is restore the correct register containing the pointer and wind back the execution counter by the size of the de-reference instruction.

Please don't use this knowledge to hurt people ...

[newgre]

A debugger sees exceptions before the application does, and it knows which exceptions it should handle by itself (e.g. breakpoints set by the user) vs passing them to the application. I don't have a windows machine rn to verify but I'd expect your assumption that it's impossible to debug an INT3 VEH to be incorrect.

[15155]

How is the VEH target not immediately visible during static analysis?

[harryfyx]

Do you mean using this trick in malware?

[userbinator]

This "self-debugging" technique is common in the Windows world too:

http://profile.maff1t.com/AntiDebugging/

Interestingly enough, VirtualBox does this too, and they call it "hardening", but IMHO it's quite an unexpected and hostile behaviour which is more characteristic of malware.

[josephcsible]

Thankfully this is easy to circumvent: have your debugger catch the ptrace syscall and lie about the result. Also, if antivirus programs haven't already added a signature for any programs that do that, they should.

[adtac]

The malware will then rely on actual ptrace behaviour as a check. You could instead use seccomp_unotif and let the target ptrace itself as much as it wants: https://man.archlinux.org/man/seccomp_unotify.2.en

[lapcat]

> Thankfully this is easy to circumvent: have your debugger catch the ptrace syscall and lie about the result.

Yeah, Apple iTunes did that IIRC, and it was super easy to bypass.

[MuffinFlavored]

> have your debugger catch the ptrace syscall

Cat + mouse: Have your program catch any signals/stops (which debuggers do on Linux when they attach I believe)

[badrabbit]

Windows also has many similar evasion techniques, like checking if there is a top level exception handler. I use scyllahide, but even on gdb you can break at ptrace and patch it or for automated analysis, just flag anything that used ptrace but isn't a debugger and run it in a sandbox without ptracing it. Audit subsystem might be enough.

https://github.com/x64dbg/ScyllaHide

[hyperman1]

If I remember it right, only 1 ptrace can be attached. So this seems easy to fix:. Just ptrace yourself, and nobody else will.

As a bonus, write important state to memory supposed to be read-only. If someone hooked your ptrace, the hook has to reimplement ptrace in a lot more detail. Or use breakpoints as a mechanism to call subroutines.

[jepler]

In an old $DAY_JOB I used a variant of this as a way to make certain internal errors execute a breakpoint when debugged, and generate an error log if not debugged. iirc this did not happen until after an error had already occurred, so it was not very useful as anti-RE.

[tlb]

I'm curious what the context is where you can't just set a breakpoint on the logging function.

[maffydub]

I've seen this used preemptively - have the process ptrace itself on startup (and then do nothing with it) to make it impossible (or at least far-from-trivial) for other interested parties to ptrace it.

[complex_exp]

You can just patch the call then, right? I.e. turn it into NOPs

[Cloudef]

Some android malware also checks TracerPid in /proc/*/status https://github.com/Cloudef/android2gnulinux/blob/master/src/...

[o11c]

There are also differences in the handling of `SIGTRAP`. For a serious use, this can implement `breakpoint()` function.

[jasondoty]

Tricks for windows

https://anti-debug.checkpoint.com