Common Windows Anti-Debugging techniques in C/C++ using winAPI

When conducting malware research, reverse engineering, or any kind of analysis, debugging will come into play sooner or later. Demonstrated within this article are a few common techniques that malware may try to utilize to evade and not let your debuggers work properly when trying to analyze the malicious pieces of code. The topics below will also include code written in C to fully demonstrate the evasion techniques.

IsDebuggerPresent()

The first and one of the most simple techniques malware author may try and use against an analyst is using the IsDebuggerPresent function which is a standard winAPI function provided by Microsoft. It’s described on MSDN as seen below.

BOOL IsDebuggerPresent();

Taken from MSDN documentation, there are no required parameters when filling out this function. A simple C program that would try and evade a user-mode debugger is as shown below.

#include <Windows.h>
#include <stdio.h>

int main(){

    if (IsDebuggerPresent()){
        printf("Debugger is active!");
        exit(1);
    }

    return 0;
}

windbg having a bad day

The way this works is that when the API function is called, it checks against the PEB’s (Process Environment Block) 3rd value.

0:000> dt _peb @$peb
ntdll!_PEB
   +0x000 InheritedAddressSpace : 0 ''
   +0x001 ReadImageFileExecOptions : 0 ''
   +0x002 BeingDebugged    : 0x1 ''

It checks this value to determine if the current process is being debugged or not. You can view the populated values version of the PEB of the application, and you can see the BeingDebugged field is set to 0x1 which tells the function call to check if it’s in a debugging environment.

FindWindowA()

Using the FindWindowA Windows function as an anti-debugger technique works under the premise to detect certain application windows that are active on your system. This specific function is used to obtain a handle to the top-level window whose class and window name matches the provided parameter specified strings.

The function is defined from MSDN as seen below.

HWND FindWindowA(
  LPCSTR lpClassName,
  LPCSTR lpWindowName
);
  • The parameters: lpClassName is used to provide the class name which you want to match which is registered by the functions RegisterClass or RegisterClassEx.
  • For lpWindowName member you set the window’s title, if this is set to NULL all the windows names will be set to match. You just need to find the window name of an application. This can be done with basic google research.

This can be implemented in an application as seen below, where it’s providing two checks, one for the OLLYDBG windows, or the WinDbgFrameClass windows, which are related to Olly debugger, or WinDBG. If it detects either, it will invoke an exit call.

#include <windows.h>
#include <stdio.h>

int main(){

    BOOL results = FALSE;
    LPCSTR DetectOlly = "OLLYDBG";
    LPCSTR DetectWinDBG = "WinDbgFrameClass";

    printf("[+] Checking for a debugger...");

    if (FindWindowA(DetectOlly, 0) || FindWindow(DetectWinDBG, 0)){
        MessageBoxA(NULL, "Debugger detected", "Notification", MB_OK);
        exit(1);
    } else {
        MessageBoxA(NULL, "No Debugger detected", "Notification", MB_OK);
    }

    return 0;
}

And now when this application is opened with Olly debugger, you can see that it’s detecting it, and sends a message box popup displaying that it’s indeed within a debugging environment.

ollydbg having a bad day

References

  • http://www.cs.utah.edu/~aburtsev/malw-sem/slides/02-anti-debugging.pdf

Bypassing anti-debugger techniques

For a malware analyst or researcher’s point of view, it may be important to understand the types of techniques that can be used to bypass anti-debugger techniques that are implemented by malware authors to prevent you from debugging their malicious software.

As for the first technique, the IsDebuggerPresent function works by checking the PEB’s third member, and if it has been set to 0x1, then the function knows it’s within a debugging environment.

IsDebuggerPresent bypass

To bypass this anti-debugger technique, we can use a few different tactics.

For runtime patching, which is a temporary solution, you can inject and modify the PEB to equal 0. And another way is you can set EAX to 0 after the function is called

And for a more permanent solution you can patch the application by overwriting the function with NOPs.

Before debugging, another function has been added to the application to note that a debugger has not been detected, the goal is to get the “A Debugger has not been detected” popup while the application is indeed within a debugger.

#include <Windows.h>
#include <stdio.h>

int main() {

    if (IsDebuggerPresent()) {
        MessageBoxA(NULL, "Debugger detected", "Notification", MB_OK);
        exit(1);
    }
    else {
        MessageBoxA(NULL, "A Debugger has not been detected", "Notification", MB_OK);
    }

    return 0;
}

Technique 1 - PEB injection / modification

The main goal of this technique to modify our processes PEB to manipulate the third member BeingDebugged and change it to a 0x0 value, which will allow us to keep debugging. It’s important to note, this is a runtime anti-debugging technique, where this will only work the current session, as the PEB will be reset/change if the application is restarted.

PEB modification can be done by adding the following ASM instructions within the application.

Note! the offsets and values may change between doing this on an x86 or x64 binary!

mov eax,dword ptr gs:[18]
mov eax,dword ptr ds:[eax+60]
mov byte ptr ds:[eax+2],0

To break down exactly what this is doing:

  1. mov eax, dword ptr gs:[18]

FS:[0x18] is the offset for the TEB (Thread Environment Block), and this assembly operation will move a pointer of this into the eax register.

  1. mov eax,dword ptr ds:[eax+60]

Now it is moving a pointer of eax + 60, which is the offset 0x60, this member of the TEB can be seen within WinDBG, this member located at 0x60 (offset) within the TEB, is the PEB. This assembly operating is moving a pointer to the PEB within eax.

0:007> dt _TEB
ntdll!_TEB
   +0x000 NtTib            : _NT_TIB
   +0x038 EnvironmentPointer : Ptr64 Void
   +0x040 ClientId         : _CLIENT_ID
   +0x050 ActiveRpcHandle  : Ptr64 Void
   +0x058 ThreadLocalStoragePointer : Ptr64 Void
   +0x060 ProcessEnvironmentBlock : Ptr64 _PEB <--------------------
  1. mov byte ptr ds:[eax+2],0

And finally, it’s setting eax + 2, which is the BeingDebugged flag (value) to 0.

0:007> dt _PEB
ntdll!_PEB
   +0x000 InheritedAddressSpace : UChar
   +0x001 ReadImageFileExecOptions : UChar
   +0x002 BeingDebugged    : UChar <--------- is set to 0

Now, with this knowledge, while the application is loaded within a debugger. You can see again, at first it detects the debugger, and terminates when ran.

debugger

Before our instructions are injected, you can view the PEB one last time, and you can see the member flag is set to 0x1.

enabled

After we inject it, and patch the application, and finally run the application, you can see the messagebox alerts us that we are in fact “not” within a debugging environment, now if you view the PEB members, you can see it has been set to 0.

[Note] If you are doing this within WinDBG, you can accomplish this same technique via the bp $exentry "eb $peb+0x2 0x0" command, where your editing the peb+0x2 member by editing a single byte “0x0” with the eb command.

windbg

windbg

Updated: