HTB Callfuscated

Overview

Callfuscated looked at first like a normal Linux crackme, but normal reverse-engineering workflows quickly broke down. The binary was stripped, the real validation routine was not represented cleanly as a function, and Ghidra could not produce a trustworthy high-level view of the critical code path.

The final solve came from changing abstraction levels: instead of trying to read the decompiler output, I traced the executed code, recognized a call-threaded VM/stack-machine, recovered its bytecode semantics, and reduced the final password check to simple 32-bit XOR/subtraction constraints.

Initial Triage

The archive contained a single target binary:

rev_callfuscated/crackme

Basic properties:

Runtime behavior:

Welcome to callfuscated crackme.
To register enter your password:
Incorrect flag. Try again

Important strings and addresses discovered during analysis:

entry:                 0x401080
real main-ish target:  0x409002
input buffer:          0x40f080
prompt string:         0x40d008
scanf format:          0x40d04b  "%63s"
success format:        0x40d050  "Correct. Validate the challenge using the flag: %s"
failure string:        0x40d084

Callfuscation Model

The binary used a repeated pattern like:

call target
pop r8
...
call next_block
pop r8

This destroys the assumptions normal disassemblers and decompilers make about calls, returns, and function boundaries. Ghidra could identify many pieces of code, but the apparent functions were not the semantic units that mattered.

Dynamic Analysis

I used controlled dynamic analysis to locate the actual validation flow. The input was stored globally at:

0x40f080

Watchpoints and traces showed that the program consumed exactly 32 bytes of input. A simple prefix oracle did not exist; wrong inputs produced similar late failure states, so the validator was not a straight character-by-character compare.

The decisive late predicate was observed around:

0x40b537: test eax, eax
0x40bcc2: jne  0x4096e1

For bad inputs, eax ended as 0xffffffff and execution reached the failure path.

VM Reduction

Tracing the semantic instruction stream revealed that the validator behaved like a small stack machine. The clean bytecode semantics were recovered as:

The VM built eight big-endian 32-bit words from the 32-byte input and checked each chunk with:

((word ^ const_a) - const_b) == 0

Therefore each required 32-bit input word was simply:

word = const_a ^ const_b

Recovering the Flag

The extracted constant pairs produced the full 32-byte input:

Verification

Combining the chunks gives:

HTB{Sliced_Up_the_Function_4_Ya}

Running the binary with that value reached the success path and printed the expected validation string.

Lessons Learned

• Decompiler failure was a signal, not a dead end.
• The right abstraction was an executed VM trace, not static function recovery.
• Call-threading can make ordinary control-flow graphs actively misleading.
• The obfuscated arithmetic ultimately collapsed to simple XOR/subtraction equations.