What happens when windows encounters an unknown instruction in a binary?-Collection of common programming errors

11 years ago

admin

1 minute

An application is compiled with SSE3 support and crashes when run on a CPU not supporting SSE3. Gee, so strange! Compiler options for choosing an instruction set must be there just because some programmer at Microsoft was bored as hell one day.

You have several options:

make a single version of the application using SSE2 instruction set only
make different versions of the application compiled with different instruction sets
use structured exception handling (SEH) to implement user-mode emulation of unsupported instructions.

The last approach is a bit more time-consuming than the first two, has some performance issues, but those downsides are much smaller than the advantages it gives you. If you choose the third solution, you will also be able to invent your own opcodes! Perfect way for obfuscating program control flow, which is again very useful for hindering reverse-engineering of your program and thus protectnig your IP.