Why is true emulation not possible?

Question

Antiviruses and similar analysis engines often face the problem of identifying whether the file is harmful. They often do so with the use of (partial)emulation and as a result often fall prey to the tricks (anti-emulation) used by the binary.Is it possible to emulate a binary to such an extent that it becomes impossible for it to identify whether it is running in a virtual environment?

Answer 1

The question is wrongly placed

You are asking the wrong question. Literally. The question is by no means why it isn't possible (it is possible in many cases). The better question is: why it isn't practical?

It's interesting to ask it, nevertheless.

Why not use hardware assisted virtualization?

For starters I've had arguments in the past with certain colleagues (I work in the AV industry) and tried to get across that in certain hardware virtualization methods you gain speed without losing security compared to own emulation implementations. Some of these colleagues held that malware must never we executed natively on a machine without an air gap. Personally I consider this a questionable statement because of the existing proliferation of malware and because todays hardware virtualization features offer (near-)native execution anyway. But I guess it's a matter of taste and politics in the end.

Aside from that you'll have to have privileged access to the system in order to control the hypervisor. This may be fine in the context of a file system filter, which runs in kernel mode, but will not be an option in other purely user mode scenarios (like a command line scanner).

And then you have only "emulated" the machine, not the (operating system) environment in which the harmless or malicious code would normally be running.

Speed matters in AV

However, concerning the practicality the problem mostly boils down to speed. If you consider that AV file system filters scan every object at least once, that's a lot.

Now the AV engines will usually try to make sure that there are static unpackers for certain executable packers so that this won't have to be emulated and so on. There are also other heuristics and static methods in place before it gets down to emulation.

But still in this case there will be a sizable fraction of the overall scanned files that will have to be emulated, even if just in part. Since emulation is usually at least an order of magnitude slower than native execution, this adds up quickly, even if only parts of the overall code get emulated in the end.

Which system to emulate?

Now this seems to be an easy one at surface. Always emulate the one on which you're running.

The problem then becomes how to put a whole OS installation into your engine. Now you may counter: "why don't you use the libraries of the OS you're running on", to which I will respond that this works only for this particular use case above. But how do I emulate Win32 APIs when running on a PowerPC under AIX? Or in your Android phone on an ARM processor?

Our scanners are expected to run across a variety of operating systems and processor architectures and that limits what's possible while maintaining the necessary speed when scanning files/objects.

How closely should the emulator follow the real environments' behavior?

If you have ever tried ReactOS - an open source project that aims to reimplement the binary interfaces of Windows XP and 2003 Server true to the last detail - with anything but the stuff that comes on the CD image, you'll know that it has all kinds of glitches. Wine as well has a lot of glitches (ReactOS and Wine share a lot of code).

AV emulation usually takes many more shortcuts than Wine, because a lot of the functionality isn't required. Let a function succeed or fail and it's fine. The problem is in the very fine details of the Win32 API. And there are loads of those.

Windows 95, 98, Me, NT 4, 2000, XP, 2003, Vista, 2008, 7, 2008 R2, 8, 2012, 8.1, 2012 R2

... and then you should care for Linux and Mac malware, too? And what about other circumstances like certain hardware configurations (think Stuxnet and how it was "tied" to certain USB keys).

Basically if you "emulate" an executable to find certain indicators for maliciousness or goodness your requirements are different from when you emulate a whole operating system or a machine on which you can run the operating system as if it ran on real hardware.

Conclusion: an approximation is enough

So an approximation of the real environment is usually enough. Besides you should keep in mind that many of the evasion attempts themselves can be detected, are suspicious and will raise flags.