I am investigating an application, which is encrypting it's files and stores them into disk. Of course i know where the files are and their corresponding filenames.

In order to find out how the decryption takes place i need to somehow break the execution on the opening of the file from the disk.

Has anybody any idea if something like this is possible?

The main problem that i have is that i am completely unable to detect how the application reads the file from disk. How am i supposed to find the module that is being called?

The application is a 64bit one and i am using Cheat Engine to debug it. I tried other 64bit debuggers as well, but none of them matched the memory search that is possible with CE.

PS: I've also posted this question in Stackoverflow, but i guess this is the best place to ask this question.

  • I'm not sure whether this should be marked a duplicate of reverseengineering.stackexchange.com/questions/9168/…, but the method i explained in my answer there would probably work for you as well. – Guntram Blohm supports Monica Jun 21 '15 at 16:55
  • You could try (unix/linux) GDB and do something like break fopen64 if strcmp($rdi, "yourfilename") == 1, if the binary still has function names available (check out with info functions). not sure if mingw-gdb (windows) works in similar ways too – phil294 Aug 15 '18 at 17:09

use procmon from Sys Internals to log the process
filter the log for file access
(latest release has a file summary tab that simplifies entering filter expressions to mere mouse clicks)

code for a simple xorrer (takes an input file and writes back a xorred file)

#include <stdio.h>
#include <stdlib.h>
#define ENOENT 2
void main(int argc, char *argv[]) {
  if(argc !=3 ) { printf("usage : %s infile outfile",argv[0]); return;}
  FILE *fp = 0; errno_t err=ENOENT; long flen =0,bread =0 ; char *buff =0;
  if (((err = fopen_s(&fp,argv[1],"rb")) == 0) && (fp !=0)) {
    flen = ftell(fp);
    if((buff = (char *)calloc(flen,sizeof(char))) !=0 ) {
      if (( bread = fread(buff,sizeof(char),flen,fp) ) == flen) {
        fclose(fp); err=ENOENT;
        for(int i = 0; i< flen; i++)  {
          buff[i] ^= 'A' ;
        if(((err = fopen_s(&fp,argv[2],"wb")) == 0) && (fp !=0)) {

a batfile logging this file access (blind run) and loading the log into procmon again for applying filters and saving back the filtered events as xml which allows powershell to parse and print

echo off
start procmon.exe /quiet  /minimized /backingfile /nofilter .\LogFile.pml
procmon.exe /waitforidle
start /wait encfile.exe rawdata.txt encdata.txt
procmon.exe /terminate
start /wait procmon.exe /openlog .\logfile.pml
powershell  ([xml] ( Get-Content .\logfile.xml)).procmon.eventlist.event[2].stack.frame 

filter used to save xml file was "Include if path contains xxxx where xxxx is the filename of interest"

here is the stack of the fileRead

PS > ([xml]
 ( Get-Content .\logfile.xml)).procmon.eventlist.event[2].stack.frame

depth               address             path                location
-----               -------             ----                --------
0                   0xb9ed5888          C:\WINDOWS\Syste... FltpPerformPreCa...
1                   0xb9ed72a0          C:\WINDOWS\Syste... FltpPassThroughI...
2                   0xb9ed7c48          C:\WINDOWS\Syste... FltpPassThrough ...
3                   0xb9ed8059          C:\WINDOWS\Syste... FltpDispatch + 0...
4                   0x804ee129          C:\WINDOWS\syste... IopfCallDriver +...
5                   0x80571d9c          C:\WINDOWS\syste... NtReadFile + 0x580
6                   0x8053d658          C:\WINDOWS\syste... KiFastCallEntry ...
7                   0x40364c            C:\Documents and... encfile.exe + 0x...
8                   0x403ac0            C:\Documents and... encfile.exe + 0x...
9                   0x4033a2            C:\Documents and... encfile.exe + 0x...
10                  0x4015bf            C:\Documents and... encfile.exe + 0x...
11                  0x401698            C:\Documents and... encfile.exe + 0x...
12                  0x4016d1            C:\Documents and... encfile.exe + 0x...
13                  0x4010d3            C:\Documents and... encfile.exe + 0x...
14                  0x401d09            C:\Documents and... encfile.exe + 0x...
15                  0x7c817077          C:\WINDOWS\syste... BaseProcessStart...
16                  0x0

ascertaining the ReadFile call

:\cdb -c "ub 40364c;q" encfile.exe | tail -n 2
00403646 ff1550b04000    call    dword ptr [image00400000+0xb050 (0040b050)]

:\cdb -c ".printf \"%y\n\",poi(40b050);q" encfile.exe | tail -n 2
kernel32!ReadFile (7c801812)
| improve this answer | |
  • 1
    omg, i completely forgot about procmon. Thanks a million times mate, did not need to use the console version, modified my filters appropriately and i found out exactly who created the file (it was another bootstrap process btw). I hope i find my way in there now :) – Greg K. Jun 21 '15 at 23:16
  • glad to be of help but i didn't use any console version i just scripted the gui version to provide me appropriate results conducible to text / xml parsing (actually procmon has a proprietery format for providing filter expression via commandline (/config.\xxxx.pmc ) which i use in a generic powershell ps1 file with Start-process cmdlet try it sometimes it eases the pain of stepping through utter crap looking for hay in pricking needle stacks – blabb Jun 22 '15 at 4:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.