Convert IDA listings to assembly without using IDA non-free

Question

I'm currently working with a huge set of IDA Pro listings (~1TiB).

For a project I need to perform analyses on the mnemonics and operands on a subroutine level. Therefor I want to convert the IDA format in a plain assembly format, as the existing parser only works for "PLAIN ASM" files.

Additonally this should be done without using a non-free version of IDA.

IDA ASM

.text:00401000                             ; =============== S U B R O U T I N E =======================================
.text:00401000
.text:00401000                             ; Attributes: bp-based frame
.text:00401000
.text:00401000                             sub_401000      proc near           ; CODE XREF: sub_43BD35+Ap
.text:00401000
.text:00401000                             var_C           = dword ptr -0Ch
.text:00401000                             var_8           = dword ptr -8
.text:00401000                             var_4           = dword ptr -4
.text:00401000
.text:00401000 55                                  push    ebp
.text:00401001 8B EC                                   mov     ebp, esp
.text:00401003 83 EC 0C                                sub     esp, 0Ch
.text:00401006 89 4D F4                                mov     [ebp+var_C], ecx
.text:00401009 8B 45 F4                                mov     eax, [ebp+var_C]
.text:00401021 59                                  pop     ecx
.text:00401022
.text:00401022                             loc_401022:                 ; CODE XREF: sub_401000+Fj
.text:00401022 8B 45 F4                                mov     eax, [ebp+var_C]
.text:00401025 83 78 04 00                             cmp     dword ptr [eax+4], 0
.text:00401029 74 12                                   jz      short locret_40103D
.text:0040102B 8B 45 F4                                mov     eax, [ebp+var_C]
.text:0040102E 8B 40 04                                mov     eax, [eax+4]
.text:00401031 89 45 F8                                mov     [ebp+var_8], eax
.text:00401034 FF 75 F8                                push    [ebp+var_8]     ; void *
.text:00401037 E8 B8 0D 00 00                              call    ??3@YAXPAX@Z    ; operator delete(void *)
.text:0040103C 59                                  pop     ecx
.text:0040103D
.text:0040103D                             locret_40103D:                  ; CODE XREF: sub_401000+29j
.text:0040103D C9                                  leave
.text:0040103E C3                                  retn
.text:0040103E                             sub_401000      endp

Convert to:

PLAIN ASM

=============== S U B R O U T I N E =======================================
push    ebp
mov     ebp, esp
sub     esp, 0Ch
mov     [ebp+var_C], ecx
mov     eax, [ebp+var_C]
pop     ecx
mov     eax, [ebp+var_C]
cmp     dword ptr [eax+4], 0
jz      short locret_40103D
mov     eax, [ebp+var_C]
mov     eax, [eax+4]
mov     [ebp+var_8], eax
push    [ebp+var_8]     ; void *
call    ??3@YAXPAX@Z    ; operator delete(void *)
pop     ecx
leave
retn

Question: Are there solutions or scripts already existing for such a (automatically or batch-mode) conversion (at the best without needing a IDA non-free version)?

Answer 1

I assume you already have the assembly output produced by ida and want to parse test only if that is the case you can improvise this.

Assuming i copy pasted the IDA.ASM in your post to a text file.

I can print all the disassembly lines only using the XXX below if you are on windows you need the gnuwin32 port of the unix utilities used in path.

---

Suggestion 1:

Details of the text file:

wc -l ida.asm & echo. & head -n 2 IDA.ASM & echo. & tail -n 2 IDA.ASM & echo. & head -n 11 IDA.ASM | tail -1

Output:

31 ida.asm

.text:00401000                             ; =============== S U B R O U T I N E =======================================
.text:00401000

.text:0040103E C3                                  retn
.text:0040103E                             sub_401000      endp
.text:00401000 55                                  push    ebp

Text parsing to isolate disassembly only from the said text file:

tac ida.asm | awk "{if(a[$1]++==0){print $0} }" | sort

Result as follows:

:\>tac ida.asm | awk "{if(a[$1]++==0){print $0} }" | sort
.text:00401000 55                                  push    ebp
.text:00401001 8B EC                                   mov     ebp, esp
.text:00401003 83 EC 0C                                sub     esp, 0Ch
.text:00401006 89 4D F4                                mov     [ebp+var_C], ecx
.text:00401009 8B 45 F4                                mov     eax, [ebp+var_C]
.text:00401021 59                                  pop     ecx
.text:00401022 8B 45 F4                                mov     eax, [ebp+var_C]
.text:00401025 83 78 04 00                             cmp     dword ptr [eax+4], 0
.text:00401029 74 12                                   jz      short locret_40103D
.text:0040102B 8B 45 F4                                mov     eax, [ebp+var_C]
.text:0040102E 8B 40 04                                mov     eax, [eax+4]
.text:00401031 89 45 F8                                mov     [ebp+var_8], eax
.text:00401034 FF 75 F8                                push    [ebp+var_8]     ; void *
.text:00401037 E8 B8 0D 00 00                              call    ??3@YAXPAX@Z    ; operator delete(void *)
.text:0040103C 59                                  pop     ecx
.text:0040103D C9                                  leave
.text:0040103E                             sub_401000      endp.text:0040103E C3                                  retn

---

Suggestion 2:

Well as i commented if you could print the last line of a group based on first column comparison you can elieminate tac. I searched around SO and found this print last line in each group of result based on comparison results of first column only .

I adapted it to the needs of this question:

awk "{a[$1]=$0} END {for (i in a ) print a[i]}" IDA.lst  | sort | awk "{ print substr($0,42) }" | tr -s [:blank:]

Output:

 push ebp
 mov ebp, esp
 sub esp, 0Ch
 mov [ebp+var_C], ecx
 mov eax, [ebp+var_C]
 pop ecx
 mov eax, [ebp+var_C]
 cmp dword ptr [eax+4], 0
 jz short locret_40103D
 mov eax, [ebp+var_C]
 mov eax, [eax+4]
 mov [ebp+var_8], eax
 push [ebp+var_8] ; void *
 call ??3@YAXPAX@Z ; operator delete(void *)
 pop ecx
 leave
 sub_401000 endp

---

Suggestion 3:

A similar solution based on the suggestions above. This approach will additionally respects the last retn statement and filters out comments (;) at the end of the line:

grep -E "^.{10,15} [0-9A-F]{2} *" IDA.ASM | sort | cut -c40-200 | tr -s [:blank:] | cut -d ";" -f1

Output:

 push ebp
 mov ebp, esp
 sub esp, 0Ch
 mov [ebp+var_C], ecx
 mov eax, [ebp+var_C]
 pop ecx
 mov eax, [ebp+var_C]
 cmp dword ptr [eax+4], 0
 jz short locret_40103D
 mov eax, [ebp+var_C]
 mov eax, [eax+4]
 mov [ebp+var_8], eax
 push [ebp+var_8]
 call ??3@YAXPAX@Z
 pop ecx
 leave
 retn

Pure AWK Based Suggestion

I have an aversion to using n number of utilities to accomplish a job
and i needed to brush up my awkyquotient so here is an awk only code
you may need to test rinse and repeat
it seems to work for the copy paste of plain.asm

{ 
    a[$1] = $0 ;                      # array a will contain last  entry in each group 
    if(b[$1]++==0) c[$1] = $0;        # array c will contain first entry of each group
} END { 
    n = asort(a); m = asort(c);       # sort both arrays
    if (n == m) {                     # some caution    
        for( k=1; k<=n; k++ ){     
            if(a[k]!= c[k]) {         # lets split input into 3 seperate arrays  
                d[k] = a[k]           # one where all entries except last is valid
                e[k] = c[k]           # one where only last entry is valid
            } else {
                f[k] = a[k]           # one where both arrays have same entries
            }
        }
    }
    o = asort(d); p = asort(e) ; q= asort(f) # sort all 3 arrays
    z = 1;
    while(z != o)    {
        g[z] = d[z]                  #insert final array with first n entries 
        z++;                         #(except last entry)
    }
    y = 1;
    while(y != q+1) {
        g[z] = f[y]                  #insert final array with all equal entries 
        z++;y++
    }
    x = p;
    while(x != p-1) {
        g[z] = e[x]                  # insert final array with last entry
        z++;x--
    }
    r = asort(g)                     # sort the final array and print substring
    startofdis = 44;                 # lets set a start    
    for (i = 1; i <= r; i++ ) {
    match(g[i] ,/;.*/);              # match everything after a ; (semicolon)
    if(RLENGTH == -1){               # awk should set the RSTART and RLENGTH
    print substr( g[i] ,startofdis)  # if RLENGHT == -1 no semicolon 
    }                                # lets print from startofdis to end
    else
    print substr(g[i] , startofdis , RSTART-startofdis ) # print only the middle portion
    }
}

Result as follows

:\>awk -f awky.txt ida.lst
        push    ebp
            mov     ebp, esp
            sub     esp, 0Ch
            mov     [ebp+var_C], ecx
            mov     eax, [ebp+var_C]
        pop     ecx
            mov     eax, [ebp+var_C]
            cmp     dword ptr [eax+4], 0
            jz      short locret_40103D
            mov     eax, [ebp+var_C]
            mov     eax, [eax+4]
            mov     [ebp+var_8], eax
            push    [ebp+var_8]
                call    ??3@YAXPAX@Z
        pop     ecx
        leave
        retn

:\>

Answer 2

The IDA database's representation of the binary code which sometimes takes a few liberties for the sake of analyzing the program logic. One of those simplifications is the auto-creation of argument and local variable representation (arg_, var_) that you see.

If you want to get closer to a cleaner assembly representation, try Capstone. If you identify the file offsets for the functions that you need to analyze using IDA (or some other disassembler), you can just feed those to Capstone to generate the disassembly. Also, you can easily write an IDC script (IIRC, the free version of IDA doesn't come with IDAPython support) to dump out the instructions you need in binary form for each function you want to analyze, and pass that into Capstone to generate the disassembly.