3

I was testing some 'C' code with VLAs. It wasn't that surprising that Hex-Rays was not able to recognize VLA. As with structures there is some way to make it understand (simply making the variable type of the structure you think it is) - is there similar thing for VLAs?

  • you could simply call malloc() after obtaining the size of the array. – user3629249 Jan 4 '16 at 7:04
1

It looks like Hexrays does not support this.

Following the instructions for variable-length arrays (https://www.hex-rays.com/products/ida/support/tutorials/varstr/index.shtml) shows that it does work for IDA disassembly and Set Type (hotkey Y), but not in the decompiler output (Hexrays).

Example source:

typedef struct {
  int a;
  int b;
  char c[0];
} foo;

int main() {
  foo* f = malloc(512);
  f->c[123] = 0;
}

Struct defined in IDA:

00000000 struc_1         struc ; (sizeof=0x8, mappedto_1, variable size)
00000000 a               dd ?
00000004 b               dd ?
00000008 c               db 0 dup(?)             ; XREF: main+1D/w
00000008 struc_1         ends

Example disassembly after annotation:

push    ebp
mov     ebp, esp
and     esp, 0FFFFFFF0h
sub     esp, 20h
mov     dword ptr [esp], 200h ; size
call    _malloc
mov     [esp+1Ch], eax
mov     eax, [esp+1Ch]
mov     [eax+(struc_1.c+7Bh)], 0
leave
retn

Example decompiled output:

int __cdecl main(int argc, const char **argv, const char **envp)
{
  struc_1 *result; // eax@1

  result = (struc_1 *)malloc(0x200u);
  BYTE3(result[0x10].a) = 0;
  return (int)&result->a;
}
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.