Is there any elegant way to calculate the maximum possible stack depth utilization of a program, using IDAPython?

currently my best approach is iterating through all functions and counting stack-increasing instructions.

  • 2
    Are you looking for the max depth available to the program or the max depth actually utilized by the program? Also, what OS are you focused on? – NirIzr Jul 8 '18 at 12:22
  • looking for max depth utilized by the program and it isn't OS dependent (lets say linux) – Yoav Danieli Jul 8 '18 at 13:59
  • @YoavDanieli maximum depth actually utilized by a program is not the same as maximum possible stack depth utilization, as @NirIzr has pointed out. Please clarify what you are asking. Resource limits such as maximum stack size available to a process are OS dependent. See getrlimit(2) for more on this. – julian Jul 8 '18 at 15:32
  • Yes, thanks, i think @NirIzr clarified this form me :) – Yoav Danieli Jul 9 '18 at 8:46

IDAPython-function get_frame_size(ea) gives size of stack in bytes. So you can get max stack through all available functions:

"0x%04X" % max(get_frame_size(f) for f in Functions(0, BADADDR))
  • Nite that it would be worth calculating the worst possible call depth (using cross references, likely as a directed graph). Special attention should be paid to recursive functions. You should also factor in a worst case for API calls. – David Sep 3 '20 at 21:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.