-1

My research requires processing memory traces of applications. For C/C++ programs, this is easy using Intel's PIN library. However, as suggested here https://stackoverflow.com/questions/51835532/use-intel-pin-to-instrument-python-scripts, I may need to instrument the Python runtime itself, which I'm not sure will represent the true memory behavior of a given python script due to some overheads(If this is not the case, please comment). Some of the existing python memory profilers only talk about the runtime memory "usage" in terms of the heap space usage, etc.

I ended up making an executable from my python script using PyInstaller and running my PINTool over it. However, I'm not sure if this is the right approach.

Is there any way(any library or any hack into the python runtime) that may help in getting the memory traces accessed by the python scripts?

Improve this question

1 Answer 1

Reset to default
1

Using pin -t <pintool> -- python <python_script.py> is nearly equivalent to compiling your code using Pyinstaller and then running it using pin. Pyinstaller bundles python runtime and your script together, so that user don't have to install python on their end.

Check pyinstrument - it is a python profiling project, it provides you stack track by interrupting program every 1ms.

pyliveupdate allows you to modify your code during runtime.

Python-Instrumentation allows you to dynamically instrument python bytecode, which is simlar to pin, but on bytecode level.

Improve this answer
5
  • Hey, It seems you forgot to insert link for Python-Instrumentation. The description seems promising. Searching on google gave this project pypi.org/project/instrument. Wanted to make sure if this is the one you're referring to?
    – Harsh Kumar
    Apr 19, 2021 at 12:13
  • Just updated the link, but there are plenty of others which use runtime hooking techniques
    – R4444
    Apr 19, 2021 at 14:29
  • Doesn't seem like doing memory access instrumentation. Neither does any branch instrumentation.
    – Harsh Kumar
    Apr 19, 2021 at 16:53
  • I don't exactly know what your use case is. These are some of the examples which may assist you in what you would like to do. you can read more about them in their documentation. otherwise you can use pin or something like dynamorio.
    – R4444
    Apr 19, 2021 at 17:19
  • I want to look at the memory access behaviour (the extent to which memory divergence can be estimated from the program behaviour). See, I already used the pin. But you do know that PIN requires a binary file. Hence the question. For DynamoRIO, I wonder how different would it be than instrumenting the runtime overheads?
    – Harsh Kumar
    Apr 19, 2021 at 18:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.