I have a simple bit of code, that's invoking OpenSSL. Disassembly shows an expected call;

int __cdecl BN_bin2bn(int a1, int a2, int a3)
{
  return BN_bin2bn(a1, a2, a3);
}

Which points BN_bin2bn to extrn BN_bin2bn:near. So I create a FLIRT signature for OpenSSL and load it. I expected the function to then automatically be transformed to BIGNUM *BN_bin2bn(const unsigned char *s, int len, BIGNUM *ret), but it did not.

I don't see BIGNUM in the structures view, and the function args didn't get converted either. Am I missing a step in getting IDA to properly understand OpenSSL? I've confirmed that the OpenSSL .so is the same one I generated the FLIRT sig file against, so I'm not sure what else I could be missing.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.