Reverse Engineering Stack Exchange is a question and answer site for researchers and developers who explore the principles of a system through analysis of its structure, function, and operation. It only takes a minute to sign up.
Sign up to join this community
I have an unknown .dll from another program which I want to work with. With DLL Export Viewer I was able to find the exported functions.
But to call them I need the information about the parameters and the return type.
Note: I am assuming 32bit x86 on Windows, your question unfortunately doesn't state for certain. But since it's Windows and you don't explicitly mention x64 this was the sanest assumption I could make.
First off, try to search for the function names with a search engine. Don't just settle for a single search engine. Failing that, inspect whatever came in the package with the DLL. Are there import LIBs included? If so, use these to provide clues (may or may not work).
Most disassemblers (read the tag wiki tools) will readily show you exported functions. So locating them won't be a problem at all. They will also usually be shown with their exported names.
From the output your screenshot shows, it looks like the names aren't mangled/decorated. This suggests - but is not conclusive proof - that the functions use the stdcall calling convention (better yet read this one by Ange, one of the moderators pro temp here). Now I don't know how much you know, but since you attempt RCE you are probably well-versed in calling conventions. If not let's sum it up like this: calling conventions govern how (order, alignment) and by what means (registers, stack) parameters get passed to functions. We'll get back to this in a moment. If you are on x64 Windows and the DLL is 64bit as well, you can rely on the Microsoft x64 calling convention (read this article).
If you happen to have a program that uses the DLL in question, you can use a debugger or disassembler to find out both the calling convention and the number of parameters passed. Simply look out for call instructions referencing the exported DLL functions and find mov or push instructions in front. If you happen to come across cdecl functions, the stack pointer (esp) will be adjusted again after the call. It's possible this is the case (see below for an example), but as unlikely as the various compiler-specific fastcall variants, since stdcall provides the broadest possible compatibility.
The methods outlined below in the second approach will also explain some of the concepts introduced here in greater detail.
If you happen to have IDA and you analyze a 32bit DLL, chances are that IDA already identified the number of parameters and the calling convention using its heuristics. Let me demonstrate (using sqlite3.dll). In the Exports tab find a function you're interested in and double-click it. This will take you to the address where the function starts (here sqlite3_open).
As you can see IDA readily found that the function takes two arguments (you can look at the SQLite3 docs to verify this finding). However, there is another thing here. After the call sqlite3_open_v2_0 we can see that the stack pointer is adjusted by 10h (=16) thereby cleaning up four parameters. Looking at the push instructions before the call we can see that indeed four 32bit (i.e. DWORD) parameters are passed via the stack.
Since there is no further cleanup on part of the function sqlite3_open itself, it is now clear that it is likely following the C calling convention (cdecl) as well. Again we can verify the finding (a benefit you won't have) by looking at the documentation. And indeed since no explicit calling convention is given, you end up defaulting to cdecl. The single retn (some disassemblers will show ret), meaning return, also doesn't clean up the stack, since otherwise it would look like retn 8 or similar.
This is a rather small function, but even with the circumstantial information we are able to deduce a lot about it.
Now for something stdcall, a case you are more likely to encounter as mentioned before. And why not go for something famous, like, say, kernel32.dll from Windows 7? Again, I'll take a trivial function as it is better to showcase the points. Note that I told IDA not to make use of the debug symbols from Microsoft and to skip using FLIRT signatures. This means some of the good stuff that kicks in by default is being suppressed to show how to identify what's going on. Look:
The green lines are uninteresting for our case, but you'll encounter them a lot. It is commonly found in several compilers and ebp is commonly referred to as "frame pointer" (frame as in stack frame), basically an offset on which to base access to the stack variables. You can see a typical use in the line push [ebp+arg_0]. IDA figured this out and shows us Attributes: bp-based frame.
We see no adjustment of the stack pointer after call sub_77E29B80, so it looks like that (internal) function follows the stdcall calling convention as well. However, the ret 4 hints that the callee (i.e. the function AddAtomA in this case) is meant to clean up the stack, which means we can exclude cdecl as a possibility. It's four bytes because that is the "natural" size on a 32bit system. You can also see from my inline comments, that parameters are passed on the stack in reverse. But you should know such things anyway before engaging in RCE, otherwise read up in the above linked articles and in some books such as those here.
In this particular case we could dare to make another assumption, but it could bite us. Say this was Microsoft's fastcall convention (keep in mind that they vary by compiler), then the registers ecx and edx would be used, followed by arguments passed on the stack. This means that in our case we might want to assume that this can't be the case, because those registers aren't saved before calling sub_77E29B80. This is a good argument for machine-generated code such as this one. However, were this hand-optimized code, the programmer could rely on the knowledge about the calling convention and skip saving/restoring the registers before/after the call. Still, in this case hand-optimized code would be less likely (or unlikely) to make use of the frame pointer. It's three instructions that aren't strictly needed to do the job. So arguing like this - even without prior knowledge - we could now set out to write a little program using the prototype:
int __stdcall AddAtomA(void* unknown)
and use a debugger to see what gets passed. It's generally a tedious process, but a lot of the process - especially finding the number of parameters - can likely be scripted. Also, once you have a single function figured out, it's likely that the calling convention would be the same (exceptions exist, of course) throughout the DLL. Just make sure you analyze a function taking at least one parameter, otherwise you won't be able to distinguish between stdcall and cdecl from the circumstantial data.
You can also simply use dumpbin or a similar tool to script the creation of a test program. This test program would then call the function, check the stack pointer before and after and could thereby distinguish between stdcall and cdecl. You could also play tricks like passing 20 arguments on the stack (if you want to assume stdcall for the experiment) and see how much of that your callee cleaned up. There are loads of possibilities to simply try instead of analyze. But you'll get better (more reliable) results with the first two approaches.
If you need to build an import LIB because you don't want to use GetProcAddress, see this answer by me over on StackOverflow. It shows how to build an import LIB just from the DLL.
The methods won't differ too much with other disassemblers, I just needed to show things in a way you can reproduce them, that's why I went with IDA. The freeware edition of IDA will likely be sufficient (32bit, PE, x86) - keep in mind it's not permissible for commercial use, though.
Screenshots taken from IDA 6.4.
