Help deciphering binary that creates 3 passwords

Question

I found a binary on a device I've been working on, it's called genrandpass and the only user input is a public key (which is a .bin file) that's stored locally on the device. It also gathers some other information, possibly from the environment to produce three passwords (there is a reference to the Box ID using strings). Looking thru the shell scripts pertaining to genrandpass, it takes the three generated passwords and:

• uses the first password as the root password
• uses the second password is called $epass and the third is called $spass (I only note the names in case it means anything to someone smarter than I.)
• it then copies $epass and $spass to the dropbear banner file and these are displayed whenever someone connects to dropbear (prior to even logging in).

This makes me think that the developer had intended on being able to use the $epass and $spass variables to generate the actual root password. There are a few certificates and one Private key (decryption.pem) on the box itself. Not sure if it's the right key to decipher the code but I'd like to try because I have a few other models of these boxes that I haven't been able to break into yet. I'm just not sure what the correct commands are to try. Any ideas?

genrandpass file at https://dl.dropboxusercontent.com/u/23091/genrandpass

examples of the last 2 passwords generated:

$epass: SxTV2Z7TFvU0XKP/lYYTDlKAhlRR2jwkDGbWPF68go/oOx6x4Pr5DeyNRlx9oQGF05sHld/vyXXchmxlbzsVzPIwocWIq3OIr3J+ZFJrJYPss9VE7YWrwpyRlGwTVHDvZGIzCKXcaipJd85ldLiWUrNxMl4g+5kzwVA2a3I8LuiuixRFVmc8ji/W2W5ZeU5FTcbaiNjlpoRHjPFUkvHKJ4nHSfXpZuLDRS53hxcSnb8ZmvTmFP4ITAdyj9Yw+C2pvD+gSEWRB/H+1cFPQOTi7wr/FY8266QEWqGZw30ZEsMCUNCC0DgiIX+H68QKcU8QFYUJC5+vui3BtcOfFXKHZl==

$spass: RtcTy7fJ11XAQi1P2HiZM4MAxMZMA2NlD6wZL8jNYdrSL5i8qtkGztKDccmGqRWgjiVKI7TcVNcX3PhUSB3UfQCAF6KpBvH7NNezkExdwdM3W2mSnXJvyRLpDSJEgALs0wurUrqIYClZOjTc+xiJzOIUP0Gxb4d2ADOaKXHQ6n6H2Ss/1smITjrbXJ1K8RentZu26sAy3DW+zRIxtxnktSAGUscdG1oytlOL15aAROSL27NUcPSoA3+4o76zggq5TspIBTSmidVRUccEdXPyAzZggR0yqGNrm99uJXHlhw4zCW+GzKJFsJSTwDHZvCoeLERCLuyXFVrgmIISKf6E2V==

Answer 1

Your code has a function at 0x400960 that looks like a main function, and, omitting all initialization (everything gets initialized to 0) and error checking, looks like this:

char input_file_buffer[256];

int finder_id_size=16;
char finder_id[16];

int temp_256=256;
char finder_public_key[256];

char password[31];

int temp_32=32;
char sha_buffer[256];
char rsa_buffer[256];
char base64_input[256];
char base64_output[345];

FILE *fp=fopen(argv[1], "rb");
fread(input_file_buffer, 256, 1, fp);
fclose(fp);

get_finder_public_key(finder_id, &finder_id_size,
                    finder_public_key, &finder_public_key_size);

make_password(password, 30);

printf("%s\n", password);

memmove(sha_buffer+32, password, 30);
memmove(sha_buffer+63, finder_id, 16);
run_sha256(sha_buffer, 256, rsa_buffer+36, &temp_32);
run_rsa_public_decrypt(sha_buffer, 256, base64_input, &temp_256,
                            input_file_buffer, 256);
    // base64_encode is a loop calling encodeblock, not a function,
    // in the original binary. Encodeblock encodes 3 bytes binary
    // input 4 bytes base64 output.
base64_encode(base64_input, 256, base64_output, 345);
printf("%s\n", base64_output);

rsa_buffer[32]=htonl(1);
run_rsa_ks(rsa_buffer, 256, base64_input, &temp_256);
base64_encode(base64_input, 256, base64_output, 345);
puts(base64_output);

So, (because some of the functions are in a shared libary, a part of this is assumptions) your code:

• generates a random password (make_password doesn't have any input)
• outputs that random password
• runs sha256 over a combination of that password and the id of your finder
• runs RSA "decryption" over the sha output, with the public key coming from your input file (?)
• outputs the base64 encoded RSA data
• runs rsa_ks over the different part of the sha output (a 32 byte buffer of something, preceded by a 1, somewhere in the middle of a 256 byte buffer?)
• outputs the result of rsa_ks.

In RSA, you encrypt a message with someone's public key so the someone can use their private key to decrypt it, or you encrypt something with your private key to prove your identity (because the public key can be used to decrypt it). Thus i wouldn't put too much weight on the fact the rsa function is called "decrypt".

I'd assume the run_sha function generates a random 32 bit key, uses that to do the encryption, and saves it to what i call rsa_buffer. Later, run_rsa_ks (ks for key save) rsa-encrypts that sha key. So, if you lose the root password to a device, and ask the vendor for help, they

• use the private key to decrypt the second code, to get the sha key
• use the private key to decrypt the first code, to get the sha output
• use the sha key and the sha output to get a buffer that contains your root password, and the finder id
• verify if the finder id matches the id you told them
• tell you the root password.

Unfortunately, and as i already said in my comment, this means unless you can crack rsa, and don't have any other means to get the private key, your quest ends here. Unless the vendor used a very weak RSA key, but this isn't very probable when they used the effort they did to secure the root password.

Update: I glanced over some of the functions in that libjsonsigner.so, and the get_finder_id_public_key as well as the rsa functions use a device named /dev/vixs/xcodedrv, which hints at a video hardware chip. There's a get_certificate function as well which uses a get_device_certx function that uses the same device. So at least a part of the certificate stuff, as well as the rsa encryption, seem to be hardware-assisted. This means without dynamical analysis on the actual hardware, good chip documentation, and a lot of time, you won't get very much farther (and will probably still smash into a wall at some point because of RSA).