-1

I'm trying to decode this binary file which is from a cache. All I want is the playlist metadata. I am getting some of it using a simple hex viewer, but the majority of the information is random ascii. Is this because there is a bit offset that isn't being accounted for? Or is there something more complicated going on such as hashing or encryption?

2
  • 1
    All of that pastebin is 'random ASCII' (but limited to the range 0-9A-F). If you want someone to take a serious look, provide a binary version -- it's not clear if this data dump is word swapped or not. What makes you think of a 'bit offset', hashing, or encryption here? It's common for a binary file to contain information in binary.
    – Jongware
    May 19 '15 at 10:03
  • @Jongware Here is the binary, any idea how to get started figuring out what the other information is? I would expect timestamps and song lengths for each record.
    – davis
    May 19 '15 at 15:33
4

Converted to binary with 010 Editor, extracted the strings with Strings, and used some clever regex work with Notepad++ to remove some obvious bad strings. Results below.

free
premium
shuffle
all
commercial
everywhere
Fetty Wap
Wiz Khalifa
SKE
Charlie Puth
T-Wayne
Major Lazer
DJ Snake
DJ Snake
Jason Derulo
Jack 
Skrillex
Cbc
Diplo
Justin Bieber
Walk the Moon
The Weeknd
Flo Rida
Sage The Gemini
Lookasx
Mark Ronson
Bruno Mars
Sna
David Guetta
Afrojack
Nicki Minaj
Ellie Goulding
Srl
Various Artistsx
Rihanna
Natalie La Rose
Jeremih
Ed Sheeran
Tove Lo
Drake
Fifth Harmony
Kid Ink
XMS
Omarion
Chris Brown
AlunaGeorge
OMI
Pitbull
Parson James
Andy Grammer
Kanye West
Paul McCartney
Hozier
doon
Lil Wayne
Meghan Trainor
Ariana Grande
Usher
Juicy J
Sam Smith
John Legend
Zedd
Selena Gomez
Rae Sremmurd
Young Thug
Big Sean
Trey Songz
AAC
Fall Out Boy
Theophilus London
Allan Kingdom
Kendrick Lamar
Sia
Deorro
Vance Joy
Trap Queen
Nasty Freestyle
Peace Is The Mission
Want To Want Me
TALKING IS HARD
MY HOUSE
Uptown Special
Love Me Like You Do
YAP
Bitch Better Have My Money
Bitch Better Have My Money
Somebody
Queen Of The Clouds
Queen Of The Clouds
Sex Playlist
You Know You Like It
You Know You Like It
Cheerleader
Globalization
Stole the Show
Magazines Or Novels
FourFiveSeconds
FourFiveSeconds
AAJ
Hozier
Truffle Butter
Lay Me Down
I Want You To Know
I Want You To Know
SremmLife
Dark Sky Paradise
Slow Motion
All Day
To Pimp A Butterfly
In The Lonely Hour
American Oxygen
American Oxygen
Five More Hours
Dream Your Life Away
Trap Queen
Nasty Freestyle
Want To Want Me
Shut Up and Dance
Uptown Funk
Sugar
Bitch Better Have My Money
YAP
Bitch Better Have My Money
Somebody
Thinking Out Loud
Talking Body
Talking Body
Know Yourself
Worth It
You Know You Like It
You Know You Like It
Time of Our Lives
Stole the Show
NPT
FourFiveSeconds
FourFiveSeconds
Take Me To Church
Take Me To Church
Ayo
Energy
Truffle Butter
Dear Future Husband
One Last Time
Lib
One Last Time
Lay Me Down
I Want You To Know
I Want You To Know
Throw Sum Mo
Slow Motion
Legend
Centuries
Centuries
All Day
King Kunta
Elastic Heart
Stay With Me
Stay With Me
American Oxygen
American Oxygen
Riptide
Blessings
Top Tracks in The United States
2
  • Awesome, thanks so much. I would have expected there to be some song lengths and timestamps associated with each record, any advice for recovering that information?
    – davis
    May 19 '15 at 15:26
  • 1
    Yes; reverse engineer the Spotify client to determine how it parses the binary file. May 19 '15 at 15:42
3

One of the close reasons on Reverse Engineering is:

Questions asking for help reverse-engineering a specific system are off-topic unless they demonstrate an understanding of the concepts involved and clearly identify a specific problem.

... and you don't appear to understand the "concepts involved". This is because I asked for a "binary", and you posted a paste bin link that starts like this:

00000000 11011010 11110111 11101111 00101111 01001100 00000000 11011010 11110111 11101111 00101111 00110000 11010011 10101010 00000001 00001001
00000100 01100110 01110010 01100101 01100101 00000001 00000111 01110000 01110010 01100101 01101101 01101001 01110101 01101101 00000001 00000111
(1998 similar lines omitted)

A 'binary', in the context of Reverse Engineering, is an original file. As it is in binary, before you can parse it, you have to know the exact file format, which seems to be not documented. A cursory search for the first few bytes lead to nothing (if they were known, they could indicate a magic number for this particular file type).

But all is not lost yet. After converting the (*cough*) binary to a real file again and inspecting it with a simple hex viewer, I noticed plain text strings, where the byte immediately before it indicated the length of that text string. Further examination - looking at the values immediately following the text strings - indicated there is at least a single other byte before the length/data sequences. Not all data are plain text, but that was enough to quickly make up a C program and display what could be read.

All the program initially did was read two bytes, show them, and then display n characters that immediately follow. I let the program start at a 'reasonable' position, further up in the file, because the first few entries did not seem to follow the exact same format.

This was okay for the very first entry (I got a few lines of data and a text string), but right after that it got lost and didn't display anything useful. Careful examination of the 'failure point' showed that at least one value was special: hex 78, followed by another number, did not indicate that the second number was a data length. So I treated that as a special case: 'no data', and looped on with the rest.

For the first 65 entries this went okay-ish: a regular list of raw data, text string, followed by 4 other lists of raw data. Only after that, the same problem appeared again: the list went 'out of sync' and displayed gibberish again. Further examination showed another problematic type byte: 08. This seems to have two bytes of fixed data. When I treated that as a special case as well, I got loads more useful output.

At that point I stopped, because the general idea was clear. I found it not worth looking further into what the 'raw' data bytes mean, because they do not clearly indicate 'time stamps' or 'song lengths'. The first set of 16 bytes may indicate a hash or an encryption key; the 4 other sets, all 20 bytes, could be the data you are looking for - but they are not in a regular format.

I skipped the first 70 or so bytes because I did not immediately could see what they were for – I strongly suspect they contain metadata about the list itself (its number of entries, for example).

Note that the 'text strings' are encoded using UTF8. The 'unknown' characters in

"Skrillex and Diplo present Jack [c3][9c]"

is actually a regular encoding of "Jack Ü", which is "an American DJ duo, side group and collaborative project" (source). Recognizing 'regular' data sequences such as these is an acquired skill (and verifying that the interpretation is valid is plain common sense; it is only a quick Wikipedia lookup).

Without further a-do, I wrote up the following C program in less time than it took to write this post.

#include <stdio.h>

int main (void)
{
    FILE *f = fopen ("spotify.bin", "rb");
    int i,type,len;

    if (!f)
    {
        printf ("no file?\n");
        return 0;
    }

    fseek (f, 0x45, SEEK_SET);

    do
    {
        type = fgetc(f);
        if (type == EOF)
            break;
        type &= 0xff;
        len = fgetc(f) & 0xff;

        printf ("type %02X len %02X: ", type, len);
        switch (type)
        {
            case 0x08:
                i = fgetc(f);
                printf (" %02X", i & 0xff);
                printf ("\n");
                break;
            case 9:
                printf ("\"");
                while (len--)
                {
                    i = fgetc(f);
                    if (i >= ' ' && i <= '~')
                        putchar (i);
                    else
                        printf ("[%02x]", i & 0xff);
                }
                printf ("\"\n");
                break;
            case 0x78:
                printf ("\n");
                break;
            default:
                while (len--)
                {
                    i = fgetc(f);
                    printf (" %02X", i & 0xff);
                }
                printf ("\n");
        }
    } while (1);

    return 0;
}

Not the answer you're looking for? Browse other questions tagged or ask your own question.