1

I'm trying to decode the file formats for Massive, FM8 and absynth VST synths. The file format is binary with several sections. Reason for this endeavour is to convert the above formats to general vst .fxp format for automated loading and rendering of presets through the vst api.

From experimenting with saving the files when changing synth parameters I've found out following facts about the format:

  • File starts out with length field
  • File contains some fixed length sections introduced by "DSIN", "hsin" markers
  • The vst binary chunk returned by effGetChunk vst sdk api calls are 99% similar to the .nmsv, .nabs, .nfm8 preset file content
  • changing a single parameter in the synth changes some bytes at several places (3-4) in the file
  • The main section of the preset file seems to be compressed, while synth parameters should be float values between 0 and 1 they don't seem to be written as floating points into the file - apart from unrecognizable binary values there are plaintext strings e.g. for user defined macros there seems to be some sort of compression applied to the raw data. Compression seems to reference previously encountered strings and reference these strings in later portions of the stream: If "THIS_IS_A_MACRO_NAME_ABC" is preceding "THIS_IS_A_MACRO_NAME_XYZ" in the stream the second string will be compressed to "[short sequence of bytes]_XYZ".
  • The files do not seem to contain any dictionary for compression which makes me think the dictionary must be stored somewhere else or there might not be a dictionary at all.

Could anyone help out here:

  • What compression scheme could be applied here?
  • Does anyone know about a similar format that has been successfully decoded?

Added an example file:

http://s000.tinyupload.com/index.php?file_id=08960658549599455274 contains a sample file. The string "TESTSTRING1" and "PREFIXTESTSTRING1SUFFIX" are contained in the uncompressed stream.

Shannon entropy is 5.84154 for the data chunk which is somewhere in the middle between english text and encrypted.

enter image description here

Here is an example which should demonstrate how the lenght field is computed: The string "TESTSTRING123" precedes the string "PREFIX...SUFFIX".

                             P  R  E  F  I  X  L1 L2 D     S  U  F  L1 D
----------------------------------------------------------------------------------------
PREFIXTESTSTRING123SUFFIX 05 50 52 45 46 49 58 E0 04 16 02 53 55 46 20 12 40 42 00
PREFIXTESTSTRING12SUFFIX  05 50 52 45 46 49 58 E0 03 16 02 53 55 46 20 11 40 41 00
PREFIXTESTSTRING1SUFFIX   05 50 52 45 46 49 58 E0 02 16 02 53 55 46 20 10 40 40 00 33 40
PREFIXTESTSTRINGSUFFIX    05 50 52 45 46 49 58 E0 01 16 02 53 55 46 20 0F 40 3F 00 33 40
PREFIXTESTSTRINSUFFIX     05 50 52 45 46 49 58 E0 00 16 02 53 55 46 20 0E 40 3E 00 33 40
PREFIXTESTSTRISUFFIX      05 50 52 45 46 49 58 C0    16 02 53 55 46 20 0D 40 3D 00 33 40
PREFIXTESTSTRSUFFIX       05 50 52 45 46 49 58 A0    16 02 53 55 46 20 0C 40 3C 00 33 40
PREFIXTESTSTSUFFIX        05 50 52 45 46 49 58 80    16 02 53 55 46 20 0B 40 3B 00 33 40
PREFIXTESTSSUFFIX         05 50 52 45 46 49 58 60    16 02 53 55 46 20 0A 40 3A 00 33 40
PREFIXTESTSUFFIX          05 50 52 45 46 49 58 60    16 01    55 46 20 09 40 39 00 33 40 
PREFIXTESSUFFIX           05 50 52 45 46 49 58 20    16 02 53 55 46 20 08 40 38 00 33 40
PREFIXTESUFFIX            05 50 52 45 46 49 58 20    16 01    55 46 20 07 40 37 00 33 40 
PREFIXTSUFFIX             09 50 52 45 46 49 58 54 53 55 46 20 06 40 36 00 33 40  
  • 1
    This sounds very like some form of LZSS compression. A similar recent question on here is reverseengineering.stackexchange.com/questions/21981/… – Ian Cook Oct 30 at 10:24
  • Hi Ian, what makes you think this would be LZSS compression? Could it be LZ77, LZ78 or LZW? I think with huffman codeing the code would look less "readable". Is there no dictionary at all or maybe a fixed one stored somewhere else? Thanks alot – Martin Oct 30 at 22:50
  • I've just come across LZSS more than the others. However, 11 consecutive plain text characters means it would have to be a different implementation to those I've seen before. Having some readable text in the compressed data does suggests it's a byte based method though. This likely rules out bit-stream based approaches e.g. anything using Huffman coding, or LZW with 12-bit or variable length codes. – Ian Cook Oct 30 at 23:12
  • I don't really understand yet how length and distance are encoded here - I thought there are definitely some "control characters" (e.g. @ 40h) which appear more frequently than other characters.I'll try to give an examples with different length strings so this might shed some light. – Martin Oct 30 at 23:32
  • It may be zlib. I had a quick look at my Massive in IDA and roughly in the neighbourhood where those DSIN and hsin markers are used and connecting stuff via RTTI names, I found zlib usage. According to the RTTI data they have some custom compressed stream class that appears to use zlib. Although the code also seems to check for 'zlib' or 'none' 4 byte markers which aren't in your sample preset. – Johann Aydinbas Nov 1 at 2:26
2

I think the results of your well-designed tests help explain pretty much everything.

It is an LZ77 variant. Try the following scheme for decoding the length and offsets. (Encoded bytes are shown below as 8 binary bits.)

1st byte  2nd byte  3rd byte      token type  length    offset(*)
========  ========  ========      ==========  ======  ==============
000qqqqq     --        --     =>  literal     1 + Q         --      
001qqqqq  rrrrrrrr     --     =>  dictionary  3       (Q<<8) + R + 1
010qqqqq  rrrrrrrr     --     =>  dictionary  4       (Q<<8) + R + 1
011qqqqq  rrrrrrrr     --     =>  dictionary  5       (Q<<8) + R + 1
100qqqqq  rrrrrrrr     --     =>  dictionary  6       (Q<<8) + R + 1
101qqqqq  rrrrrrrr     --     =>  dictionary  7       (Q<<8) + R + 1
110qqqqq  rrrrrrrr     --     =>  dictionary  8       (Q<<8) + R + 1
111qqqqq  rrrrrrrr  ssssssss  =>  dictionary  9 + R   (Q<<8) + S + 1  

* the offsets are backwards in the decoded data from the current output position.

Edit The whole file from offset 00000410 onwards appears to be decompressible using the above encoding scheme. Doing so gives me the below. You can see that the end of my first excerpt has a number of 32 bit floating point values. The 2nd excerpt has your test string.

00000000:  F9 15 00 00 00 00 00 00 01 00 00 00 68 73 69 6E  ............hsin
00000010:  01 00 00 00 00 00 00 00 EC 2D F1 91 50 BC 4D 41  .........-..P.MA
00000020:  96 E1 F3 EA B1 70 A9 B9 18 00 00 00 00 00 00 00  .....p..........
00000030:  44 53 49 4E 01 00 00 00 01 00 00 00 01 00 00 00  DSIN............
00000040:  01 00 00 00 01 00 00 00 00 00 00 00 44 53 49 4E  ............DSIN
00000050:  6D 00 00 00 A5 15 00 00 00 00 00 00 01 00 00 00  m...............
00000060:  68 73 69 6E 01 00 00 00 00 00 00 00 85 50 7B 20  hsin.........P{ 
00000070:  83 65 FA 41 80 7B CA 65 CA E4 1D FA 75 15 00 00  .e.A.{.e....u...
00000080:  00 00 00 00 44 53 49 4E 6D 00 00 00 01 00 00 00  ....DSINm.......
00000090:  18 00 00 00 00 00 00 00 44 53 49 4E 01 00 00 00  ........DSIN....
000000A0:  01 00 00 00 01 00 00 00 01 00 00 00 01 00 00 00  ................
000000B0:  01 00 00 00 2D 15 00 00 00 00 00 00 18 00 00 00  ....-...........
000000C0:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000000D0:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000000E0:  00 00 00 3F 00 00 7D 3F 00 00 80 3F 00 00 80 3F  ...?..}?...?...?
000000F0:  00 00 00 3F BA 9E FF 3E 00 00 80 3F 00 00 80 3F  ...?...>...?...?
00000100:  00 00 80 3F 00 00 00 3F A6 30 00 3F 00 00 80 3F  ...?...?.0.?...?
00000110:  00 00 70 3F 00 00 80 3F 00 00 00 3F 00 00 80 3F  ..p?...?...?...?
00000120:  00 80 91 3E 00 00 00 3F 00 00 00 00 00 00 00 3F  ...>...?.......?
00000130:  00 00 00 3F 00 00 00 3F 00 00 00 3F 00 00 00 3F  ...?...?...?...?
00000140:  00 00 00 3F 00 40 41 3F 00 00 00 3F 00 00 00 00  ...?.@A?...?....
00000150:  00 80 99 3E 00 00 00 3F 00 00 00 3F 00 00 00 00  ...>...?...?....
00000160:  00 00 00 3F 00 00 00 3F 00 00 00 3F 00 00 00 3F  ...?...?...?...?
...
00000430:  00 00 00 00 00 01 00 00 00 07 00 00 00 00 00 00  ................
00000440:  00 00 00 00 00 01 01 00 00 00 00 C0 00 00 00 40  ...............@
00000450:  01 01 00 00 00 00 00 01 00 00 00 0B 00 00 00 54  ...............T
00000460:  45 53 54 53 54 52 49 4E 47 31 17 00 00 00 50 52  ESTSTRING1....PR
00000470:  45 46 49 58 54 45 53 54 53 54 52 49 4E 47 31 53  EFIXTESTSTRING1S
00000480:  55 46 46 49 58 01 00 00 00 33 01 00 00 00 34 01  UFFIX....3....4.
00000490:  00 00 00 35 01 00 00 00 36 01 00 00 00 37 01 00  ...5....6....7..
000004A0:  00 00 38 04 00 00 00 00 00 00 00 55 55 29 41 02  ..8........UU)A.
...
  • As a note: zlib's DEFLATE is a variation of LZ77 so this is one more hint towards zlib being used. – Johann Aydinbas Nov 3 at 21:32
  • I use gchq.github.io/CyberChef to compress a sample string w DEFLATE - in the compressed string there are no ascii chars of the original text left. If i understand correctly this is due to huffman working on bit-level. I've looked for parameters to run DEFLATE without huffman but no luck. About the dictionary: to INFLATE above block I think I'd require a dictionary which i cannot spot in the preset file. – Martin Nov 5 at 1:43
  • The dictionary is the previously decompressed data. – Ian Cook Nov 6 at 6:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.