The strategy used by Metasm is referenced in the peer-reviewed literature on their website. Look at the article published in the Journal of Computer Virology in 2008, in section 3.1. To quote them,
Standard disassembly.
Out of the box, the disassembly engine in Metasm
works this way :
- Disassemble the binary instruction at the instruction pointer.
- Analyse the effects of the instruction.
- Update the instruction pointer.
That sounds more like recursive traversal to me, and less like linear sweep. The engine disassembles the next instruction based upon the effects of the previous instruction, which would allow the disassembly engine to follow branches in the logic, etc.
Also, I have not examined their code in-depth, but in metasm/disassemble.rb
it looks like they maintain some sort of autoanalysis queue for addresses to continue analyzing. Look for functions referencing backtracing - it definitely seems like recursive traversal.