How can I monitor a usb dongle's traffic? I would like to see how a program and its usb dongle talk to each other, if it is possible replay this traffic?
Since I am new to this type of thing, any tutorial or tool suggestion is welcome.
It depends on what your budget is like. The best USB analyzers are hardware devices with good protocol dissectors. If you have a huge budget you can go with the various solutions from LeCroy such as the LeCroy Voyager M3i. If you have a decent size budget and you only need USB 2.0, I would go for the Ellisys USB Explorer 200. If you want to replay and change packets you can take a look at the Ellisys USB Explorer 260 as I don't think the 200 is capable of replay. If you need USB 3.0 I would go with the Ellisys USB Explorer 280. On the budget hardware side you have the Beagle 480 and even more budget the Beagle 12.
There's also the more DIY solution which involves running the process you want to monitor in a virtual machine such as VirtualBox and then routing the traffic which goes through the USB ports to your own dissector. You can use Wireshark as a dissector for VM USB traffic.
Personally I would go with the Ellisys Explorer 200 or 260. Either one presents a good compromise between price and quality depending on your needs.
Tho maybe not directly what you are looking for I'd just like to add one item to Peter Andersson's thorough answer. Travis Goodspeed's facedancer (some more recent info ). Its design is also open source.
Facedancer Board, a tool for implementing USB devices in host-side Python using the GoodFET framework. Access to the USB chip is extremely low-level, so protocols may be mis-implemented in all sorts of creative ways. This allows a clever neighbor to quickly find and exploit USB driver vulnerabilities from the comfort of a modern workstation, only later porting such exploits to run standalone.
It can be used for sniffing too.
I couldn't immediately find a link to buy one Pre assembled boards are not yet available, but you can buy the circuit boards here, also Travis is, as he likes to say, a good neighbor and gives them away at conferences. Anyway, it should be pretty cheap option if you can assemble it yourself.
If you aren't afraid to get your hands dirty, on Windows you could always write a filter driver on top of or below the device object for the dongle. The
IoAttachDevice() function and the three other functions starting with that name are your friend. The advantage of that is to have a complete in-software solution for the problem without the expenses involved with hardware sniffers. You'll notice that this is actually what the USBpcap project does in
USBPcapFilterManager.c. So if you are merely writing this for research, this would be a good starting point, unless the GPL is too restrictive for you.
There is a potential downside with in-software solutions. A filter driver will only see what the other drivers want it to see. Keep in mind that in Windows kernel mode all drivers have the same privileges. Even attaching to the root hub may not always yield meaningful results. So if one driver decides to do funny things such as stealing entry points (IRP major function codes) to proactively counter filtering or debugging of any kind, it becomes an arms race. Otherwise it will be the cheapest solution you can get, given you only need this to work on a particular OS.
However, if I was you I'd first sniff the user mode traffic between the library (hook
WriteFile et al, or their native counterparts) and the driver or the library and the application that requires the dongle. Of course if the dongle poses as a HID device you'd have to look for the respective IOCTLs and decode them or hook the HID functions directly. This is the method I was using to investigate the traffic sent between a sports watch with USB connectivity and my box. I based it on PaiMai and pydbg, because the application wasn't guarding against being debugged. If yours does, you may have to cheat a little to "convince" the application to play nicely IDAStealth provided a few good pointers (link is dead as of May 2016) how to go about.