How was PSP hacked? On first hand, how do people rip the binary out of PSP? I am just curious how all those hardware reverse engineering works.
P.S. Is there a PSP Custom firmware developer community around?
Reverse Engineering Stack Exchange is a question and answer site for researchers and developers who explore the principles of a system through analysis of its structure, function, and operation. It only takes a minute to sign up.Sign up to join this community
I don't know the whole story but AFAIK the initial release of PSP did not use encryption or code signing, so a skilled person was able to make custom code run based off a game. This was used to explore and document the system and develop a customSDK. later firmware updates did implement encryption, but by this time hackers were able to debug code, so they could analyse the firmware update and extract the keys from it.
To answer the question, how did reverse engineers hack the psp?
First the reverse engineer rips the machine apart, and checks all of its components. Then he finds points of interaction. Perhaps the machine has a debugger port, like a JTAG. From there, the engineer attempts to circumvent any security which prevents him from dumping the firmware image. For this stage, researching the exact functionality of each hardware component is critical.
In the link I gave above, the engineer figures out how the NAND chips communicate by guessing various pin configurations. He then uses that knowledge to overwrite parts of memory, and observe the boot up process. Once you understand how the device's operating system boots, it is often possible to overwrite security protections which are selected during boot. Essentially force the device to boot into developer mode, where it may allow you to view sensitive information.
Usually the firmware image wont have source files. It will be a binary dump, which the reverse engineer may disassemble for further analysis. If you are lucky you will be able to dump a the entire file system, or the kernal the device is running.
Once the reverse engineer is possession of the firmware image, if it is the goal to recreate their own firmware, they may begin writing it based on the firmware image:
If you are interested in reverse engineering sony products:
Sony's game consoles are kind of unique in terms of computers. They are usually not set up like a typical x86 computer.
You should learn Sony's proprietary architectures. PSP uses MIPS architecture, which is thankfully, very common. However sony is very fond of making up its own CPU architectures. the PS3 runs on the notoriously complex "Cell" architecture (6 core!) for instance. http://www.gamefaqs.com/boards/620272-playstation-vita/64995907 gives the basics.
Read up on the various play station hacking sites: http://www.ps3hax.net/forum.php (beware, invasive ads)