It so happens that for my "Intro to IA class" I wrote what ended up being a decently thorough summary of ROP. No, this was never published, but we were told we had to follow the academic format for the paper.
What your question references is actually a summary of three previous papers, which means you're missing some valuable detail on Schacham's excellent work. And of course, a long list of hacker's going back at minimum to phrack 49's Aleph One piece. There's a graphic here that details the long history that has culminated in the creation of ROP. It was essentially a roadmap for my own research.
The paper that will explicitly answer your question is "The Geometry of Innocent Flesh on the Bone: Return-into-libc without Function Calls (on the x86)" as perror has already suggested. In it is provided pseudocode that tells you we're looking for "non-boring sequences" which is defined fully in section 2 with pseudocode. Later papers demonstrated that ROP was cross-architecture in nature and we can use certain jmp sequences that jbh discussed.
A link to my non-peer reviewed unpublished summary is here. Included is a list of references, which I will just drop here:
1 FX & Halvar Flake , We Can Take it From Here. Defcon 12 Media
Archives,
http://www.defcon.org/html/links/dc-archives/dc-12-archive.html
2 Alan Turing. “On Computable Numbers, with an Application to the
Entscheidungsproblem” Proceedings of the London Mathematical Society,
(Ser. 2, Vol. 42, 1937)
3 Nergal.The advanced return-into-lib(c) exploits (PaX case
study).Phrack Mag-azine, 58(4), Dec. 2001.
http://www.phrack.org/archives/58/p58_0x04_Advanced%20return-into-lib(c)%20exploits%20(PaX%20case%20study)_by_nergal.txt.
4 Hovav Shacham. The Geometry of Innocent Flesh on the Bone:
Return-into-libc without Function Calls (on the x86) . CCS '07
Proceedings of the 14th ACM conference on Computer and communications
security, 2007
[5] Erick Buchanan, Ryan Roemer, Hovav Shacham, Steven Savage. When
Good Instructions Go Bad: Generalizing Return-Oriented Programming to
RISC. CCS '08 Proceedings of the 15th ACM conference on Computer and
communications security
[6] ASCC Reference Room
http://www-03.ibm.com/ibm/history/exhibits/markI/markI_feeds.html
[7] Aurélien Francillon, Claude Castelluccia . Code Injection Attacks
on Harvard-Architecture Devices. CCS '08 Proceedings of the 15th ACM
conference on Computer and communications security
[8] Processors – ARM. http://www.arm.com/products/processors/index.php
[9] Wikipedia. ARM9. http://en.wikipedia.org/wiki/ARM9#Chips
[10] Stephen Checkoway, Hovav Sacham, et. al. Return-oriented
programming without returns. CCS '10 Proceedings of the 17th ACM
conference on Computer and communications security
[11] Ryan Roemer, Erik Buchanan, Hovav Shacham, Stefan Savage.
Return-Oriented Programming: Systems, Languages, and Applications.
ACM Transactions on Information and System Security (TISSEC) - Special
Issue on Computer and Communications Security Volume 15 Issue 1, March
2012.
push dword 0
,ret
.call
to a register or function.