RapidIO vs. PCIe Peer Communications
By Tom Cox, Executive Director, RapidIO Trade Association

RapidIO supports simple, reliable, efficient true peer-to-peer communications, implemented in hardware that scales from small to large (thousands of processors) systems - PCIe does not!

PCIe specifies that switches must support “peer to peer communications”, which they define at the hardware level as the ability to route a packet headed upstream back downstream if its address matches the BAR configuration on a downstream port. RapidIO peer to peer communications is directly between devices, and it is hardware based and routed.

For small systems, PCIe’s implementation allows multiple processors to communicate. For situations involving simple and infrequent communications where overhead and scalability are not issues, it will work. Typically, however, the processors in such systems are not true peers. The root complex controls and in many cases acts as the source or target for communications, as in the case of a graphics GPU board. In this application, the GPUs are also very tightly coupled from a software perspective, and the processor could be a bottleneck and delay their functions.

Multiprocessing systems larger than a PC typically require multiple cooperating control entities. They often use M+N sparing, load sharing, and other elegant hardware and software mechanisms which require processors to behave as true peers to boost system capacity and reduce cost while maintaining high reliability and efficiency.

PCIe “peer to peer communication” does not scale to meet the requirements of such systems, as PCIe is based on a flat address space shared hierarchically among all components. Vendor-specific support for non-transparent bridging or opaque address spaces does not change this basic reality.

Even from a strictly address-oriented perspective, RapidIO is far more flexible than PCIe, as it routes packets based on a destination ID rather than by an address known to every system processor. This allows the RapidIO protocol to be more efficient and flexible than PCIe, even in small systems.

For more information on RapidIO and technical comparisons of interconnect technologies, go to www.rapidio.org.

Tom Cox is Executive Director of the RapidIO Trade Association. You can reach him at tom.cox@rapidio.org.

RapidIO is a trademark of the RapidIO Trade Association.