Efficient Connection-Oriented Communication on High-Performance Networks (Thesis)
Report ID: TR-582-98Author: Damianakis, Stefanos N.
Date: 1998-04-00
Pages: 111
Download Formats: |Postscript|
Abstract:
Network protocols have traditionally been implemented in the kernel in order to provide protected communication between processes. As gigabit network fabrics become available, the kernel protocol stack becomes a communication performance bottleneck, affecting both latency and bandwidth. This dissertation investigates how to use protected, user-level communication mechanisms to support efficient, connection-oriented communication.
Most user-level communication subsystems are designed for custom communication models. To support fast user-level communication, new network interfaces place incoming data directly in user memory, at an address specified by the sender. This capability enables the construction of efficient communication subsystems since the network interface can deliver messages without any software intervention. Such communication subsystems do not support connection-oriented communication well for two reasons. First, direct data transfer does not provide message detection as required by the connection-oriented model, and second, sender-based communication does not provide the anonymity of receivers required by the connection-oriented model.
This dissertation studies how to support efficient, user-level, connection-oriented communication on high-performance networks. First, the issue of message arrival detection for user-level communication models is addressed. The traditional approach uses interrupts to indicate the arrival of new messages, which defeats the benefit of user-level communication since a protection crossing is needed with every message. To minimize the interrupt overhead, a competitive mechanism that combines polling and interrupts to efficiently detect the arrival of new messages at user-level is proposed. Second, the mismatch between connection-oriented APIs and sender-based communication models is addressed. The traditional approach requires receivers to copy messages to their ultimate destinations, which are unknown by senders. To enable zero-copy, connection-oriented protocols, a mechanism that allows the receiver to dynamically change the destination of a message in a sender-based model is proposed.
These ideas were integrated into a user-level communication model and evaluated using a stream sockets implementation on two different multiprocessor platforms. The results from experiments with benchmarks and applications show that these techniques provide significant performance improvements for user-level, connection-oriented communication.