Novell Home

OpenPGM Critique

From Developer Community

This section covers the points which might be considered negatives about this PGM implementation. Is could also conversely be considered a collection of goals for future development.


Contents

GLib Dependency

In order to accelerate the development of OpenPGM and provide a proven cross platform foundation the GLib toolkit is used. Functionality provided by GLib includes:

  • Basic data types: 32 bit, 64 bit integers and pointer definitions.
  • Containers: linked lists, queues, pointer arrays, trash stacks, and hash tables.
  • Asynchronous queue: for delivery of received contiguous packets from the wire to the application.
  • Threading: mutexes, conditions, and read-write locks.
  • Event loops: sources, dispatchers, and IO bindings.


Complicated Parameters

OpenPGM does not provide PGM default parameters, this is necessary to introduce that the parameters need tuning to the destination environment. With very high speed messaging, which means from 100,000 to millions of packets-per-second (pps), it is not practical to have a reliability buffer many seconds in length. Conversely low data rate messaging, such as system monitoring, it is superfluous trying to ensure microsecond latencies.


"Advance with Data" only

The transmit window only advances with data. This means the entire window size of memory is permanently in use. With an "Advance with Time" scheme in quiet period the window will be flushed out freeing memory for the application.


No Adaptive NAK Implosion

Large communication environments can see a significant generation of NAKs. When across multiple networks it can be difficult to fix and determine optimum PGM parameters. Appendix E of the PGM specification introduces a mechanism of adapting the NAK back-off interval to the current network conditions.


No Congestion Control

An extra draft proposal was created for congestion control: draft-ietf-rmt-bb-pgmcc-03. This was not incorporated into the main RFC for PGM, for which OpenPGM just covers the basic functionality. OpenPGM does include rate regulation using a leaky bucket scheme.


No RDMA Support

New technologies like 10 GigE and InfiniBand provide user-space transports through the RDMA protocol. This alleviates the biggest performance problem of system call overhead.

Alternatives to RDMA are more invasive to the operating environment such as multi-packet variants of sendmsg() and recvmsg(), and simply moving the core PGM logic inside kernel space as part of the core networking set. Both still need context switches and do not scale well on the faster networking fabrics.


User-space Protocol Disadvantages

Not integrated with an operating system kernel gives OpenPGM some advantages and disadvantages.

  • High cost of user-space to kernel context switch per packet.
  • No zero-copy UDP system calls increase switching cost.
  • No hardware acceleration of PGM checksumming.
  • NAK processing requires kernel-to-user-to-kernel switches.


General PGM Disadvantages

OpenPGM inherits all the issues fundamental to PGM as a network protocol compared with regular unreliable UDP, or connection orientated TCP.

  • Increased latency over unreliable UDP.
  • High data rates can overwhelm receivers.
  • High packet rates can overwhelm network devices.
  • No security for authentication of peers or verification of content.
  • Back-link from subscriber to publisher.

Novell® Making IT Work As One

© 2008 Novell, Inc. All Rights Reserved.