Papers by Marcel Waldvogel
IEEE Micro, 2002
A library of layered protocol wrappers has been developed that process Internet packets in reconf... more A library of layered protocol wrappers has been developed that process Internet packets in reconfigurable hardware. These wrappers can be used with a reprogrammable network platform called the Field Programmable Port Extender (FPX) to rapidly prototype hardware circuits for processing Internet packets. We present a framework to streamline and simplify the development of networking applications that process ATM cells, AAL5 frames, Internet Protocol (IP) packets and UDP datagrams directly in hardware.
In recent years, Field Programmable Gate Arrays (FP-GAS) have become sufficiently capable to impl... more In recent years, Field Programmable Gate Arrays (FP-GAS) have become sufficiently capable to implement com-plex networking applications directly in hardware. By using hardware that can be reprogrammed, network equipment can dynamically load new functionality.. Such ...
Internet address lookup is a challenging problem because of increasing routing table sizes, incre... more Internet address lookup is a challenging problem because of increasing routing table sizes, increased traffic, higher speed links, and the migration to 128 bit IPv6 addresses. IP routing lookup requires computing the best matching prefix, for which standard solutions like hashing were believed to be inapplicable. The best existing solution we know of, BSD radix tries, scales badly as IP moves to 128 bit addresses. Our paper describes a new algorithm for best matching prefix using binary search on hash tables organized by prefix lengths. Our scheme scales very well as address and routing table sizes increase: independent of the table size, it requires a worst case time of log2(address bits) hash lookups. Thus only 5 hash lookups are needed for IPv4 and 7 for IPv6. We also introduce Mutating Binary Search and other optimizations that, for a typical IPv4 backbone router with over 33,000 entries, considerably reduce the average number of hashes to less than 2, of which one hash can be simplified to an indexed array access. We expect similar average case behavior for IPv6.
The project Crossbow provides a framework to investigate services and mecha-nisms including resou... more The project Crossbow provides a framework to investigate services and mecha-nisms including resource management and packet scheduling for multimedia/mul-ticast applications. In particular the Internet Protocol version 6 (IPv6, IP next generation, IPng) protocol suite on top of ATM is considered to demonstrate possible synergy between ATM and IPv6. The presented architecture includes IPv6 and RSVP, running on BSD Unix using the 1.2 Gbps APIC (ATM Port Interconnect Controller) chip, as well as support for Ethernet networks.
Proposals for multicast security that have been published so far are complex, often require trust... more Proposals for multicast security that have been published so far are complex, often require trust in network components or are inefficient. In this paper we propose a series of novel approaches for achieving scalable security in IP multicast, providing privacy and authentication on a group-wide basis. They can be employed to efficiently secure multi-party applications where members of highly dynamic groups of arbitrary size may participate. Supporting dynamic groups implies that newly joining members must not be able to understand past group communications, and that leaving members may not follow future communications. Key changes are required for all group members when a leave or join occurs, which poses a problem if groups are large. The algorithms presented here require no trust in third parties, support either centralized or fully distributed management of keying material, and have low complexity (O(log N) or less). This grants scalability even for large groups.
All global routing protocols use hierarchies to allow scaling to a world wide community while kee... more All global routing protocols use hierarchies to allow scaling to a world wide community while keeping the routing database size manageable. Databases of variable length prefixes are a powerful tool for providing this in a flexible manner, but require a Longest Prefix Matching algorithm. In this paper, we report a fundamentally new solution that is both algorithmically interesting and practical.
Our scheme is based on doing binary search on hash tables organized by prefix lengths, and scales very well as address and routing table sizes increase: independent of the table size, it requires a worst case time of log2(address bits) hash lookups. With the current Internet Protocol, which uses 32 bit addresses, at most 5 hash lookups are needed; for the upcoming 128 bit addresses of the next generation Internet Protocol (IPv6), 7 lookups suffice. Several refinements, including specializing the Binary Search with every match, considerably reduce the average number of hash search steps to less than 2.
In Layer Four switching, the route and resources allocated to a packet are determined by the dest... more In Layer Four switching, the route and resources allocated to a packet are determined by the destination address as well as other header fields of the packet such as source address, TCP and UDP port numbers. Layer Four switching unifies firewall processing, RSVP style resource reservation filters, QoS Routing, and normal unicast and multicast forwarding into a single framework. In this framework, the forwarding database of a router consists of a potentially large number of filters on key header fields. A given packet header can match multiple filters, so each filter is given a cost, and the packet is forwarded using the least cost matching filter. In this paper, we describe two new algorithms for solving the least cost matching filter problem at high speeds. Our first algorithm is based on a grid-of-tries construction and works optimally for processing filters consisting of two prefix fields (such as destination-source filters) using linear space. Our second algorithm, cross-producting, provides fast lookup times for arbitrary filters but potentially requires large storage. We describe a combination scheme that combines the advantages of both schemes. The combination scheme can be optimized to handle pure destination prefix filters in 4 memory accesses, destination-source filters in 8 memory accesses worst case, and all other filters in 11 memory accesses in the typical case.
Distributed multimedia applications require a variety of communication services. These services a... more Distributed multimedia applications require a variety of communication services. These services and different application requirements have to be provided and supported within (1) end-systems in an efficient and integrated manner, combining the precise specification of Quality-of-Service (QoS) requirements, application interfaces, multicast support, and security features, and within (2) the network. The Da CaPo++ system presented here provides an efficient end-system middleware for multimedia applications, capable of handling various types of applications in a modular fashion. Application needs and communication demands are specified by values in terms of QoS attributes and functional properties, such as encryption requirements or multicast support. Da CaPo++ automatically configures suitable communication protocols, provides for an efficient run-time support, and offers an easy-to-use, object-oriented application programming interface. While its applicability to real-life applications was shown by prototype implementations, performance evaluations have been carried out yielding practical experiences and numerical results.
Middleware supporting secure applications in a distributed environment faces several challenges. ... more Middleware supporting secure applications in a distributed environment faces several challenges. Scalable security in the context of multicasting or broadcasting is especially hard when privacy and authenticity is to be assured to highly dynamic groups where the application allows participants to join and leave at any time.
Unicast security is well-known and has widely advanced into production state. But proposals for multicast security solutions that have been published so far are complex, often require trust in network components or are inefficient. In this paper, we propose a framework of new approaches for achieving scalable security in IP multicasting. Our solutions assure that that newly joining members are not able to understand past group traffic, and that leaving members may not follow future communication.
For versatility, our framework supports a range of closely related schemes for key management, ranging from tightly centralized to fully distributed and even allows switching between these schemes on-the-fly with low overhead. Operations have low complexity (O(log N) for joins or leaves), thus granting scalability even for very large groups. We also present a novel concurrency-enabling scheme, which was devised for fully distributed key management.
In this paper we discuss the requirements for secure multicasting, present our flexible system, and evaluate its properties, based on the existing prototype implementation.
Packet classification is the problem of matching each incoming packet at a router against a datab... more Packet classification is the problem of matching each incoming packet at a router against a database of filters, which specify forwarding rules for the packets. The filters are a powerful and uniform way to implement new network services such as firewalls, Network Address Translation (NAT), Virtual Private Networks (VPN), and per-flow or class-based Quality of Service (QOS) guarantees. While several schemes have been proposed recently that can perform packet classification at high speeds, none of them achieves fast worst-case time for adding or deleting filters from the database. In this paper, we present a new scheme, based on space decomposition, whose search time is comparable to the best existing schemes, but which also offers fast worst-case filter update time. The three key ideas in this algorithm are as follows: (1) innovative data-structure based on quadtrees for a hierarchical representation of the recursively decomposed search space, (2) fractional cascading and precomputation to improve packet classification time, and (3) prefix partitioning to improve update time. Depending on the actual requirements of the system this algorithm is deployed in, a single parameter can be used to tradeoff search time for update time. Also, this algorithm is amenable to fast software and hardware implementation.
With the increasing popularity of firewalls, virtual private networks (VPNs) and Quality of Servi... more With the increasing popularity of firewalls, virtual private networks (VPNs) and Quality of Service (QoS) routing, packet classification becomes increasingly important in the Internet. The high-performance solutions known so far strongly rely on certain properties of the filter database to match against, such as a small number of distinct prefixes or the absence of conflicts. In this paper, we present Line Search as a two-dimensional generalization of the one-dimensional binary search on prefix lengths, exploiting the advantage given by the different approach therein. This algorithm also works best on the filter databases that are expected to occur most often, but degrades gracefully when these assumptions no longer hold. We also show how to efficiently extend the algorithm to a complete five-dimensional Internet Protocol (IP) and transport header match.
Finding the longest matching prefix from a database of keywords is an old problem with a number o... more Finding the longest matching prefix from a database of keywords is an old problem with a number of applications, ranging from dictionary searches to advanced memory management to computational geometry. But perhaps today's most frequent best matching prefix lookups occur in the Internet, when forwarding packets from router to router. Internet traffic volume and link speeds are rapidly increasing; at the same time, a growing user population is increasing the size of routing tables against which packets must be matched. Both factors make router prefix matching extremely performance critical.In this paper, we introduce a taxonomy for prefix matching technologies, which we use as a basis for describing, categorizing, and comparing existing approaches. We then present in detail a fast scheme using binary search over hash tables, which is especially suited for matching long addresses, such as the 128 bit addresses proposed for use in the next generation Internet Protocol, IPv6. We also present optimizations that exploit the structure of existing databases to further improve access time and reduce storage space.
In response to the increasing network speeds, many operations in IP routers and similar devices a... more In response to the increasing network speeds, many operations in IP routers and similar devices are being made more efficient. With the advances in other areas of packet processing, the verification and regeneration of cyclic redundancy check (CRC) codes of the data link layer is likely to become a bottleneck in the near future. In this paper, we present a mechanism to defer CRC verification without compromising reliability. This opens the possibility of incremental updates of the CRC. We introduce a new high-speed technique and present efficient implementations, speeding up CRC processing by a factor of 15. Although the paper and analysis focuses on IP over ATM, the scheme applies to a much wider set of network protocols.
A library of layered protocol wrappers has been developed that process Internet packets in reconf... more A library of layered protocol wrappers has been developed that process Internet packets in reconfigurable hardware. These wrappers can be used with a reprogrammable network platform called the Field Programmable Port Extender (FPX) to rapidly prototype hardware circuits for processing Internet packets. We present a framework to streamline and simplify the development of networking applications that process ATM cells, AAL5 frames, Internet Protocol (IP) packets and UDP datagrams directly in hardware.
The ongoing increases of line speed in the Internet backbone combined with the need for increased... more The ongoing increases of line speed in the Internet backbone combined with the need for increased functionality of network devices presents a major challenge for designers of Internet routers. These demands call for the use of reprogrammable hardware to provide the required performance and functionality at all network layers. The Field Programmable Port Extender (FPX) provides such an environment for development of networking components in reprogrammable hardware. We present a framework to streamline and simplify networking applications that process ATM cells, AAL5 frames, Internet Protocol (IP) packets and UDP datagrams directly in hardware. We also describe a highspeed IP routing module “OBIWAN” built on top of this framework.
We present a new algorithm and framework for dynamic routing of bandwidth-guaranteed flows. The p... more We present a new algorithm and framework for dynamic routing of bandwidth-guaranteed flows. The problem is motivated by the need to set up bandwidth-guaranteed paths in carrier and ISP networks dynamically. Traditional routing algorithms such as minimum-hop or widest-path routing do not take advantage of any knowledge about the traffic distribution or ingress-egress pairs, and therefore can often lead to severe network underutilization. Our work is inspired by the recently proposed "minimum interference routing" algorithm (MIRA) of Kodialam and Lakshman, but it improves on their approach in several ways. Our main idea is to use a "traffic profile" of the network, obtained by measurements or service-level agreements as a rough predictor of the future traffic distribution. We use this profile to solve a multi-commodity network flow problem, whose output is used both to guide our online path-selection algorithm as well as to impose admission control. The offline multi-commodity solution seems very effective at distributing the routes and avoiding bottlenecks around hot spots. In particular, our algorithm can anticipate a flow's blocking effect on groups of ingress-egress pairs, whereas MIRA only considers one ingress-egress pair at a time. Our simulation results show that the new algorithm outperforms shortest-path, widest-path, and minimum interference routing algorithms on several metrics, including the fraction of requests routed and the fraction of requested bandwidth routed. Finally, the framework is quite general and can be extended in numerous ways to accommodate a variety of traffic management priorities in the network.
Label switched networks have become increasingly attractive to both network providers and custome... more Label switched networks have become increasingly attractive to both network providers and customers. By creating aggregate, bandwidth-reserved flows, these networks are known for their routing flexibility, predictable bandwidth usage, and quality-of-service (QoS) provisioning. This flexibility in routing enables fault-persistent QoS reservations, where connectivity and allotted bandwidth remains available, even if some links or network nodes fail. The automatic switch-over from a now-defunct path to a new, working path is known as restoration. Restoring bandwidth-guaranteed paths requires allocation of resources to be used in presence of faults, so-called backup paths. In this paper, we investigate distributed algorithms for routing with backup restoration. Specifically, we propose a new concept of Backup Load Distribution Matrix, that captures partial network state, greatly reducing the amount of routing information maintained and transmitted while achieving efficient bandwidth usage. We present and simulate two new distributed routing algorithms, which provide significant improvements in rejection rates and provide substantial savings in call setup time compared to existing algorithms.
The IP multicast model allows scalable and efficient multi-party communication, particularly for ... more The IP multicast model allows scalable and efficient multi-party communication, particularly for groups of large size. However, deployment of IP multicast requires substantial infrastructure modifications and is hampered by a host of unresolved open problems. To circumvent this situation, we have designed and implemented ALMI, an application level group communication middleware, which allows accelerated application deployment and simplified network configuration, without the need of network infrastructure support. ALMI is tailored toward support of multicast groups of relatively small size (several 10s of members) with many to many semantics. Session participants are connected via a virtual multicast tree, which consists of unicast connections between end hosts and is formed as a minimum spanning tree (MST) using application-specific performance metric. Using simulation, we show that the performance penalties, introduced by this shift of multicast to end systems, is a relatively small increase in traffic load and that ALMI multicast trees approach the efficiency of IP multicast trees. We have also implemented ALMI as a Java based middleware package and performed experiments over the Internet. Experimental results show that ALMI is able to cope with network dynamics and keep the multicast tree efficient.
The keyserver network serves as a repository of OpenPGP keys, providing replication throughout th... more The keyserver network serves as a repository of OpenPGP keys, providing replication throughout the Internet. It currently uses an inefficient and insufficient protocol to keep its nodes synchronized: highly redundant network traffic and excessive overhead due to several thousand e-mail messages per day. Under these conditions, even short network outages cause massive mail server overloads and losses, resulting in continuously diverging databases. In this paper, we present a new protocol to achieve complete synchronization efficiently and automatically, drastically reducing the need for manual intervention. Our protocol transmits only the updates and uses multicast to optimize the amount of data sent. Since support for native multicast is not widely available in the underlying network and current Internet multicast does not scale well, we base our keyserver on ALMI. ALMI is a middleware for reliable applicationlevel multicast, providing scalable join/leave notification of neighbors, significantly reducing the complexity of the application. As a part of this work, we have also implemented a keyserver software which uses our protocol and an efficient RDBMS back-end to hold the keys.
Application-level multicast is a new mechanism for enabling multicast in the Internet. Driven by ... more Application-level multicast is a new mechanism for enabling multicast in the Internet. Driven by the fast growth of network audio/video streams, application-level multicast has become increasingly important for its efficiency of data delivery and its ability of providing value-added services to satisfy application specific requirements. From a network design perspective, application-level multicast differs drastically from traditional IP multicast in its network cost model and routing strategies. We present these differences and formulate them as a network design problem consisting of two parts: one is bandwidth assignment in the overlay network, the other is load-balancing multicast routing with delay constraints. We use analytical methods and simulations to show that our design solution is a valid and cost-effective approach. Simulation results show that we are able to achieve network utilization within 10% of the best possible utilization while keeping the session rejection rate low.
Uploads
Papers by Marcel Waldvogel
Our scheme is based on doing binary search on hash tables organized by prefix lengths, and scales very well as address and routing table sizes increase: independent of the table size, it requires a worst case time of log2(address bits) hash lookups. With the current Internet Protocol, which uses 32 bit addresses, at most 5 hash lookups are needed; for the upcoming 128 bit addresses of the next generation Internet Protocol (IPv6), 7 lookups suffice. Several refinements, including specializing the Binary Search with every match, considerably reduce the average number of hash search steps to less than 2.
Unicast security is well-known and has widely advanced into production state. But proposals for multicast security solutions that have been published so far are complex, often require trust in network components or are inefficient. In this paper, we propose a framework of new approaches for achieving scalable security in IP multicasting. Our solutions assure that that newly joining members are not able to understand past group traffic, and that leaving members may not follow future communication.
For versatility, our framework supports a range of closely related schemes for key management, ranging from tightly centralized to fully distributed and even allows switching between these schemes on-the-fly with low overhead. Operations have low complexity (O(log N) for joins or leaves), thus granting scalability even for very large groups. We also present a novel concurrency-enabling scheme, which was devised for fully distributed key management.
In this paper we discuss the requirements for secure multicasting, present our flexible system, and evaluate its properties, based on the existing prototype implementation.
Our scheme is based on doing binary search on hash tables organized by prefix lengths, and scales very well as address and routing table sizes increase: independent of the table size, it requires a worst case time of log2(address bits) hash lookups. With the current Internet Protocol, which uses 32 bit addresses, at most 5 hash lookups are needed; for the upcoming 128 bit addresses of the next generation Internet Protocol (IPv6), 7 lookups suffice. Several refinements, including specializing the Binary Search with every match, considerably reduce the average number of hash search steps to less than 2.
Unicast security is well-known and has widely advanced into production state. But proposals for multicast security solutions that have been published so far are complex, often require trust in network components or are inefficient. In this paper, we propose a framework of new approaches for achieving scalable security in IP multicasting. Our solutions assure that that newly joining members are not able to understand past group traffic, and that leaving members may not follow future communication.
For versatility, our framework supports a range of closely related schemes for key management, ranging from tightly centralized to fully distributed and even allows switching between these schemes on-the-fly with low overhead. Operations have low complexity (O(log N) for joins or leaves), thus granting scalability even for very large groups. We also present a novel concurrency-enabling scheme, which was devised for fully distributed key management.
In this paper we discuss the requirements for secure multicasting, present our flexible system, and evaluate its properties, based on the existing prototype implementation.