Recently there have been several papers examining aspects of the Internet topology. This paper fo... more Recently there have been several papers examining aspects of the Internet topology. This paper follows in that tradition and addresses two issues related to Internet topology. First, we use three properties -- expansion, resilience, and distortion -- to characterize real and generated networks. For these metrics, we find that existing network topology generators differ qualitatively from most real networks. Second,
Proceedings of the 4th international conference on Embedded networked sensor systems - SenSys '06, 2006
Geographic techniques promise highly scalable any-toany routing in wireless sensor networks. In o... more Geographic techniques promise highly scalable any-toany routing in wireless sensor networks. In one thread of research on geographic routing, researchers have explored robust, distributed graph planarization. Arguing that such planarization techniques have high overhead, researchers have more recently pursued a thread in which they propose precomputation of routing structures (e.g., hull trees and grids) to achieve low-overhead geographic routing.
Proceedings of the 2nd international conference on Embedded networked sensor systems - SenSys '04, 2004
Existing geographic face routing algorithms use planarization techniques that rely on the unit-gr... more Existing geographic face routing algorithms use planarization techniques that rely on the unit-graph assumption, and thus can exhibit persistent routing failure when used with real radios, whose connectivity violates that assumption. In this paper, we describe the Cross-Link Detection Protocol (CLDP), which enables provably correct geographic routing on arbitrary graphs. Our simulations show that the protocol is practical: it incurs low overhead and exhibits low path stretch in wireless networks.
Proceedings of the second ACM SIGCOMM workshop on Hot topics in software defined networking - HotSDN '13, 2013
ABSTRACT The CAP theorem showed that it is impossible for datastore systems to achieve all three ... more ABSTRACT The CAP theorem showed that it is impossible for datastore systems to achieve all three of strong consistency, availability and partition tolerance. In this paper we investigate how these trade-offs apply to software-defined networks. Specifically, we investigate network policies such as tenant isolation and middlebox traversal, and prove that it is impossible for implementations to enforce them without sacrificing availability. We conclude by distilling practical design lessons from our observations.
Proceedings of the 11th ACM Workshop on Hot Topics in Networks - HotNets-XI, 2012
ABSTRACT We present, pFabric, a minimalistic datacenter fabric design that provides near-optimal ... more ABSTRACT We present, pFabric, a minimalistic datacenter fabric design that provides near-optimal performance in terms of completion time for high-priority flows and overall network utilization. pFabric's design eliminates nearly all buffering on switches (switches have only ~20KB of buffering per port), requires almost no congestion control and uses only simple mechanisms at each switch. Specifically, switches are only required to locally and greedily decide what packets to schedule and drop according to priorities in the packet header and do not maintain any flow state or rate estimates. Rate-control is almost unnecessary, all flows start at line-rate and only slow down in the extreme case of congestion collapse. We show via simulations using realistic workloads and topologies that this simple design achieves near optimal flow completion times and network utilization.
Given the need to provide users with reasonable feedback about the "costs" their network usage in... more Given the need to provide users with reasonable feedback about the "costs" their network usage incurs and the increasingly commercial nature of the Internet, we believe that the allocation of cost among users will play an important role in future networks. This paper discusses cost allocation in the context of multicast flows. The question we discuss is this. When a single data flow is shared among many receivers, how does one split the cost of that flow among the receivers? Multicast routing increases network efficiency by using a single shared delivery tree. We address the issue of how these savings are allocated among the various members of the multicast group. We first consider an axiomatic approach to the problem, analyzing the implications of different distributive notions on the resulting allocations. We then consider a "one-pass" mechanism to implement such allocation schemes and investigate the family of allocation schemes such mechanisms can support.
ABSTRACT Small jobs, that are typically run for interactive data analyses in datacenters, continu... more ABSTRACT Small jobs, that are typically run for interactive data analyses in datacenters, continue to be plagued by disproportionately long-running tasks called stragglers. In the production clusters at Facebook and Microsoft Bing, even after applying state-of-the-art straggler mitigation techniques, these latency sensitive jobs have stragglers that are on average 8 times slower than themedian task in that job. Such stragglers increase the average job duration by 47%. This is because current mitigation techniques all involve an element of waiting and speculation. We instead propose full cloning of small jobs, avoiding waiting and speculation altogether. Cloning of small jobs only marginally increases utilization because workloads show that while the majority of jobs are small, they only consume a small fraction of the resources. The main challenge of cloning is, however, that extra clones can cause contention for intermediate data. We use a technique, delay assignment, which efficiently avoids such contention. Evaluation of our system, Dolly, using production workloads shows that the small jobs speedup by 34% to 46% after state-of-the-artmitigation techniques have been applied, using just 5% extra resources for cloning.
ABSTRACT We typically think of network architectures as having two basic components: a data plane... more ABSTRACT We typically think of network architectures as having two basic components: a data plane responsible for forwarding packets at line-speed, and a control plane that instantiates the forwarding state the data plane needs. With this separation of concerns, ensuring connectivity is the responsibility of the control plane. However, the control plane typically operates at timescales several orders of magnitude slower than the data plane, which means that failure recovery will always be slow compared to data plane forwarding rates. In this paper we propose moving the responsibility for connectivity to the data plane. Our design, called Data-Driven Connectivity (DDC) ensures routing connectivity via data plane mechanisms. We believe this new separation of concerns -- basic connectivity on the data plane, optimal paths on the control plane -- will allow networks to provide a much higher degree of availability, while still providing flexible routing control.
Recently there have been several papers examining aspects of the Internet topology. This paper fo... more Recently there have been several papers examining aspects of the Internet topology. This paper follows in that tradition and addresses two issues related to Internet topology. First, we use three properties -- expansion, resilience, and distortion -- to characterize real and generated networks. For these metrics, we find that existing network topology generators differ qualitatively from most real networks. Second,
Proceedings of the 4th international conference on Embedded networked sensor systems - SenSys '06, 2006
Geographic techniques promise highly scalable any-toany routing in wireless sensor networks. In o... more Geographic techniques promise highly scalable any-toany routing in wireless sensor networks. In one thread of research on geographic routing, researchers have explored robust, distributed graph planarization. Arguing that such planarization techniques have high overhead, researchers have more recently pursued a thread in which they propose precomputation of routing structures (e.g., hull trees and grids) to achieve low-overhead geographic routing.
Proceedings of the 2nd international conference on Embedded networked sensor systems - SenSys '04, 2004
Existing geographic face routing algorithms use planarization techniques that rely on the unit-gr... more Existing geographic face routing algorithms use planarization techniques that rely on the unit-graph assumption, and thus can exhibit persistent routing failure when used with real radios, whose connectivity violates that assumption. In this paper, we describe the Cross-Link Detection Protocol (CLDP), which enables provably correct geographic routing on arbitrary graphs. Our simulations show that the protocol is practical: it incurs low overhead and exhibits low path stretch in wireless networks.
Proceedings of the second ACM SIGCOMM workshop on Hot topics in software defined networking - HotSDN '13, 2013
ABSTRACT The CAP theorem showed that it is impossible for datastore systems to achieve all three ... more ABSTRACT The CAP theorem showed that it is impossible for datastore systems to achieve all three of strong consistency, availability and partition tolerance. In this paper we investigate how these trade-offs apply to software-defined networks. Specifically, we investigate network policies such as tenant isolation and middlebox traversal, and prove that it is impossible for implementations to enforce them without sacrificing availability. We conclude by distilling practical design lessons from our observations.
Proceedings of the 11th ACM Workshop on Hot Topics in Networks - HotNets-XI, 2012
ABSTRACT We present, pFabric, a minimalistic datacenter fabric design that provides near-optimal ... more ABSTRACT We present, pFabric, a minimalistic datacenter fabric design that provides near-optimal performance in terms of completion time for high-priority flows and overall network utilization. pFabric's design eliminates nearly all buffering on switches (switches have only ~20KB of buffering per port), requires almost no congestion control and uses only simple mechanisms at each switch. Specifically, switches are only required to locally and greedily decide what packets to schedule and drop according to priorities in the packet header and do not maintain any flow state or rate estimates. Rate-control is almost unnecessary, all flows start at line-rate and only slow down in the extreme case of congestion collapse. We show via simulations using realistic workloads and topologies that this simple design achieves near optimal flow completion times and network utilization.
Given the need to provide users with reasonable feedback about the "costs" their network usage in... more Given the need to provide users with reasonable feedback about the "costs" their network usage incurs and the increasingly commercial nature of the Internet, we believe that the allocation of cost among users will play an important role in future networks. This paper discusses cost allocation in the context of multicast flows. The question we discuss is this. When a single data flow is shared among many receivers, how does one split the cost of that flow among the receivers? Multicast routing increases network efficiency by using a single shared delivery tree. We address the issue of how these savings are allocated among the various members of the multicast group. We first consider an axiomatic approach to the problem, analyzing the implications of different distributive notions on the resulting allocations. We then consider a "one-pass" mechanism to implement such allocation schemes and investigate the family of allocation schemes such mechanisms can support.
ABSTRACT Small jobs, that are typically run for interactive data analyses in datacenters, continu... more ABSTRACT Small jobs, that are typically run for interactive data analyses in datacenters, continue to be plagued by disproportionately long-running tasks called stragglers. In the production clusters at Facebook and Microsoft Bing, even after applying state-of-the-art straggler mitigation techniques, these latency sensitive jobs have stragglers that are on average 8 times slower than themedian task in that job. Such stragglers increase the average job duration by 47%. This is because current mitigation techniques all involve an element of waiting and speculation. We instead propose full cloning of small jobs, avoiding waiting and speculation altogether. Cloning of small jobs only marginally increases utilization because workloads show that while the majority of jobs are small, they only consume a small fraction of the resources. The main challenge of cloning is, however, that extra clones can cause contention for intermediate data. We use a technique, delay assignment, which efficiently avoids such contention. Evaluation of our system, Dolly, using production workloads shows that the small jobs speedup by 34% to 46% after state-of-the-artmitigation techniques have been applied, using just 5% extra resources for cloning.
ABSTRACT We typically think of network architectures as having two basic components: a data plane... more ABSTRACT We typically think of network architectures as having two basic components: a data plane responsible for forwarding packets at line-speed, and a control plane that instantiates the forwarding state the data plane needs. With this separation of concerns, ensuring connectivity is the responsibility of the control plane. However, the control plane typically operates at timescales several orders of magnitude slower than the data plane, which means that failure recovery will always be slow compared to data plane forwarding rates. In this paper we propose moving the responsibility for connectivity to the data plane. Our design, called Data-Driven Connectivity (DDC) ensures routing connectivity via data plane mechanisms. We believe this new separation of concerns -- basic connectivity on the data plane, optimal paths on the control plane -- will allow networks to provide a much higher degree of availability, while still providing flexible routing control.
Uploads
Papers by Scott Shenker