Due to the falling costs of data acquisition and storage, researchers and industry analysts often... more Due to the falling costs of data acquisition and storage, researchers and industry analysts often want to find all instances of rare events in large datasets. For instance, scientists can cheaply capture thousands of hours of video, but are limited by the need to manually inspect long videos to identify relevant objects and events. To reduce this cost, recent work proposes to use cheap proxy models, such as image classifiers, to identify an approximate set of data points satisfying a data selection filter. Unfortunately, this recent work does not provide the statistical accuracy guarantees necessary in scientific and production settings. In this work, we introduce novel algorithms for approximate selection queries with statistical accuracy guarantees. Namely, given a limited number of exact identifications from an oracle, often a human or an expensive machine learning model, our algorithms meet a minimum precision or recall target with high probability. In contrast, existing approaches can catastrophically fail in satisfying these recall and precision targets. We show that our algorithms can improve query result quality by up to 30× for both the precision and recall targets in both real and synthetic datasets.
Predictive Models for Determining If and When to Display Online Lead Forms
Proceedings of the AAAI Conference on Artificial Intelligence
This paper will demonstrate a machine learning application for predicting positive lead conversio... more This paper will demonstrate a machine learning application for predicting positive lead conversion events on the Edmunds.com website, an American destination for car shopping. A positive conversion event occurs when a user fills out and submits a lead form interstitial. We used machine learning to identify which users might want to fill out lead forms, and where in their sessions to present the interstitials. There are several factors that make these predictions difficult, such as (a) far more negative than positive responses (b) seasonality effects due to car sales events near holidays, which require the model to be easily tunable and (c) the need for computationally fast predictions for real-time decision-making in order to minimize any impact on the website’s usability. Rather than develop a single highly complex model, we used an ensemble of three simple models: Naive Bayes, Markov Chain, and Vowpal Wabbit. The ensemble generated significant lift over random predictions and demo...
ML models are increasingly deployed in settings with real world interactions such as vehicles, bu... more ML models are increasingly deployed in settings with real world interactions such as vehicles, but unfortunately, these models can fail in systematic ways. To prevent errors, ML engineering teams monitor and continuously improve these models. We propose a new abstraction, model assertions, that adapts the classical use of program assertions as a way to monitor and improve ML models. Model assertions are arbitrary functions over a model's input and output that indicate when errors may be occurring, e.g., a function that triggers if an object rapidly changes its class in a video. We propose methods of using model assertions at all stages of ML system deployment, including runtime monitoring, validating labels, and continuously improving ML models. For runtime monitoring, we show that model assertions can find high confidence errors, where a model returns the wrong output with high confidence, which uncertainty-based monitoring techniques would not detect. For training, we propose two methods of using model assertions. First, we propose a bandit-based active learning algorithm that can sample from data flagged by assertions and show that it can reduce labeling costs by up to 40% over traditional uncertainty-based methods. Second, we propose an API for generating "consistency assertions" (e.g., the class change example) and weak labels for inputs where the consistency assertions fail, and show that these weak labels can improve relative model quality by up to 46%. We evaluate model assertions on four real-world tasks with video, LIDAR, and ECG data.
Systems for performing ML inference are widely deployed today. However, they typically use techni... more Systems for performing ML inference are widely deployed today. However, they typically use techniques designed for conventional data serving workloads, missing critical opportunities to leverage the statistical nature of ML inference. In this paper, we present Willump, an optimizer for ML inference that introduces two statistically-motivated optimizations targeting ML applications whose performance bottleneck is feature computation. First, Willump automatically cascades feature computation. Willump classifies most data inputs using only high-value, low-cost features selected by a dataflow analysis algorithm and cost model, improving performance by up to 5× without statistically significant accuracy loss. Second, Willump accurately approximates ML top-K queries, discarding low-scoring inputs with an automatically constructed approximate model then ranking the remainder with a more powerful model, improving performance by up to 10× with minimal accuracy loss. Both optimizations automa...
Researchers have proposed a range of model compression techniques to reduce the computational and... more Researchers have proposed a range of model compression techniques to reduce the computational and memory footprint of deep neural networks (DNNs). In this work, we introduce Learned Intermediate representation Training (LIT), a novel model compression technique that outperforms a range of recent model compression techniques by leveraging the highly repetitive structure of modern DNNs (e.g., ResNet). LIT uses a teacher DNN to train a student DNN of reduced depth by leveraging two key ideas: 1) LIT directly compares intermediate representations of the teacher and student model and 2) LIT uses the intermediate representation from the teacher model’s previous block as input to the current student block during training, improving stability of intermediate representations in the student network. We show that LIT can substantially reduce network size without loss in accuracy on a range of DNN architectures and datasets. For example, LIT can compress ResNet on CIFAR10 by 3.4× outperforming ...
Model Specialization for Inference Via End-to-End Distillation, Pruning, and Cascades
The availability of general-purpose reference and benchmark datasets such as ImageNet have spurre... more The availability of general-purpose reference and benchmark datasets such as ImageNet have spurred the development of general-purpose popular reference model architectures and pre-trained weights. However, in practice, neural net- works are often employed to perform specific, more restrictive tasks, that are narrower in scope and complexity. Thus, simply fine-tuning or transfer learn- ing from a general-purpose network inherits a large computational cost that may not be necessary for a given task. In this work, we investigate the potential for model specialization, or reducing a model’s computational footprint by leverag- ing task-specific knowledge, such as a restricted inference distribution. We study three methods for model specialization—1) task-aware distillation, 2) task-aware pruning, and 3) specialized model cascades—and evaluate their performance on a range of classification tasks. Moreover, for the first time, we investigate how these techniques complement one another, ena...
Today's robotic systems are increasingly turning to computationally expensive models such as deep... more Today's robotic systems are increasingly turning to computationally expensive models such as deep neural networks (DNNs) for tasks like localization, perception, planning, and object detection. However, resource-constrained robots, like lowpower drones, often have insufficient on-board compute resources or power reserves to scalably run the most accurate, state-ofthe art neural network compute models. Cloud robotics allows mobile robots the benefit of offloading compute to centralized servers if they are uncertain locally or want to run more accurate, compute-intensive models. However, cloud robotics comes with a key, often understated cost: communicating with the cloud over congested wireless networks may result in latency or loss of data. In fact, sending high data-rate video or LIDAR from multiple robots over congested networks can lead to prohibitive delay for real-time applications, which we measure experimentally. In this paper, we formulate a novel Robot Offloading Problem-how and when should robots offload sensing tasks, especially if they are uncertain, to improve accuracy while minimizing the cost of cloud communication? We formulate offloading as a sequential decision making problem for robots, and propose a solution using deep reinforcement learning. In both simulations and hardware experiments using state-of-the art vision DNNs, our offloading strategy improves vision task performance by between 1.3-2.6x of benchmark offloading strategies, allowing robots the potential to significantly transcend their on-board sensing accuracy but with limited cost of cloud communication. Offload Logic Robot Model Cloud Model Mobile Robot Limited Network Sensory Input Cloud Offload Compute Local Compute Image, Map Databases Query the cloud for better accuracy? Latency vs. Accuracy vs. Power …
Researchers and industry analysts are increasingly interested in computing aggregation queries ov... more Researchers and industry analysts are increasingly interested in computing aggregation queries over large, unstructured datasets with selective predicates that are computed using expensive deep neural networks (DNNs). As these DNNs are expensive and because many applications can tolerate approximate answers, analysts are interested in accelerating these queries via approximations. Unfortunately, standard approximate query processing techniques to accelerate such queries are not applicable because they assume the result of the predicates are available ahead of time. Furthermore, recent work using cheap approximations (i.e., proxies) do not support aggregation queries with predicates. To accelerate aggregation queries with expensive predicates, we develop and analyze a query processing algorithm that leverages proxies (ABAE). ABAE must account for the key challenge that it may sample records that do not satisfy the predicate. To address this challenge, we first use the proxy to group ...
While deep neural networks (DNNs) are an increasingly popular way to query large corpora of data,... more While deep neural networks (DNNs) are an increasingly popular way to query large corpora of data, their significant runtime remains an active area of research. As a result, researchers have proposed systems and optimizations to reduce these costs by allowing users to trade off accuracy and speed. In this work, we examine end-to-end DNN execution in visual analytics systems on modern accelerators. Through a novel measurement study, we show that the preprocessing of data (e.g., decoding, resizing) can be the bottleneck in many visual analytics systems on modern hardware. To address the bottleneck of preprocessing, we introduce two optimizations for end-to-end visual analytics systems. First, we introduce novel methods of achieving accuracy and throughput trade-offs by using natively present, low-resolution visual data. Second, we develop a runtime engine for efficient visual DNN inference. This runtime engine a) efficiently pipelines preprocessing and DNN execution for inference, b) p...
Microchannel surfaces are common to microfluidics, biofluidics, thermal management, and energy ap... more Microchannel surfaces are common to microfluidics, biofluidics, thermal management, and energy applications. Due to processing limitations for the majority of metallic materials, the majority of hyperfine microchannels used in microfluidics and thermo-fluids are fabricated on non-metallic substrates, for example, silicon and polydimethylsiloxane. Here, a technique to fabricate ultrasmall microchannels on arbitrary metallic materials is developed using photolithography in combination with electrochemical deposition. The technique is used to prepare copper microchannels and to investigate the pool boiling heat transfer performance with a focus on the three-phase contact line dynamics. The hydrodynamics of nucleating bubbles during boiling are observed in situ using in-liquid endoscopy. The results show that the variation of critical heat flux enhancement has a linear relationship with the contact line increase ratio. The scalable microchannel surfaces exhibit superior heat transfer performance with a maximum heat transfer coefficient enhancement of 930% with ultra-low wall superheat of 5 °C. This work not only develops a scalable manufacturing method to develop ultra-small microchannels on metallic materials, it outlines design guidelines for structure optimization of pool boiling heat transfer for temperature sensitive applications, such as electronics thermal management.
Background: Value-based care models call for better interpretation of patient-reported outcomes. ... more Background: Value-based care models call for better interpretation of patient-reported outcomes. Patients may reference health status differently when appraising if an intervention was successful versus if their current state is acceptable. The purpose of this study was to determine the association between success of treatment (SOT), patient acceptable symptom state (PASS), and PROMIS measure T-scores, following a single primary care physical therapy encounter. Methods: Cross-sectional study. Ninety-two patients with musculoskeletal complaints were administered standard SOT and PASS questions, and PROMIS Physical Function, Pain Interference, and Self-Efficacy for Managing Symptoms measures. Association of PASS and SOT was determined using rank biserial correlation and chi-squared analysis. Accuracy of PROMIS T-scores to discriminate PASS and SOT was determined through receiver-operator curve analysis and likelihood ratios. Results: There was significant association between PASS and SOT (r=.393, p<0.001; X 2 =15.7, p=0.001). The three PROMIS measures discriminated PASS with AUCs of 0.73 to 0.88 (accuracy 67.3% to 82.6%), Self-Efficacy being the strongest discriminator. Only Pain Interference T-scores discriminated SOT with AUC >0.70 (accuracy 76.1%). Conclusion: PASS was more strongly associated with health status than SOT. Patients make a meaningful distinction between these two questions. Accurate clinician interpretation of PASS, SOT, and PROMIS T-scores can allow more targeted goal setting and treatment decision making.
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020
Neural language models are usually trained to match the distributional properties of largescale c... more Neural language models are usually trained to match the distributional properties of largescale corpora by minimizing the log loss. While straightforward to optimize, this approach forces the model to reproduce all variations in the dataset, including noisy and invalid references (e.g., misannotations and hallucinated facts). Even a small fraction of noisy data can degrade the performance of log loss. As an alternative, prior work has shown that minimizing the distinguishability of generated samples is a principled and robust loss that can handle invalid references. However, distinguishability has not been used in practice due to challenges in optimization and estimation. We propose loss truncation: a simple and scalable procedure which adaptively removes high log loss examples as a way to optimize for distinguishability. Empirically, we demonstrate that loss truncation outperforms existing baselines on distinguishability on a summarization task. Furthermore, we show that samples generated by the loss truncation model have factual accuracy ratings that exceed those of baselines and match human references.
Recent advances in neural networks (NNs) have enabled automatic querying of large volumes of vide... more Recent advances in neural networks (NNs) have enabled automatic querying of large volumes of video data with high accuracy. While these deep NNs can produce accurate annotations of an object's position and type in video, they are computationally expensive and require complex, imperative deployment code to answer queries. Prior work uses approximate filtering to reduce the cost of video analytics, but does not handle two important classes of queries, aggregation and limit queries; moreover, these approaches still require complex code to deploy. To address the computational and usability challenges of querying video at scale, we introduce BlazeIt, a system that optimizes queries of spatiotemporal information of objects in video. BlazeIt accepts queries via FrameQL, a declarative extension of SQL for video analytics that enables video-specific query optimization. We introduce two new query optimization techniques in BlazeIt that are not supported by prior work. First, we develop me...
Researchers have proposed hardware, software, and algorithmic optimizations to improve the comput... more Researchers have proposed hardware, software, and algorithmic optimizations to improve the computational performance of deep learning. While some of these optimizations perform the same operations faster (e.g., increasing GPU clock speed), many others modify the semantics of the training procedure (e.g., reduced precision), and can impact the final model's accuracy on unseen data. Due to a lack of standard evaluation criteria that considers these trade-offs, it is difficult to directly compare these optimizations. To address this problem, we recently introduced DAWNBENCH, a benchmark competition focused on end-to-end training time to achieve near-state-of-the-art accuracy on an unseen dataset-a combined metric called time-to-accuracy (TTA). In this work, we analyze the entries from DAWNBENCH, which received optimized submissions from multiple industrial groups, to investigate the behavior of TTA as a metric as well as trends in the best-performing entries. We show that TTA has a...
To provide an analysis of closed malpractice claims brought against anesthesiologists for positio... more To provide an analysis of closed malpractice claims brought against anesthesiologists for positioning-related perioperative nerve injury (PRPNI). Design: In this retrospective study, we analyzed closed claims data from the Controlled Risk Insurance Company (CRICO) Comparative Benchmarking System (CBS) database. Setting: Closed claims involving nerve injuries that occurred between January 1, 1996 and December 31, 2015 in all surgical settings, provided the alleged damaging event occurred under general anesthesia. Patients: Patient ages ranged from 23 to 94. Patients underwent a variety of surgical procedures. Severity of nerve injury ranged from "Insignificant" to "Grave" according to the NAIC Severity of Injury Code. Interventions: None. Measurements: Patient age and gender, alleged nerve injury type and severity, operating surgical specialty, contributing factors to the alleged nerve injury, and case outcome. Some of these data were drawn directly from coded variables in the CBS database, and some were gathered by the authors from CRICO-encoded narrative case summaries. Main results: Seventy-five claims were determined to represent PRPNI. Ninety-two percent of all PRPNI claims involved practitioner technical knowledge/performance. Of all the recorded nerve injuries in this series, 56% involved brachial plexus injuries, and supine patient positioning represented 55% of brachial plexus claims. Settlement payments were made in 33% of claims, and the average payment for all cases was $46,269. Twentyfour percent of PRPNI claims were found to be temporary, while 76% were permanent. Conclusions: PRPNI is multifactorial, and stems both from practitioner errors as well as from patient comorbidities and pre-existing neuropathies. Supine positioning can increase PRPNI risk. There are likely still causes of PRPNI of which we are not yet aware, given that despite concerted efforts towards positioning and padding interventions, injuries such as those described in this study still occur.
We characterize how genomic variants that alter chromatin accessibility influence regulatory fact... more We characterize how genomic variants that alter chromatin accessibility influence regulatory factor binding with a new method called DeltaBind that predicts condition specific factor binding more accurately than other methods based on DNase-seq data. Using DeltaBind and DNase-seq experiments we predicted the differential binding of 18 factors in K562 and GM12878 cells with an average precision of 28% at 10% recall, with the prediction of individual factors ranging from 5% to 65% precision. We further found that genome variants that alter chromatin accessibility are not necessarily predictive of altering proximal factor binding. Taken together these findings suggest that DNase-seq or ATAC-seq Quantitative Trait Loci (dsQTLs), while important, must be considered in a broader context to establish causality for phenotypic changes.
Recent advances in computer vision---in the form of deep neural networks---have made it possible ... more Recent advances in computer vision---in the form of deep neural networks---have made it possible to query increasing volumes of video data with high accuracy. However, neural network inference is computationally expensive at scale: applying a state-of-the-art object detector in real time (i.e., 30+ frames per second) to a single video requires a $4000 GPU. In response, we present N o S cope , a system for querying videos that can reduce the cost of neural network video analysis by up to three orders of magnitude via inference-optimized model search. Given a target video, object to detect, and reference neural network, N o S cope automatically searches for and trains a sequence, or cascade, of models that preserves the accuracy of the reference network but is specialized to the target video and are therefore far less computationally expensive. N o S cope cascades two types of models: specialized models that forego the full generality of the reference model but faithfully mimic its be...
Outcomes Following Complex Combat-Related Lumbosacral Dissociation Injuries
Global Spine Journal, 2015
Introduction As war injury patterns have changed throughout Operations Iraqi and Enduring Freedom... more Introduction As war injury patterns have changed throughout Operations Iraqi and Enduring Freedom (OIF/OEF), a relative increase in the incidence of complex lumbosacral dissociation (LSD) injuries has been noted. LSD injuries are an anatomic separation of the spinal column from the pelvis, and represent a manifestation of severe, high energy trauma. We assessed the clinical outcomes of combat-related LSD injuries after operative treatment. Methods We performed a retrospective review of all patients surgically managed for complex LSD injuries since the beginning of current combat operations in the Middle East. Results A total of 20 patients met inclusion criteria and were treated as follows: posterior spinal fusion (12), sacroiliac screw fixation (7), and combined anterior–posterior fusion (1). The mean age was 28.2 years. The most common mechanism of injury was mounted improvised explosive device (IED, 50%). On average, 2.2 spinal regions were injured per patient. Neurologic dysfunction was present in four patients. Three patients underwent operative stabilization of their injuries before evacuation to the United States. Median time to surgery from injury was 12 days (range, 0–111 days). There was a 20% wound infection rate. Median follow-up was 45.5 months (range, 23.2–105.3 months), and one-third of all patients were medically retired because of their injuries. At most recent follow-up, 45% complained of chronic low back pain and 35% had persistent neurologic deficits; however, 30% were actively engaged in strenuous physical activities, including swimming, distance running, and training for special forces. Conclusion This is the largest series of operatively managed lumbosacral dissociation patients currently reported. Our series suggests that combat-related lumbosacral dissociation injuries frequently result in persistent, long-term neurologic dysfunction, disability, and chronic pain. Operative management carries a high postoperative risk of infection. However, two-thirds of patients were able to remain on active duty after their injuries, and two patients were able to complete marathon training.
Due to the falling costs of data acquisition and storage, researchers and industry analysts often... more Due to the falling costs of data acquisition and storage, researchers and industry analysts often want to find all instances of rare events in large datasets. For instance, scientists can cheaply capture thousands of hours of video, but are limited by the need to manually inspect long videos to identify relevant objects and events. To reduce this cost, recent work proposes to use cheap proxy models, such as image classifiers, to identify an approximate set of data points satisfying a data selection filter. Unfortunately, this recent work does not provide the statistical accuracy guarantees necessary in scientific and production settings. In this work, we introduce novel algorithms for approximate selection queries with statistical accuracy guarantees. Namely, given a limited number of exact identifications from an oracle, often a human or an expensive machine learning model, our algorithms meet a minimum precision or recall target with high probability. In contrast, existing approaches can catastrophically fail in satisfying these recall and precision targets. We show that our algorithms can improve query result quality by up to 30× for both the precision and recall targets in both real and synthetic datasets.
Predictive Models for Determining If and When to Display Online Lead Forms
Proceedings of the AAAI Conference on Artificial Intelligence
This paper will demonstrate a machine learning application for predicting positive lead conversio... more This paper will demonstrate a machine learning application for predicting positive lead conversion events on the Edmunds.com website, an American destination for car shopping. A positive conversion event occurs when a user fills out and submits a lead form interstitial. We used machine learning to identify which users might want to fill out lead forms, and where in their sessions to present the interstitials. There are several factors that make these predictions difficult, such as (a) far more negative than positive responses (b) seasonality effects due to car sales events near holidays, which require the model to be easily tunable and (c) the need for computationally fast predictions for real-time decision-making in order to minimize any impact on the website’s usability. Rather than develop a single highly complex model, we used an ensemble of three simple models: Naive Bayes, Markov Chain, and Vowpal Wabbit. The ensemble generated significant lift over random predictions and demo...
ML models are increasingly deployed in settings with real world interactions such as vehicles, bu... more ML models are increasingly deployed in settings with real world interactions such as vehicles, but unfortunately, these models can fail in systematic ways. To prevent errors, ML engineering teams monitor and continuously improve these models. We propose a new abstraction, model assertions, that adapts the classical use of program assertions as a way to monitor and improve ML models. Model assertions are arbitrary functions over a model's input and output that indicate when errors may be occurring, e.g., a function that triggers if an object rapidly changes its class in a video. We propose methods of using model assertions at all stages of ML system deployment, including runtime monitoring, validating labels, and continuously improving ML models. For runtime monitoring, we show that model assertions can find high confidence errors, where a model returns the wrong output with high confidence, which uncertainty-based monitoring techniques would not detect. For training, we propose two methods of using model assertions. First, we propose a bandit-based active learning algorithm that can sample from data flagged by assertions and show that it can reduce labeling costs by up to 40% over traditional uncertainty-based methods. Second, we propose an API for generating "consistency assertions" (e.g., the class change example) and weak labels for inputs where the consistency assertions fail, and show that these weak labels can improve relative model quality by up to 46%. We evaluate model assertions on four real-world tasks with video, LIDAR, and ECG data.
Systems for performing ML inference are widely deployed today. However, they typically use techni... more Systems for performing ML inference are widely deployed today. However, they typically use techniques designed for conventional data serving workloads, missing critical opportunities to leverage the statistical nature of ML inference. In this paper, we present Willump, an optimizer for ML inference that introduces two statistically-motivated optimizations targeting ML applications whose performance bottleneck is feature computation. First, Willump automatically cascades feature computation. Willump classifies most data inputs using only high-value, low-cost features selected by a dataflow analysis algorithm and cost model, improving performance by up to 5× without statistically significant accuracy loss. Second, Willump accurately approximates ML top-K queries, discarding low-scoring inputs with an automatically constructed approximate model then ranking the remainder with a more powerful model, improving performance by up to 10× with minimal accuracy loss. Both optimizations automa...
Researchers have proposed a range of model compression techniques to reduce the computational and... more Researchers have proposed a range of model compression techniques to reduce the computational and memory footprint of deep neural networks (DNNs). In this work, we introduce Learned Intermediate representation Training (LIT), a novel model compression technique that outperforms a range of recent model compression techniques by leveraging the highly repetitive structure of modern DNNs (e.g., ResNet). LIT uses a teacher DNN to train a student DNN of reduced depth by leveraging two key ideas: 1) LIT directly compares intermediate representations of the teacher and student model and 2) LIT uses the intermediate representation from the teacher model’s previous block as input to the current student block during training, improving stability of intermediate representations in the student network. We show that LIT can substantially reduce network size without loss in accuracy on a range of DNN architectures and datasets. For example, LIT can compress ResNet on CIFAR10 by 3.4× outperforming ...
Model Specialization for Inference Via End-to-End Distillation, Pruning, and Cascades
The availability of general-purpose reference and benchmark datasets such as ImageNet have spurre... more The availability of general-purpose reference and benchmark datasets such as ImageNet have spurred the development of general-purpose popular reference model architectures and pre-trained weights. However, in practice, neural net- works are often employed to perform specific, more restrictive tasks, that are narrower in scope and complexity. Thus, simply fine-tuning or transfer learn- ing from a general-purpose network inherits a large computational cost that may not be necessary for a given task. In this work, we investigate the potential for model specialization, or reducing a model’s computational footprint by leverag- ing task-specific knowledge, such as a restricted inference distribution. We study three methods for model specialization—1) task-aware distillation, 2) task-aware pruning, and 3) specialized model cascades—and evaluate their performance on a range of classification tasks. Moreover, for the first time, we investigate how these techniques complement one another, ena...
Today's robotic systems are increasingly turning to computationally expensive models such as deep... more Today's robotic systems are increasingly turning to computationally expensive models such as deep neural networks (DNNs) for tasks like localization, perception, planning, and object detection. However, resource-constrained robots, like lowpower drones, often have insufficient on-board compute resources or power reserves to scalably run the most accurate, state-ofthe art neural network compute models. Cloud robotics allows mobile robots the benefit of offloading compute to centralized servers if they are uncertain locally or want to run more accurate, compute-intensive models. However, cloud robotics comes with a key, often understated cost: communicating with the cloud over congested wireless networks may result in latency or loss of data. In fact, sending high data-rate video or LIDAR from multiple robots over congested networks can lead to prohibitive delay for real-time applications, which we measure experimentally. In this paper, we formulate a novel Robot Offloading Problem-how and when should robots offload sensing tasks, especially if they are uncertain, to improve accuracy while minimizing the cost of cloud communication? We formulate offloading as a sequential decision making problem for robots, and propose a solution using deep reinforcement learning. In both simulations and hardware experiments using state-of-the art vision DNNs, our offloading strategy improves vision task performance by between 1.3-2.6x of benchmark offloading strategies, allowing robots the potential to significantly transcend their on-board sensing accuracy but with limited cost of cloud communication. Offload Logic Robot Model Cloud Model Mobile Robot Limited Network Sensory Input Cloud Offload Compute Local Compute Image, Map Databases Query the cloud for better accuracy? Latency vs. Accuracy vs. Power …
Researchers and industry analysts are increasingly interested in computing aggregation queries ov... more Researchers and industry analysts are increasingly interested in computing aggregation queries over large, unstructured datasets with selective predicates that are computed using expensive deep neural networks (DNNs). As these DNNs are expensive and because many applications can tolerate approximate answers, analysts are interested in accelerating these queries via approximations. Unfortunately, standard approximate query processing techniques to accelerate such queries are not applicable because they assume the result of the predicates are available ahead of time. Furthermore, recent work using cheap approximations (i.e., proxies) do not support aggregation queries with predicates. To accelerate aggregation queries with expensive predicates, we develop and analyze a query processing algorithm that leverages proxies (ABAE). ABAE must account for the key challenge that it may sample records that do not satisfy the predicate. To address this challenge, we first use the proxy to group ...
While deep neural networks (DNNs) are an increasingly popular way to query large corpora of data,... more While deep neural networks (DNNs) are an increasingly popular way to query large corpora of data, their significant runtime remains an active area of research. As a result, researchers have proposed systems and optimizations to reduce these costs by allowing users to trade off accuracy and speed. In this work, we examine end-to-end DNN execution in visual analytics systems on modern accelerators. Through a novel measurement study, we show that the preprocessing of data (e.g., decoding, resizing) can be the bottleneck in many visual analytics systems on modern hardware. To address the bottleneck of preprocessing, we introduce two optimizations for end-to-end visual analytics systems. First, we introduce novel methods of achieving accuracy and throughput trade-offs by using natively present, low-resolution visual data. Second, we develop a runtime engine for efficient visual DNN inference. This runtime engine a) efficiently pipelines preprocessing and DNN execution for inference, b) p...
Microchannel surfaces are common to microfluidics, biofluidics, thermal management, and energy ap... more Microchannel surfaces are common to microfluidics, biofluidics, thermal management, and energy applications. Due to processing limitations for the majority of metallic materials, the majority of hyperfine microchannels used in microfluidics and thermo-fluids are fabricated on non-metallic substrates, for example, silicon and polydimethylsiloxane. Here, a technique to fabricate ultrasmall microchannels on arbitrary metallic materials is developed using photolithography in combination with electrochemical deposition. The technique is used to prepare copper microchannels and to investigate the pool boiling heat transfer performance with a focus on the three-phase contact line dynamics. The hydrodynamics of nucleating bubbles during boiling are observed in situ using in-liquid endoscopy. The results show that the variation of critical heat flux enhancement has a linear relationship with the contact line increase ratio. The scalable microchannel surfaces exhibit superior heat transfer performance with a maximum heat transfer coefficient enhancement of 930% with ultra-low wall superheat of 5 °C. This work not only develops a scalable manufacturing method to develop ultra-small microchannels on metallic materials, it outlines design guidelines for structure optimization of pool boiling heat transfer for temperature sensitive applications, such as electronics thermal management.
Background: Value-based care models call for better interpretation of patient-reported outcomes. ... more Background: Value-based care models call for better interpretation of patient-reported outcomes. Patients may reference health status differently when appraising if an intervention was successful versus if their current state is acceptable. The purpose of this study was to determine the association between success of treatment (SOT), patient acceptable symptom state (PASS), and PROMIS measure T-scores, following a single primary care physical therapy encounter. Methods: Cross-sectional study. Ninety-two patients with musculoskeletal complaints were administered standard SOT and PASS questions, and PROMIS Physical Function, Pain Interference, and Self-Efficacy for Managing Symptoms measures. Association of PASS and SOT was determined using rank biserial correlation and chi-squared analysis. Accuracy of PROMIS T-scores to discriminate PASS and SOT was determined through receiver-operator curve analysis and likelihood ratios. Results: There was significant association between PASS and SOT (r=.393, p<0.001; X 2 =15.7, p=0.001). The three PROMIS measures discriminated PASS with AUCs of 0.73 to 0.88 (accuracy 67.3% to 82.6%), Self-Efficacy being the strongest discriminator. Only Pain Interference T-scores discriminated SOT with AUC >0.70 (accuracy 76.1%). Conclusion: PASS was more strongly associated with health status than SOT. Patients make a meaningful distinction between these two questions. Accurate clinician interpretation of PASS, SOT, and PROMIS T-scores can allow more targeted goal setting and treatment decision making.
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020
Neural language models are usually trained to match the distributional properties of largescale c... more Neural language models are usually trained to match the distributional properties of largescale corpora by minimizing the log loss. While straightforward to optimize, this approach forces the model to reproduce all variations in the dataset, including noisy and invalid references (e.g., misannotations and hallucinated facts). Even a small fraction of noisy data can degrade the performance of log loss. As an alternative, prior work has shown that minimizing the distinguishability of generated samples is a principled and robust loss that can handle invalid references. However, distinguishability has not been used in practice due to challenges in optimization and estimation. We propose loss truncation: a simple and scalable procedure which adaptively removes high log loss examples as a way to optimize for distinguishability. Empirically, we demonstrate that loss truncation outperforms existing baselines on distinguishability on a summarization task. Furthermore, we show that samples generated by the loss truncation model have factual accuracy ratings that exceed those of baselines and match human references.
Recent advances in neural networks (NNs) have enabled automatic querying of large volumes of vide... more Recent advances in neural networks (NNs) have enabled automatic querying of large volumes of video data with high accuracy. While these deep NNs can produce accurate annotations of an object's position and type in video, they are computationally expensive and require complex, imperative deployment code to answer queries. Prior work uses approximate filtering to reduce the cost of video analytics, but does not handle two important classes of queries, aggregation and limit queries; moreover, these approaches still require complex code to deploy. To address the computational and usability challenges of querying video at scale, we introduce BlazeIt, a system that optimizes queries of spatiotemporal information of objects in video. BlazeIt accepts queries via FrameQL, a declarative extension of SQL for video analytics that enables video-specific query optimization. We introduce two new query optimization techniques in BlazeIt that are not supported by prior work. First, we develop me...
Researchers have proposed hardware, software, and algorithmic optimizations to improve the comput... more Researchers have proposed hardware, software, and algorithmic optimizations to improve the computational performance of deep learning. While some of these optimizations perform the same operations faster (e.g., increasing GPU clock speed), many others modify the semantics of the training procedure (e.g., reduced precision), and can impact the final model's accuracy on unseen data. Due to a lack of standard evaluation criteria that considers these trade-offs, it is difficult to directly compare these optimizations. To address this problem, we recently introduced DAWNBENCH, a benchmark competition focused on end-to-end training time to achieve near-state-of-the-art accuracy on an unseen dataset-a combined metric called time-to-accuracy (TTA). In this work, we analyze the entries from DAWNBENCH, which received optimized submissions from multiple industrial groups, to investigate the behavior of TTA as a metric as well as trends in the best-performing entries. We show that TTA has a...
To provide an analysis of closed malpractice claims brought against anesthesiologists for positio... more To provide an analysis of closed malpractice claims brought against anesthesiologists for positioning-related perioperative nerve injury (PRPNI). Design: In this retrospective study, we analyzed closed claims data from the Controlled Risk Insurance Company (CRICO) Comparative Benchmarking System (CBS) database. Setting: Closed claims involving nerve injuries that occurred between January 1, 1996 and December 31, 2015 in all surgical settings, provided the alleged damaging event occurred under general anesthesia. Patients: Patient ages ranged from 23 to 94. Patients underwent a variety of surgical procedures. Severity of nerve injury ranged from "Insignificant" to "Grave" according to the NAIC Severity of Injury Code. Interventions: None. Measurements: Patient age and gender, alleged nerve injury type and severity, operating surgical specialty, contributing factors to the alleged nerve injury, and case outcome. Some of these data were drawn directly from coded variables in the CBS database, and some were gathered by the authors from CRICO-encoded narrative case summaries. Main results: Seventy-five claims were determined to represent PRPNI. Ninety-two percent of all PRPNI claims involved practitioner technical knowledge/performance. Of all the recorded nerve injuries in this series, 56% involved brachial plexus injuries, and supine patient positioning represented 55% of brachial plexus claims. Settlement payments were made in 33% of claims, and the average payment for all cases was $46,269. Twentyfour percent of PRPNI claims were found to be temporary, while 76% were permanent. Conclusions: PRPNI is multifactorial, and stems both from practitioner errors as well as from patient comorbidities and pre-existing neuropathies. Supine positioning can increase PRPNI risk. There are likely still causes of PRPNI of which we are not yet aware, given that despite concerted efforts towards positioning and padding interventions, injuries such as those described in this study still occur.
We characterize how genomic variants that alter chromatin accessibility influence regulatory fact... more We characterize how genomic variants that alter chromatin accessibility influence regulatory factor binding with a new method called DeltaBind that predicts condition specific factor binding more accurately than other methods based on DNase-seq data. Using DeltaBind and DNase-seq experiments we predicted the differential binding of 18 factors in K562 and GM12878 cells with an average precision of 28% at 10% recall, with the prediction of individual factors ranging from 5% to 65% precision. We further found that genome variants that alter chromatin accessibility are not necessarily predictive of altering proximal factor binding. Taken together these findings suggest that DNase-seq or ATAC-seq Quantitative Trait Loci (dsQTLs), while important, must be considered in a broader context to establish causality for phenotypic changes.
Recent advances in computer vision---in the form of deep neural networks---have made it possible ... more Recent advances in computer vision---in the form of deep neural networks---have made it possible to query increasing volumes of video data with high accuracy. However, neural network inference is computationally expensive at scale: applying a state-of-the-art object detector in real time (i.e., 30+ frames per second) to a single video requires a $4000 GPU. In response, we present N o S cope , a system for querying videos that can reduce the cost of neural network video analysis by up to three orders of magnitude via inference-optimized model search. Given a target video, object to detect, and reference neural network, N o S cope automatically searches for and trains a sequence, or cascade, of models that preserves the accuracy of the reference network but is specialized to the target video and are therefore far less computationally expensive. N o S cope cascades two types of models: specialized models that forego the full generality of the reference model but faithfully mimic its be...
Outcomes Following Complex Combat-Related Lumbosacral Dissociation Injuries
Global Spine Journal, 2015
Introduction As war injury patterns have changed throughout Operations Iraqi and Enduring Freedom... more Introduction As war injury patterns have changed throughout Operations Iraqi and Enduring Freedom (OIF/OEF), a relative increase in the incidence of complex lumbosacral dissociation (LSD) injuries has been noted. LSD injuries are an anatomic separation of the spinal column from the pelvis, and represent a manifestation of severe, high energy trauma. We assessed the clinical outcomes of combat-related LSD injuries after operative treatment. Methods We performed a retrospective review of all patients surgically managed for complex LSD injuries since the beginning of current combat operations in the Middle East. Results A total of 20 patients met inclusion criteria and were treated as follows: posterior spinal fusion (12), sacroiliac screw fixation (7), and combined anterior–posterior fusion (1). The mean age was 28.2 years. The most common mechanism of injury was mounted improvised explosive device (IED, 50%). On average, 2.2 spinal regions were injured per patient. Neurologic dysfunction was present in four patients. Three patients underwent operative stabilization of their injuries before evacuation to the United States. Median time to surgery from injury was 12 days (range, 0–111 days). There was a 20% wound infection rate. Median follow-up was 45.5 months (range, 23.2–105.3 months), and one-third of all patients were medically retired because of their injuries. At most recent follow-up, 45% complained of chronic low back pain and 35% had persistent neurologic deficits; however, 30% were actively engaged in strenuous physical activities, including swimming, distance running, and training for special forces. Conclusion This is the largest series of operatively managed lumbosacral dissociation patients currently reported. Our series suggests that combat-related lumbosacral dissociation injuries frequently result in persistent, long-term neurologic dysfunction, disability, and chronic pain. Operative management carries a high postoperative risk of infection. However, two-thirds of patients were able to remain on active duty after their injuries, and two patients were able to complete marathon training.
Uploads
Papers by daniel kang