Proceedings 2022 Network and Distributed System Security Symposium
Embedded and Internet-of-Things (IoT) devices have seen an increase in adoption in many domains. ... more Embedded and Internet-of-Things (IoT) devices have seen an increase in adoption in many domains. The security of these devices is of great importance as they are often used to control critical infrastructure, medical devices, and vehicles. Existing solutions to isolate microcontroller (MCU) resources in order to increase their security face significant challenges such as specific hardware unavailability, Memory Protection Unit (MPU) limitations and a significant lack of Direct Memory Access (DMA) support. Nevertheless, DMA is fundamental for the power and performance requirements of embedded applications. In this paper, we present D-Box, a systematic approach to enable secure DMA operations for compartmentalization solutions of embedded applications using real-time operating systems (RTOS). D-Box defines a reference architecture and a workflow to protect DMA operations holistically. It provides practical methods to harden the kernel and define capabilitybased security policies for easy definition of DMA operations with strong security properties. We implemented a D-Box prototype for the Cortex-M3/M4 on top of the popular FreeRTOS-MPU (F-MPU). The D-Box procedures and a stricter security model enabled DMA operations, yet it exposed 41 times less ROP (return-orienting-programming) gadgets when compared with the standard F-MPU. D-Box adds only a 2% processor overhead while reducing the power consumption of peripheral operation benchmarks by 18.2%. The security properties and performance of D-Box were tested and confirmed on a real-world case study of a Programmable Logic Controller (PLC) application.
Existing malware detectors on safety-critical devices have difficulties in runtime detection due ... more Existing malware detectors on safety-critical devices have difficulties in runtime detection due to the performance overhead. In this paper, we introduce PROPEDEUTICA 1 , a framework for efficient and effective real-time malware detection, leveraging the best of conventional machine learning (ML) and deep learning (DL) techniques. In PROPEDEUTICA, all software start execution are considered as benign and monitored by a conventional ML classifier for fast detection. If the software receives a borderline classification from the ML detector (e.g. the software is 50% likely to be benign and 50% likely to be malicious), the software will be transferred to a more accurate, yet performance demanding DL detector. To address spatial-temporal dynamics and software execution heterogeneity, we introduce a novel DL architecture (DEEPMALWARE) for PROPEDEUTICA with multistream inputs. We evaluated PROPEDEUTICA with 9,115 malware samples and 1,338 benign software from various categories for the Windows OS. With a borderline interval of [30%-70%], PROPEDEUTICA achieves an accuracy of 94.34% and a falsepositive rate of 8.75%, with 41.45% of the samples moved for DEEPMALWARE analysis. Even using only CPU, PROPEDEUTICA can detect malware within less than 0.1 seconds.
Programmable Logic Controllers (PLCs) play a critical role in the industrial control systems. Vul... more Programmable Logic Controllers (PLCs) play a critical role in the industrial control systems. Vulnerabilities in PLC programs might lead to attacks causing devastating consequences to the critical infrastructure, as shown in Stuxnet and similar attacks. In recent years, we have seen an exponential increase in vulnerabilities reported for PLC control logic. Looking back on past research, we found extensive studies explored control logic modification attacks, as well as formal verification-based security solutions. We performed systematization on these studies, and found attacks that can compromise a full chain of control and evade detection. However, the majority of the formal verification research investigated ad-hoc techniques targeting PLC programs. We discovered challenges in every aspect of formal verification, rising from (1) the ever-expanding attack surface from evolved system design, (2) the real-time constraint during the program execution, and (3) the barrier in security e...
The operating system is increasingly regarded as untrustworthy. Applications, hardware, and hyper... more The operating system is increasingly regarded as untrustworthy. Applications, hardware, and hypervisors are erecting defenses to insulate themselves from the operating system. This paper explores the potential benefits if operating systems simply embraced these lowered expectations and deliberately varied API behavior. We argue that, even for trusted or benign applications, diversity roughly within the specification can improve resilience to attack and improve robustness. Malicious software tends to be brittle; a preliminary case study indicates that, for software of questionable origin, a somewhat hostile operating system may do more good than harm for system security. This paper describes the architecture of Chameleon, an ongoing project to implement spectrum-behavior as an operating system feature.
On-device machine learning (ML) is quickly gaining popularity among mobile apps. It allows offlin... more On-device machine learning (ML) is quickly gaining popularity among mobile apps. It allows offline model inference while preserving user privacy. However, ML models, considered as core intellectual properties of model owners, are now stored on billions of untrusted devices and subject to potential thefts. Leaked models can cause both severe financial loss and security consequences. This paper presents the first empirical study of ML model protection on mobile devices. Our study aims to answer three open questions with quantitative evidence: How widely is model protection used in apps? How robust are existing model protection techniques? How much can (stolen) models cost? To that end, we built a simple app analysis pipeline and analyzed 46,753 popular apps collected from the US and Chinese app markets. We identified 1,468 ML apps spanning all popular app categories. We found that, alarmingly, 41% of ML apps do not protect their models at all, which can be trivially stolen from app pa...
A promising avenue for improving the effectiveness of behavioral-based malware detectors would be... more A promising avenue for improving the effectiveness of behavioral-based malware detectors would be to combine fast traditional machine learning detectors with high-accuracy, but time-consuming deep learning models. The main idea would be to place software receiving borderline classifications by traditional machine learning methods in an environment where uncertainty is added, while software is analyzed by more time-consuming deep learning models. The goal of uncertainty would be to rate-limit actions of potential malware during the time consuming deep analysis. In this paper, we present a detailed description of the analysis and implementation of CHAMELEON, a framework for realizing this uncertain environment for Linux. CHAMELEON offers two environments for software: (i) standard - for any software identified as benign by conventional machine learning methods and (ii) uncertain - for software receiving borderline classifications when analyzed by these conventional machine learning me...
This paper investigates whether computer usage profiles comprised of process-, network-, mouseand... more This paper investigates whether computer usage profiles comprised of process-, network-, mouseand keystroke-related events are unique and temporally consistent in a naturalistic setting, discussing challenges and opportunities of using such profiles in applications of continuous authentication. We collected ecologically-valid computer usage profiles from 28 MS Windows 10 computer users over 8 weeks and submitted this data to comprehensive machine learning analysis involving a diverse set of online and offline classifiers. We found that (i) computer usage profiles have the potential to uniquely characterize computer users (with a maximum F-score of 99, 94%); (ii) network-related events were the most useful features to properly recognize profiles (95.14% of the top features distinguishing users being network-related); (iii) user profiles were mostly inconsistent over the 8-week data collection period, with 92.86% of users exhibiting drifts in terms of time and usage habits; and (iv) o...
IEEE Transactions on Dependable and Secure Computing, 2020
A promising avenue for improving the effectiveness of behavioral-based malware detectors is to le... more A promising avenue for improving the effectiveness of behavioral-based malware detectors is to leverage two-phase detection mechanisms. Existing problem in two-phase detection is that after the first phase produces borderline decision, suspicious behaviors are not well contained before the second phase completes. This paper improves CHAMELEON, a framework to realize the uncertain environment. CHAMELEON offers two environments: standardfor software identified as benign by the first phase, and uncertainfor software received borderline classification from the first phase. The uncertain environment adds obstacles to software execution through random perturbations applied probabilistically. We introduce a dynamic perturbation threshold that can target malware disproportionately more than benign software. We analyzed the effects of the uncertain environment by manually studying 113 software and 100 malware, and found that 92% malware and 10% benign software disrupted during execution. The results were then corroborated by an extended dataset (5,679 Linux malware samples) on a newer system. Finally, a careful inspection of the benign software crashes revealed some software bugs, highlighting CHAMELEON's potential as a practical complementary antimalware solution.
2021 IEEE European Symposium on Security and Privacy (EuroS&P), 2021
Programmable Logic Controllers (PLCs) play a critical role in the industrial control systems. Vul... more Programmable Logic Controllers (PLCs) play a critical role in the industrial control systems. Vulnerabilities in PLC programs might lead to attacks causing devastating consequences to the critical infrastructure, as shown in Stuxnet and similar attacks. In recent years, we have seen an exponential increase in vulnerabilities reported for PLC control logic. Looking back on past research, we found extensive studies explored control logic modification attacks, as well as formal verification-based security solutions. We performed systematization on these studies, and found attacks that can compromise a full chain of control and evade detection. However, the majority of the formal verification research investigated ad-hoc techniques targeting PLC programs. We discovered challenges in every aspect of formal verification, rising from (1) the ever-expanding attack surface from evolved system design, (2) the real-time constraint during the program execution, and (3) the barrier in security evaluation given proprietary and vendor-specific dependencies on different techniques. Based on the knowledge systematization, we provide a set of recommendations for future research directions, and we highlight the need of defending security issues besides safety issues.
2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE), 2016
Applications are generally written assuming a predictable and well-behaved OS. In practice, they ... more Applications are generally written assuming a predictable and well-behaved OS. In practice, they experience unpredictable misbehavior at the OS level and across OSes: different OSes can handle network events differently, APIs can behave differently across OSes, and OSes may be compromised or buggy. This unpredictability is challenging because its sources typically manifest during deployment and are hard to reproduce. This paper introduces Bear, a framework for statistical analysis of application sensitivity to OS unpredictability that can help developers build more resilient software, discover challenging bugs and identify the scenarios that most need validation. Bear analyzes a program with a set of perturbation strategies on a set of commonly used system calls in order to discover the most sensitive system calls for each application, the most impactful strategies, and how they predict abnormal program outcome. We evaluated Bear with 113 CPU and IO-bound programs, and our results show that null memory dereferencing and erroneous buffer operations are the most impactful strategies for predicting abnormal program execution and that their impacts increase tenfold with workload increase (e.g. number of network requests from 10 to 1000). Generic system calls are more sensitive than specialized system calls-for example, write and sendto can both be used to send data through a socket, but the sensitivity of write is twice that of sendto. System calls with an array parameter (e.g. read) are more sensitive to perturbations than those having a struct parameter with a buffer (e.g readv). Moreover, the fewer parameters a system call has, the more sensitive it is.
As the malware research field became more established over the last two decades, new research que... more As the malware research field became more established over the last two decades, new research questions arose, such as how to make malware research reproducible, how to bring scientific rigor to attack papers, or what is an appropriate malware dataset for relevant experimental results. The challenges these questions pose also brings pitfalls that affect the multiple malware research stakeholders. To help answering those questions and to highlight potential research pitfalls to be avoided, in this paper, we present a systematic literature review of 491 papers on malware research published in major security conferences between 2000 and 2018. We identified the most common pitfalls present in past literature and propose a method for assessing current (and future) malware research. Our goal is towards integrating science and engineering best practices to develop further, improved research by learning from issues present in the published body of work. As far as we know, this is the largest literature review of its kind and the first to summarize research pitfalls in a research methodology that avoids them. In total, we discovered 20 pitfalls that limit current research impact and reproducibility. The identified pitfalls range from (i) the lack of a proper threat model, that complicates paper's evaluation, to (ii) the use of closed-source solutions and private datasets, that limit reproducibility. We also report yet-to-be-overcome challenges that are inherent to the malware nature, such as non-deterministic analysis results. Based on our findings, we propose a set of actions to be taken by the malware research and development community for future work: (i) Consolidation of malware research as a field constituted of diverse research approaches (e.g., engineering solutions, offensive research, landscapes/observational studies, and network traffic/system traces analysis); (ii) design of engineering solutions with clearer, direct assumptions (e.g., positioning solutions as proofs-of-concept vs. deployable); (iii) design of experiments that reflects (and emphasizes) the target scenario for the proposed solution (e.g., corporation, user, country-wide); (iv) clearer exposition and discussion of limitations of used technologies and exercised norms/standards for research (e.g., the use of closedsource antiviruses as ground-truth). Hypothesis Definition & Research Requirements Background Research Solution Requirements Solution Design Solution Development / Prototyping Research Objective Definition Engineering Method Common Core Experiment Design Test of Hypothesis / Evaluation of Solution Analysis of Results Results align with Hypothesis / Requirements? Communicate Results
On-device machine learning (ML) is quickly gaining popularity among mobile apps. It allows offlin... more On-device machine learning (ML) is quickly gaining popularity among mobile apps. It allows offline model inference while preserving user privacy. However, ML models, considered as core intellectual properties of model owners, are now stored on billions of untrusted devices and subject to potential thefts. Leaked models can cause both severe financial loss and security consequences. This paper presents the first empirical study of ML model protection on mobile devices. Our study aims to answer three open questions with quantitative evidence: How widely is model protection used in apps? How robust are existing model protection techniques? What impacts can (stolen) models incur? To that end, we built a simple app analysis pipeline and analyzed 46,753 popular apps collected from the US and Chinese app markets. We identified 1,468 ML apps spanning all popular app categories. We found that, alarmingly, 41% of ML apps do not protect their models at all, which can be trivially stolen from a...
Proceedings 2022 Network and Distributed System Security Symposium
Embedded and Internet-of-Things (IoT) devices have seen an increase in adoption in many domains. ... more Embedded and Internet-of-Things (IoT) devices have seen an increase in adoption in many domains. The security of these devices is of great importance as they are often used to control critical infrastructure, medical devices, and vehicles. Existing solutions to isolate microcontroller (MCU) resources in order to increase their security face significant challenges such as specific hardware unavailability, Memory Protection Unit (MPU) limitations and a significant lack of Direct Memory Access (DMA) support. Nevertheless, DMA is fundamental for the power and performance requirements of embedded applications. In this paper, we present D-Box, a systematic approach to enable secure DMA operations for compartmentalization solutions of embedded applications using real-time operating systems (RTOS). D-Box defines a reference architecture and a workflow to protect DMA operations holistically. It provides practical methods to harden the kernel and define capabilitybased security policies for easy definition of DMA operations with strong security properties. We implemented a D-Box prototype for the Cortex-M3/M4 on top of the popular FreeRTOS-MPU (F-MPU). The D-Box procedures and a stricter security model enabled DMA operations, yet it exposed 41 times less ROP (return-orienting-programming) gadgets when compared with the standard F-MPU. D-Box adds only a 2% processor overhead while reducing the power consumption of peripheral operation benchmarks by 18.2%. The security properties and performance of D-Box were tested and confirmed on a real-world case study of a Programmable Logic Controller (PLC) application.
Existing malware detectors on safety-critical devices have difficulties in runtime detection due ... more Existing malware detectors on safety-critical devices have difficulties in runtime detection due to the performance overhead. In this paper, we introduce PROPEDEUTICA 1 , a framework for efficient and effective real-time malware detection, leveraging the best of conventional machine learning (ML) and deep learning (DL) techniques. In PROPEDEUTICA, all software start execution are considered as benign and monitored by a conventional ML classifier for fast detection. If the software receives a borderline classification from the ML detector (e.g. the software is 50% likely to be benign and 50% likely to be malicious), the software will be transferred to a more accurate, yet performance demanding DL detector. To address spatial-temporal dynamics and software execution heterogeneity, we introduce a novel DL architecture (DEEPMALWARE) for PROPEDEUTICA with multistream inputs. We evaluated PROPEDEUTICA with 9,115 malware samples and 1,338 benign software from various categories for the Windows OS. With a borderline interval of [30%-70%], PROPEDEUTICA achieves an accuracy of 94.34% and a falsepositive rate of 8.75%, with 41.45% of the samples moved for DEEPMALWARE analysis. Even using only CPU, PROPEDEUTICA can detect malware within less than 0.1 seconds.
Programmable Logic Controllers (PLCs) play a critical role in the industrial control systems. Vul... more Programmable Logic Controllers (PLCs) play a critical role in the industrial control systems. Vulnerabilities in PLC programs might lead to attacks causing devastating consequences to the critical infrastructure, as shown in Stuxnet and similar attacks. In recent years, we have seen an exponential increase in vulnerabilities reported for PLC control logic. Looking back on past research, we found extensive studies explored control logic modification attacks, as well as formal verification-based security solutions. We performed systematization on these studies, and found attacks that can compromise a full chain of control and evade detection. However, the majority of the formal verification research investigated ad-hoc techniques targeting PLC programs. We discovered challenges in every aspect of formal verification, rising from (1) the ever-expanding attack surface from evolved system design, (2) the real-time constraint during the program execution, and (3) the barrier in security e...
The operating system is increasingly regarded as untrustworthy. Applications, hardware, and hyper... more The operating system is increasingly regarded as untrustworthy. Applications, hardware, and hypervisors are erecting defenses to insulate themselves from the operating system. This paper explores the potential benefits if operating systems simply embraced these lowered expectations and deliberately varied API behavior. We argue that, even for trusted or benign applications, diversity roughly within the specification can improve resilience to attack and improve robustness. Malicious software tends to be brittle; a preliminary case study indicates that, for software of questionable origin, a somewhat hostile operating system may do more good than harm for system security. This paper describes the architecture of Chameleon, an ongoing project to implement spectrum-behavior as an operating system feature.
On-device machine learning (ML) is quickly gaining popularity among mobile apps. It allows offlin... more On-device machine learning (ML) is quickly gaining popularity among mobile apps. It allows offline model inference while preserving user privacy. However, ML models, considered as core intellectual properties of model owners, are now stored on billions of untrusted devices and subject to potential thefts. Leaked models can cause both severe financial loss and security consequences. This paper presents the first empirical study of ML model protection on mobile devices. Our study aims to answer three open questions with quantitative evidence: How widely is model protection used in apps? How robust are existing model protection techniques? How much can (stolen) models cost? To that end, we built a simple app analysis pipeline and analyzed 46,753 popular apps collected from the US and Chinese app markets. We identified 1,468 ML apps spanning all popular app categories. We found that, alarmingly, 41% of ML apps do not protect their models at all, which can be trivially stolen from app pa...
A promising avenue for improving the effectiveness of behavioral-based malware detectors would be... more A promising avenue for improving the effectiveness of behavioral-based malware detectors would be to combine fast traditional machine learning detectors with high-accuracy, but time-consuming deep learning models. The main idea would be to place software receiving borderline classifications by traditional machine learning methods in an environment where uncertainty is added, while software is analyzed by more time-consuming deep learning models. The goal of uncertainty would be to rate-limit actions of potential malware during the time consuming deep analysis. In this paper, we present a detailed description of the analysis and implementation of CHAMELEON, a framework for realizing this uncertain environment for Linux. CHAMELEON offers two environments for software: (i) standard - for any software identified as benign by conventional machine learning methods and (ii) uncertain - for software receiving borderline classifications when analyzed by these conventional machine learning me...
This paper investigates whether computer usage profiles comprised of process-, network-, mouseand... more This paper investigates whether computer usage profiles comprised of process-, network-, mouseand keystroke-related events are unique and temporally consistent in a naturalistic setting, discussing challenges and opportunities of using such profiles in applications of continuous authentication. We collected ecologically-valid computer usage profiles from 28 MS Windows 10 computer users over 8 weeks and submitted this data to comprehensive machine learning analysis involving a diverse set of online and offline classifiers. We found that (i) computer usage profiles have the potential to uniquely characterize computer users (with a maximum F-score of 99, 94%); (ii) network-related events were the most useful features to properly recognize profiles (95.14% of the top features distinguishing users being network-related); (iii) user profiles were mostly inconsistent over the 8-week data collection period, with 92.86% of users exhibiting drifts in terms of time and usage habits; and (iv) o...
IEEE Transactions on Dependable and Secure Computing, 2020
A promising avenue for improving the effectiveness of behavioral-based malware detectors is to le... more A promising avenue for improving the effectiveness of behavioral-based malware detectors is to leverage two-phase detection mechanisms. Existing problem in two-phase detection is that after the first phase produces borderline decision, suspicious behaviors are not well contained before the second phase completes. This paper improves CHAMELEON, a framework to realize the uncertain environment. CHAMELEON offers two environments: standardfor software identified as benign by the first phase, and uncertainfor software received borderline classification from the first phase. The uncertain environment adds obstacles to software execution through random perturbations applied probabilistically. We introduce a dynamic perturbation threshold that can target malware disproportionately more than benign software. We analyzed the effects of the uncertain environment by manually studying 113 software and 100 malware, and found that 92% malware and 10% benign software disrupted during execution. The results were then corroborated by an extended dataset (5,679 Linux malware samples) on a newer system. Finally, a careful inspection of the benign software crashes revealed some software bugs, highlighting CHAMELEON's potential as a practical complementary antimalware solution.
2021 IEEE European Symposium on Security and Privacy (EuroS&P), 2021
Programmable Logic Controllers (PLCs) play a critical role in the industrial control systems. Vul... more Programmable Logic Controllers (PLCs) play a critical role in the industrial control systems. Vulnerabilities in PLC programs might lead to attacks causing devastating consequences to the critical infrastructure, as shown in Stuxnet and similar attacks. In recent years, we have seen an exponential increase in vulnerabilities reported for PLC control logic. Looking back on past research, we found extensive studies explored control logic modification attacks, as well as formal verification-based security solutions. We performed systematization on these studies, and found attacks that can compromise a full chain of control and evade detection. However, the majority of the formal verification research investigated ad-hoc techniques targeting PLC programs. We discovered challenges in every aspect of formal verification, rising from (1) the ever-expanding attack surface from evolved system design, (2) the real-time constraint during the program execution, and (3) the barrier in security evaluation given proprietary and vendor-specific dependencies on different techniques. Based on the knowledge systematization, we provide a set of recommendations for future research directions, and we highlight the need of defending security issues besides safety issues.
2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE), 2016
Applications are generally written assuming a predictable and well-behaved OS. In practice, they ... more Applications are generally written assuming a predictable and well-behaved OS. In practice, they experience unpredictable misbehavior at the OS level and across OSes: different OSes can handle network events differently, APIs can behave differently across OSes, and OSes may be compromised or buggy. This unpredictability is challenging because its sources typically manifest during deployment and are hard to reproduce. This paper introduces Bear, a framework for statistical analysis of application sensitivity to OS unpredictability that can help developers build more resilient software, discover challenging bugs and identify the scenarios that most need validation. Bear analyzes a program with a set of perturbation strategies on a set of commonly used system calls in order to discover the most sensitive system calls for each application, the most impactful strategies, and how they predict abnormal program outcome. We evaluated Bear with 113 CPU and IO-bound programs, and our results show that null memory dereferencing and erroneous buffer operations are the most impactful strategies for predicting abnormal program execution and that their impacts increase tenfold with workload increase (e.g. number of network requests from 10 to 1000). Generic system calls are more sensitive than specialized system calls-for example, write and sendto can both be used to send data through a socket, but the sensitivity of write is twice that of sendto. System calls with an array parameter (e.g. read) are more sensitive to perturbations than those having a struct parameter with a buffer (e.g readv). Moreover, the fewer parameters a system call has, the more sensitive it is.
As the malware research field became more established over the last two decades, new research que... more As the malware research field became more established over the last two decades, new research questions arose, such as how to make malware research reproducible, how to bring scientific rigor to attack papers, or what is an appropriate malware dataset for relevant experimental results. The challenges these questions pose also brings pitfalls that affect the multiple malware research stakeholders. To help answering those questions and to highlight potential research pitfalls to be avoided, in this paper, we present a systematic literature review of 491 papers on malware research published in major security conferences between 2000 and 2018. We identified the most common pitfalls present in past literature and propose a method for assessing current (and future) malware research. Our goal is towards integrating science and engineering best practices to develop further, improved research by learning from issues present in the published body of work. As far as we know, this is the largest literature review of its kind and the first to summarize research pitfalls in a research methodology that avoids them. In total, we discovered 20 pitfalls that limit current research impact and reproducibility. The identified pitfalls range from (i) the lack of a proper threat model, that complicates paper's evaluation, to (ii) the use of closed-source solutions and private datasets, that limit reproducibility. We also report yet-to-be-overcome challenges that are inherent to the malware nature, such as non-deterministic analysis results. Based on our findings, we propose a set of actions to be taken by the malware research and development community for future work: (i) Consolidation of malware research as a field constituted of diverse research approaches (e.g., engineering solutions, offensive research, landscapes/observational studies, and network traffic/system traces analysis); (ii) design of engineering solutions with clearer, direct assumptions (e.g., positioning solutions as proofs-of-concept vs. deployable); (iii) design of experiments that reflects (and emphasizes) the target scenario for the proposed solution (e.g., corporation, user, country-wide); (iv) clearer exposition and discussion of limitations of used technologies and exercised norms/standards for research (e.g., the use of closedsource antiviruses as ground-truth). Hypothesis Definition & Research Requirements Background Research Solution Requirements Solution Design Solution Development / Prototyping Research Objective Definition Engineering Method Common Core Experiment Design Test of Hypothesis / Evaluation of Solution Analysis of Results Results align with Hypothesis / Requirements? Communicate Results
On-device machine learning (ML) is quickly gaining popularity among mobile apps. It allows offlin... more On-device machine learning (ML) is quickly gaining popularity among mobile apps. It allows offline model inference while preserving user privacy. However, ML models, considered as core intellectual properties of model owners, are now stored on billions of untrusted devices and subject to potential thefts. Leaked models can cause both severe financial loss and security consequences. This paper presents the first empirical study of ML model protection on mobile devices. Our study aims to answer three open questions with quantitative evidence: How widely is model protection used in apps? How robust are existing model protection techniques? What impacts can (stolen) models incur? To that end, we built a simple app analysis pipeline and analyzed 46,753 popular apps collected from the US and Chinese app markets. We identified 1,468 ML apps spanning all popular app categories. We found that, alarmingly, 41% of ML apps do not protect their models at all, which can be trivially stolen from a...
Uploads
Papers by Ruimin Sun