Papers by Christopher Theisen
Stack traces reveal attack surfaces
Researchers at North Carolina State University and Microsoft Research have developed a technique ... more Researchers at North Carolina State University and Microsoft Research have developed a technique called attack surface approximation. In this approach, the attack surface of a system is determined by looking at stack traces from crash dumps from the target system. The crash dump stack traces are parsed out into the individual code elements that are seen on each stack trace. Depending on the stack trace in question, these elements could be binaries, files, or functions. Any code element that is seen on at least one stack trace is considered to be on the attack surface of the system, as determined by this approach. Researchers also explored other metrics, such as the frequency of appearance of code elements, how often code elements appear next to each other, and the shapes that form within the graph representation of the stack traces.
How bad is it, really? an analysis of severity scores for vulnerabilities: poster
Proceedings of the 5th Annual Symposium and Bootcamp on Hot Topics in the Science of Security, 2018
To date, vulnerability research has focused on the binary classification of code as vulnerable or... more To date, vulnerability research has focused on the binary classification of code as vulnerable or not vulnerable. To better understand the conditions in which vulnerabilities occur, researchers must consider the severity of these vulnerabilities in addition to a binary classification system. To explore this issue, we mined 2,979 publicly disclosed vulnerabilities from Fedora 24 and 25. We then found severity scores from the Common Vulnerability Scoring System (CVSS) and plotted the distribution of these vulnerabilities. We found that publicly scored vulnerabilities skew high, with few vulnerabilities rated lower than a 5. We then explore other potential issues with the use of CVSS in practice, such as imbalances in Confidentiality, Availability, and Integrity scores.
With an increasing number of cybersecurity attacks threatening consumers, organizations, and gove... more With an increasing number of cybersecurity attacks threatening consumers, organizations, and governments, the need for trained software security professionals is greater than ever. However, the industry is experiencing a shortage in security professionals for roles at all levels of cybersecurity. Massively Open Online Courses (MOOCs) offer educators an opportunity to retrain current professionals on cybersecurity topics to meet new and ongoing threats. The goal of this paper is to assist instructors of online software security courses in making their courses engaging and effective. In this paper, we present the details of our online software security course, including the technologies used and the material presented. We conducted a preand post-survey of course participants and report information on their backgrounds, motivations, and learning objectives. Based on our reflection on the course, we recommend that future instructors of online security courses seed peer discussion on onl...
Risk-based attack surface approximation: how much data is enough?
2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP)
Proactive security reviews and test efforts are a necessary component of the software development... more Proactive security reviews and test efforts are a necessary component of the software development lifecycle. Resource limitations often preclude reviewing the entire code base. Making informed decisions on what code to review can improve a team's ability to find and remove vulnerabilities. Risk-based attack surface approximation (RASA) is a technique that uses crash dump stack traces to predict what code may contain exploitable vulnerabilities. The goal of this research is to help software development teams prioritize security efforts by the efficient development of a risk-based attack surface approximation. We explore the use of RASA using Mozilla Firefox and Microsoft Windows stack traces from crash dumps. We create RASA at the file level for Firefox, in which the 15.8% of the files that were part of the approximation contained 73.6% of the vulnerabilities seen for the product. We also explore the effect of random sampling of crashes on the approximation, as it may be impractical for organizations to store and process every crash received. We find that 10-fold random sampling of crashes at a rate of 10% resulted in 3% less vulnerabilities identified than using the entire set of stack traces for Mozilla Firefox. Sampling crashes in Windows 8.1 at a rate of 40% resulted in insignificant differences in vulnerability and file coverage as compared to a rate of 100%.
Information and Software Technology
Context: Michael Howard conceptualized the attack surface of a software system as a metaphor for ... more Context: Michael Howard conceptualized the attack surface of a software system as a metaphor for risk assessment during the development and maintenance of software. While the phrase attack surface is used in a variety of contexts in cybersecurity, professionals have different conceptions of what the phrase means. Objective: The goal of this systematic literature review is to aid researchers and practitioners in reasoning about security in terms of attack surface by exploring various definitions of the phrase attack surface. Method: We reviewed 644 works from prior literature, including research papers, magazine articles, and technical reports, that use the phrase attack surface and categorized them into those that provided their own definition; cited another definition; or expected the reader to intuitively understand the phrase. Results: In our study, 71% of the papers used the phrase without defining it or citing another paper. Additionally, we found six themes of definitions for the phrase attack surface. Conclusion: Based on our analysis, we recommend practitioners choose a definition of attack surface appropriate for their domain based on the six themes we identified in our study.
IEEE Transactions on Software Engineering
Software engineers can find vulnerabilities with less effort if they are directed towards code th... more Software engineers can find vulnerabilities with less effort if they are directed towards code that might contain more vulnerabilities. HARMLESS is an incremental support vector machine tool that builds a vulnerability prediction model from the source code inspected to date, then suggests what source code files should be inspected next. In this way, HARMLESS can reduce the time and effort required to achieve some desired level of recall for finding vulnerabilities. The tool also provides feedback on when to stop (at that desired level of recall) while at the same time, correcting human errors by double-checking suspicious files. This paper evaluates HARMLESS on Mozilla Firefox vulnerability data. HARMLESS found 80, 90, 95, 99% of the vulnerabilities by inspecting 10, 16, 20, 34% of the source code files. When targeting 90, 95, 99% recall, HARMLESS could stop after inspecting 23, 30, 47% of the source code files. Even when human reviewers fail to identify half of the vulnerabilities (50% false negative rate), HARMLESS could detect 96% of the missing vulnerabilities by double-checking half of the inspected files. Our results serve to highlight the very steep cost of protecting software from vulnerabilities (in our case study that cost is, for example, the human effort of inspecting 28,750 × 20% = 5,750 source code files to identify 95% of the vulnerabilities). While this result could benefit the mission-critical projects where human resources are available for inspecting thousands of source code files, the research challenge for future work is how to further reduce that cost. The conclusion of this paper discusses various ways that goal might be achieved.
Proceedings of the Symposium and Bootcamp on the Science of Security, 2016
THEISEN, CHRISTOPHER RICHARD. Risk-Based Attack Surface Approximation. (Under the direction of La... more THEISEN, CHRISTOPHER RICHARD. Risk-Based Attack Surface Approximation. (Under the direction of Laurie Williams.) Motivation: Security testing and reviewing efforts are a necessity for software projects, but are time-consuming and expensive to apply. Organizations look to utilize security professionals as efficiently as possible. Identifying vulnerable code supports decision-making during all phases of software development. Potentially vulnerable code identified early in the development of software can facilitate "building security in" before release. Identifying potentially vulnerable code after release can prevent potential vulnerabilities from becoming major breaches. However, identification of potentially vulnerable code is difficult, as vulnerabilities are rare. Research Problem: Prioritizing security reviews is difficult, as we lack effective metrics for determining what code should be considered security relevant. Objective: The goal of this research is to aid software engineers in prioritizing security efforts by approximating the attack surface of a system via crash dump stack trace analysis. Approach: We propose Risk-Based Attack Surface Approximation (RASA), an approach to determine the attack surface of software systems by using crash dump stack traces and the code that appears on them as a metric for prioritizing security reviews. We call our approach "risk-based" as it identifies the "riskiest" code in the system. Crashes are empirical evidence that the code involved in the crash was executed in an unexpected way, in the same way that vulnerabilities are often instances of code executed in an unexpected way. In the case of a crash, some outside input was able to cause the state in the software system that caused the crash conditions. The same types of input and states could be correlated with input and states that result in vulnerabilities. Results: We have performed studies on the effectiveness of RASA on Microsoft Windows and Mozilla Firefox, including the effect of randomly sampling crash dump stack traces from the targeted software system. For Windows, 48.4% of binaries appeared on the approximated attack surface measured by RASA, while 94.6% of historical vulnerabilities occurred on the approximated attack
Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, 2015
25th International Conference on Software Engineering, 2003. Proceedings., 2003
Software engineering researchers solve problems of several different kinds. To do so, they produc... more Software engineering researchers solve problems of several different kinds. To do so, they produce several different kinds of results, and they should develop appropriate evidence to validate these results. They often report their research in conference papers. I analyzed the abstracts of research papers submitted to ICSE 2002 in order to identify the types of research reported in the submitted and accepted papers, and I observed the program committee discussions about which papers to accept. This report presents the research paradigms of the papers, common concerns of the program committee, and statistics on success rates. This information should help researchers design better research projects and write papers that present their results to best advantage.
International Conference on Software Engineering, 2003
Writing a good paper depends on having good research to write about > If the result is not signif... more Writing a good paper depends on having good research to write about > If the result is not significant, it doesn't matter how good the paper is > If your claims don't match your results, you'll have trouble providing convincing evidence Basic Research Recognize problem, Invent ideas Concept Formation Refine ideas, publish solutions Development & Extension Try it out, clarify, refine Internal Exploration Stabilize, port, use for real problems External Exploration Broaden user group, extend
Proceedings of the 38th International Conference on Software Engineering Companion, 2016
Massively Open Online Courses (MOOCs) provide a unique opportunity to reach out to students who w... more Massively Open Online Courses (MOOCs) provide a unique opportunity to reach out to students who would not normally be reached by alleviating the need to be physically present in the classroom. However, teaching software security coursework outside of a classroom setting can be challenging. What are the challenges when converting security material from an on-campus course to the MOOC format? The goal of this research is to assist educators in constructing software security coursework by providing a comparison of classroom courses and MOOCs. In this work, we compare demographic information, student motivations, and student results from an on-campus software security course and a MOOC version of the same course. We found that the two populations of students differed, with the MOOC reaching a more diverse set of students than the on-campus course. We found that students in the on-campus course had higher quiz scores, on average, than students in the MOOC. Finally, we document our experience running the courses and what we would do differently to assist future educators constructing similar MOOC's.
Approximating Attack Surfaces with Stack Traces
2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, 2015
Uploads
Papers by Christopher Theisen