2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE), 2017
Although there has been increasing demand for more reliable web applications, JavaScript bugs abo... more Although there has been increasing demand for more reliable web applications, JavaScript bugs abound in web applications. In response to this issue, researchers have proposed automated fault detection tools, which statically analyze the web application code to find bugs. While useful, these tools either only target a limited set of bugs based on predefined rules, or they do not detect bugs caused by cross-language interactions, which occur frequently in web application code. To address this problem, we present an anomaly-based inconsistency detection approach, implemented in a tool called HOLOCRON. The main novelty of our approach is that it does not look for hard-coded inconsistency classes. Instead, it applies subtree pattern matching to infer inconsistency classes and association rule mining to detect inconsistencies that occur both within a single language, and between two languages. We evaluated HOLOCRON, and it successfully detected 51 previously unreported inconsistenciesincluding 18 bugs and 33 code smells-in 12 web applications.
With web application usage becoming ubiquitous, there is greater demand for making such applicati... more With web application usage becoming ubiquitous, there is greater demand for making such applications more reliable. This is especially true as more users rely on web applications to conduct day-to-day tasks, and more companies rely on these applications to drive their business. Since the advent of Web 2.0, developers often implement much of the web application’s functionality at the client-side, using client-side JavaScript. Unfortunately, despite repeated complaints from developers about confusing aspects of the JavaScript language, little work has been done analyzing the language’s reliability characteristics. With this problem in mind, we conducted an empirical study of real-world JavaScript bugs, with the goal of understanding their root cause and impact. We found that most of these bugs are DOM-related, which means they occur as a result of the JavaScript code’s interaction with the Document Object Model (DOM). Having gained a thorough understanding of JavaScript bugs, we desig...
Software Testing, Verification and Reliability, 2020
A performance regression in software is defined as an increase in an application step's response ... more A performance regression in software is defined as an increase in an application step's response time as a result of code changes. Detecting such regressions can be done using profiling tools; however, investigating their root cause is a mostly-manual and time-consuming task. This statement holds true especially when comparing execution timelines, which are dynamic function call trees augmented with response time data; these timelines are compared to find the performance regression-causesthe lowest-level function calls that regressed during execution. When done manually, these comparisons often require the investigator to analyze thousands of function call nodes. Further, performing these comparisons on web applications is challenging due to JavaScript's asynchronous and event-driven model, which introduce noise in the timelines. In response, we propose a design-ZAMthat automatically compares execution timelines collected from web applications, to identify performance regression-causes. Our approach uses a hybrid node matching algorithm that recursively attempts to find the longest common subsequence in each call tree level, then aggregates multiple comparisons' results to eliminate noise. Our evaluation of ZAM on 10 web applications indicates that it can identify performance regression-causes with a path recall of 100% and a path precision of 96%, while performing comparisons in under a minute on average. We also demonstrate the real-world applicability of ZAM, which has been used to successfully complete performance investigations by the performance and reliability team in SAP.
Client-side JavaScript is widely used in web applications to improve user-interactivity and minim... more Client-side JavaScript is widely used in web applications to improve user-interactivity and minimize client-server communications. Unfortunately, JavaScript is known to be error-prone. While prior studies have demonstrated the prevalence of JavaScript faults, no attempts have been made to determine their causes and consequences. The goal of our study is to understand the root causes and impact of JavaScript faults and how the results can impact JavaScript programmers, testers and tool developers. We perform an empirical study of 502 bug reports from 19 bug repositories. The bug reports are thoroughly examined to classify and extract information about each bug's cause (the error) and consequence (the failure and impact). Our results show that the majority (68%) of JavaScript faults are DOM-related, meaning they are caused by faulty interactions of the JavaScript code with the Document Object Model (DOM). Further, 80% of the highest impact JavaScript faults are DOM-related. Finally, most JavaScript faults originate from programmer mistakes committed in the JavaScript code itself, as opposed to other web application components. These results indicate that JavaScript programmers and testers need tools that can help them reason about the DOM. Additionally, developers can use the error patterns we found to design more powerful static analysis tools for JavaScript.
2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, 2015
Higher demands for more reliable and maintainable JavaScript-based web applications have led to t... more Higher demands for more reliable and maintainable JavaScript-based web applications have led to the recent development of MVC (Model-View-Controller) frameworks. One of the main advantages of using these frameworks is that they abstract out DOM API method calls, which are one of the leading causes of web application faults, due to their often complicated interaction patterns. However, MVC frameworks are susceptible to inconsistencies between the identifiers and types of variables and functions used throughout the application. In response to this problem, we introduce a formal consistency model for web applications made using MVC frameworks. We propose an approach-called AUREBESH-that automatically detects inconsistencies in such applications. We evaluate AUREBESH by conducting a fault injection experiment and by running it on real applications. Our results show that AUREBESH is accurate, with an overall recall of 96.1% and a precision of 100%. It is also useful in detecting bugs, allowing us to find 15 real-world bugs in applications built on AngularJS, a popular MVC framework.
Client-side JavaScript is being widely used in popular web applications to improve functionality,... more Client-side JavaScript is being widely used in popular web applications to improve functionality, increase responsiveness, and decrease load times. However, it is challenging to build reliable applications using JavaScript. This work presents an empirical characterization of the error messages printed by JavaScript code in web applications, and attempts to understand their root causes. We find that JavaScript errors occur in production web applications, and that the errors fall into a small number of categories. In addition, we find that certain types of web applications are more prone to JavaScript errors than others. We further find that both non-deterministic and deterministic errors occur in the applications, and that the speed of testing plays an important role in exposing errors. Finally, we study the correlations among the static and dynamic properties of the application and the frequency of errors in it in order to understand the root causes of the errors.
Software Testing, Verification and Reliability, 2015
JAVASCRIPT is a scripting language that plays a prominent role in web applications today. It is d... more JAVASCRIPT is a scripting language that plays a prominent role in web applications today. It is dynamic, loosely typed, asynchronous, and is extensively used to interact with the DOM at runtime. All these characteristics make JAVASCRIPT code error-prone; unfortunately, JAVASCRIPT fault localization remains a tedious and mainly manual task. Despite these challenges, the problem has received very limited research attention. This paper proposes an automated technique to localize JAVASCRIPT faults based on dynamic analysis, tracing, and backward slicing of JAVASCRIPT code. This technique is capable of handling features of JAVASCRIPT code that have traditionally been difficult to analyze, including eval, anonymous functions, and minified code. The approach is implemented in an open source tool called AUTOFLOX, and evaluation results indicate that it is capable of (1) automatically localizing DOM-related JAVASCRIPT faults with high accuracy (over 96%) and no false-positives, and (2) isolating JAVASCRIPT faults in production websites, as well as actual bugs from real-world web applications.
2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, 2012
JAVASCRIPT is a scripting language that plays a prominent role in modern web applications today. ... more JAVASCRIPT is a scripting language that plays a prominent role in modern web applications today. It is dynamic, loosely typed, and asynchronous. In addition, it is extensively used to interact with the DOM at runtime. All these characteristics make JAVASCRIPT code error-prone and challenging to debug. JAVASCRIPT fault localization is currently a tedious and mainly manual task. Despite these challenges, the problem has received very limited attention from the research community. We propose an automated technique to localize JAVASCRIPT faults based on dynamic analysis of the web application, tracing, and backward slicing of JAVASCRIPT code. Our fault localization approach is implemented in an open source tool called AUTOFLOX. The results of our empirical evaluation indicate that (1) DOM-related errors are prominent in web applications, i.e., they form at least 79% of reported JAVASCRIPT bugs, (2) our approach is capable of automatically localizing DOM-related JAVASCRIPT errors with a high degree of accuracy (over 90%) and no false-positives, and (3) our approach is capable of isolating JAVASCRIPT errors in a production web application, viz., Tumblr.
Proceedings of the 36th International Conference on Software Engineering, 2014
JavaScript is used in web applications for achieving rich user interfaces and implementing core f... more JavaScript is used in web applications for achieving rich user interfaces and implementing core functionality. Unfortunately, Java-Script code is known to be prone to faults. In an earlier study, we found that over 65% of such faults are caused by the interaction of JavaScript code with the DOM at runtime (DOM-related faults). In this paper, we first perform an analysis of 190 bug reports to understand fixes commonly applied by programmers to these DOMrelated faults; we observe that parameter replacements and DOM element validations are common fix categories. Based on these findings, we propose an automated technique and tool, called VE-JOVIS, for suggesting repairs for DOM-based JavaScript faults. To evaluate VEJOVIS, we conduct a case study in which we subject VEJOVIS to 22 real-world bugs across 11 applications. We find that VEJOVIS accurately suggests repairs for 20 out of the 22 bugs, and in 13 of the 20 cases, the correct fix was the top ranked one.
2011 IEEE 22nd International Symposium on Software Reliability Engineering, 2011
Client-side JavaScript is being widely used in popular websites to improve functionality, increas... more Client-side JavaScript is being widely used in popular websites to improve functionality, increase responsiveness, and decrease load times. However, the use of JavaScript can decrease the reliability of these websites. This paper presents an empirical characterization of the error messages printed by JavaScript as it executes in these popular websites, and attempts to understand their root causes. We find that JavaScript errors abound in real websites, and that the errors fall into a small number of categories. We further find that both non-deterministic and deterministic errors occur in the applications, and that the speed of testing plays an important role in exposing errors. Finally, we study the correlations among the static and dynamic properties of the application and the frequency of errors exhibited by it in an attempt to understand the root causes of the errors. The study is carried out using the Firefox web browser.
2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement, 2013
Client-side JavaScript is widely used in web applications to improve user-interactivity and minim... more Client-side JavaScript is widely used in web applications to improve user-interactivity and minimize clientserver communications. Unfortunately, web applications are prone to JavaScript faults. While prior studies have demonstrated the prevalence of these faults, no attempts have been made to determine their root causes and consequences. Objective: The goal of our study is to understand the root causes and impact of JavaScript faults and how the results can impact JavaScript programmers, testers and tool developers. Method: We perform an empirical study of 317 bug reports from 12 bug repositories. The bug reports are thoroughly examined to classify and extract information about the fault's cause (the error) and consequence (the failure and impact). Result: The majority (65%) of JavaScript faults are DOM-related, meaning they are caused by faulty interactions of the JavaScript code with the Document Object Model (DOM). Further, 80% of the highest impact JavaScript faults are DOM-related. Finally, most JavaScript faults originate from programmer mistakes committed in the JavaScript code itself, as opposed to other web application components such as the server-side or HTML code. Conclusion: Given the prevalence of DOM-related faults, JavaScript programmers need development tools that can help them reason about the DOM. Also, testers should prioritize detection of DOM-related faults as most high impact faults belong to this category. Finally, developers can use the error patterns we found to design more powerful static analysis tools for JavaScript.
2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE), 2017
Although there has been increasing demand for more reliable web applications, JavaScript bugs abo... more Although there has been increasing demand for more reliable web applications, JavaScript bugs abound in web applications. In response to this issue, researchers have proposed automated fault detection tools, which statically analyze the web application code to find bugs. While useful, these tools either only target a limited set of bugs based on predefined rules, or they do not detect bugs caused by cross-language interactions, which occur frequently in web application code. To address this problem, we present an anomaly-based inconsistency detection approach, implemented in a tool called HOLOCRON. The main novelty of our approach is that it does not look for hard-coded inconsistency classes. Instead, it applies subtree pattern matching to infer inconsistency classes and association rule mining to detect inconsistencies that occur both within a single language, and between two languages. We evaluated HOLOCRON, and it successfully detected 51 previously unreported inconsistenciesincluding 18 bugs and 33 code smells-in 12 web applications.
With web application usage becoming ubiquitous, there is greater demand for making such applicati... more With web application usage becoming ubiquitous, there is greater demand for making such applications more reliable. This is especially true as more users rely on web applications to conduct day-to-day tasks, and more companies rely on these applications to drive their business. Since the advent of Web 2.0, developers often implement much of the web application’s functionality at the client-side, using client-side JavaScript. Unfortunately, despite repeated complaints from developers about confusing aspects of the JavaScript language, little work has been done analyzing the language’s reliability characteristics. With this problem in mind, we conducted an empirical study of real-world JavaScript bugs, with the goal of understanding their root cause and impact. We found that most of these bugs are DOM-related, which means they occur as a result of the JavaScript code’s interaction with the Document Object Model (DOM). Having gained a thorough understanding of JavaScript bugs, we desig...
Software Testing, Verification and Reliability, 2020
A performance regression in software is defined as an increase in an application step's response ... more A performance regression in software is defined as an increase in an application step's response time as a result of code changes. Detecting such regressions can be done using profiling tools; however, investigating their root cause is a mostly-manual and time-consuming task. This statement holds true especially when comparing execution timelines, which are dynamic function call trees augmented with response time data; these timelines are compared to find the performance regression-causesthe lowest-level function calls that regressed during execution. When done manually, these comparisons often require the investigator to analyze thousands of function call nodes. Further, performing these comparisons on web applications is challenging due to JavaScript's asynchronous and event-driven model, which introduce noise in the timelines. In response, we propose a design-ZAMthat automatically compares execution timelines collected from web applications, to identify performance regression-causes. Our approach uses a hybrid node matching algorithm that recursively attempts to find the longest common subsequence in each call tree level, then aggregates multiple comparisons' results to eliminate noise. Our evaluation of ZAM on 10 web applications indicates that it can identify performance regression-causes with a path recall of 100% and a path precision of 96%, while performing comparisons in under a minute on average. We also demonstrate the real-world applicability of ZAM, which has been used to successfully complete performance investigations by the performance and reliability team in SAP.
Client-side JavaScript is widely used in web applications to improve user-interactivity and minim... more Client-side JavaScript is widely used in web applications to improve user-interactivity and minimize client-server communications. Unfortunately, JavaScript is known to be error-prone. While prior studies have demonstrated the prevalence of JavaScript faults, no attempts have been made to determine their causes and consequences. The goal of our study is to understand the root causes and impact of JavaScript faults and how the results can impact JavaScript programmers, testers and tool developers. We perform an empirical study of 502 bug reports from 19 bug repositories. The bug reports are thoroughly examined to classify and extract information about each bug's cause (the error) and consequence (the failure and impact). Our results show that the majority (68%) of JavaScript faults are DOM-related, meaning they are caused by faulty interactions of the JavaScript code with the Document Object Model (DOM). Further, 80% of the highest impact JavaScript faults are DOM-related. Finally, most JavaScript faults originate from programmer mistakes committed in the JavaScript code itself, as opposed to other web application components. These results indicate that JavaScript programmers and testers need tools that can help them reason about the DOM. Additionally, developers can use the error patterns we found to design more powerful static analysis tools for JavaScript.
2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, 2015
Higher demands for more reliable and maintainable JavaScript-based web applications have led to t... more Higher demands for more reliable and maintainable JavaScript-based web applications have led to the recent development of MVC (Model-View-Controller) frameworks. One of the main advantages of using these frameworks is that they abstract out DOM API method calls, which are one of the leading causes of web application faults, due to their often complicated interaction patterns. However, MVC frameworks are susceptible to inconsistencies between the identifiers and types of variables and functions used throughout the application. In response to this problem, we introduce a formal consistency model for web applications made using MVC frameworks. We propose an approach-called AUREBESH-that automatically detects inconsistencies in such applications. We evaluate AUREBESH by conducting a fault injection experiment and by running it on real applications. Our results show that AUREBESH is accurate, with an overall recall of 96.1% and a precision of 100%. It is also useful in detecting bugs, allowing us to find 15 real-world bugs in applications built on AngularJS, a popular MVC framework.
Client-side JavaScript is being widely used in popular web applications to improve functionality,... more Client-side JavaScript is being widely used in popular web applications to improve functionality, increase responsiveness, and decrease load times. However, it is challenging to build reliable applications using JavaScript. This work presents an empirical characterization of the error messages printed by JavaScript code in web applications, and attempts to understand their root causes. We find that JavaScript errors occur in production web applications, and that the errors fall into a small number of categories. In addition, we find that certain types of web applications are more prone to JavaScript errors than others. We further find that both non-deterministic and deterministic errors occur in the applications, and that the speed of testing plays an important role in exposing errors. Finally, we study the correlations among the static and dynamic properties of the application and the frequency of errors in it in order to understand the root causes of the errors.
Software Testing, Verification and Reliability, 2015
JAVASCRIPT is a scripting language that plays a prominent role in web applications today. It is d... more JAVASCRIPT is a scripting language that plays a prominent role in web applications today. It is dynamic, loosely typed, asynchronous, and is extensively used to interact with the DOM at runtime. All these characteristics make JAVASCRIPT code error-prone; unfortunately, JAVASCRIPT fault localization remains a tedious and mainly manual task. Despite these challenges, the problem has received very limited research attention. This paper proposes an automated technique to localize JAVASCRIPT faults based on dynamic analysis, tracing, and backward slicing of JAVASCRIPT code. This technique is capable of handling features of JAVASCRIPT code that have traditionally been difficult to analyze, including eval, anonymous functions, and minified code. The approach is implemented in an open source tool called AUTOFLOX, and evaluation results indicate that it is capable of (1) automatically localizing DOM-related JAVASCRIPT faults with high accuracy (over 96%) and no false-positives, and (2) isolating JAVASCRIPT faults in production websites, as well as actual bugs from real-world web applications.
2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, 2012
JAVASCRIPT is a scripting language that plays a prominent role in modern web applications today. ... more JAVASCRIPT is a scripting language that plays a prominent role in modern web applications today. It is dynamic, loosely typed, and asynchronous. In addition, it is extensively used to interact with the DOM at runtime. All these characteristics make JAVASCRIPT code error-prone and challenging to debug. JAVASCRIPT fault localization is currently a tedious and mainly manual task. Despite these challenges, the problem has received very limited attention from the research community. We propose an automated technique to localize JAVASCRIPT faults based on dynamic analysis of the web application, tracing, and backward slicing of JAVASCRIPT code. Our fault localization approach is implemented in an open source tool called AUTOFLOX. The results of our empirical evaluation indicate that (1) DOM-related errors are prominent in web applications, i.e., they form at least 79% of reported JAVASCRIPT bugs, (2) our approach is capable of automatically localizing DOM-related JAVASCRIPT errors with a high degree of accuracy (over 90%) and no false-positives, and (3) our approach is capable of isolating JAVASCRIPT errors in a production web application, viz., Tumblr.
Proceedings of the 36th International Conference on Software Engineering, 2014
JavaScript is used in web applications for achieving rich user interfaces and implementing core f... more JavaScript is used in web applications for achieving rich user interfaces and implementing core functionality. Unfortunately, Java-Script code is known to be prone to faults. In an earlier study, we found that over 65% of such faults are caused by the interaction of JavaScript code with the DOM at runtime (DOM-related faults). In this paper, we first perform an analysis of 190 bug reports to understand fixes commonly applied by programmers to these DOMrelated faults; we observe that parameter replacements and DOM element validations are common fix categories. Based on these findings, we propose an automated technique and tool, called VE-JOVIS, for suggesting repairs for DOM-based JavaScript faults. To evaluate VEJOVIS, we conduct a case study in which we subject VEJOVIS to 22 real-world bugs across 11 applications. We find that VEJOVIS accurately suggests repairs for 20 out of the 22 bugs, and in 13 of the 20 cases, the correct fix was the top ranked one.
2011 IEEE 22nd International Symposium on Software Reliability Engineering, 2011
Client-side JavaScript is being widely used in popular websites to improve functionality, increas... more Client-side JavaScript is being widely used in popular websites to improve functionality, increase responsiveness, and decrease load times. However, the use of JavaScript can decrease the reliability of these websites. This paper presents an empirical characterization of the error messages printed by JavaScript as it executes in these popular websites, and attempts to understand their root causes. We find that JavaScript errors abound in real websites, and that the errors fall into a small number of categories. We further find that both non-deterministic and deterministic errors occur in the applications, and that the speed of testing plays an important role in exposing errors. Finally, we study the correlations among the static and dynamic properties of the application and the frequency of errors exhibited by it in an attempt to understand the root causes of the errors. The study is carried out using the Firefox web browser.
2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement, 2013
Client-side JavaScript is widely used in web applications to improve user-interactivity and minim... more Client-side JavaScript is widely used in web applications to improve user-interactivity and minimize clientserver communications. Unfortunately, web applications are prone to JavaScript faults. While prior studies have demonstrated the prevalence of these faults, no attempts have been made to determine their root causes and consequences. Objective: The goal of our study is to understand the root causes and impact of JavaScript faults and how the results can impact JavaScript programmers, testers and tool developers. Method: We perform an empirical study of 317 bug reports from 12 bug repositories. The bug reports are thoroughly examined to classify and extract information about the fault's cause (the error) and consequence (the failure and impact). Result: The majority (65%) of JavaScript faults are DOM-related, meaning they are caused by faulty interactions of the JavaScript code with the Document Object Model (DOM). Further, 80% of the highest impact JavaScript faults are DOM-related. Finally, most JavaScript faults originate from programmer mistakes committed in the JavaScript code itself, as opposed to other web application components such as the server-side or HTML code. Conclusion: Given the prevalence of DOM-related faults, JavaScript programmers need development tools that can help them reason about the DOM. Also, testers should prioritize detection of DOM-related faults as most high impact faults belong to this category. Finally, developers can use the error patterns we found to design more powerful static analysis tools for JavaScript.
Uploads
Papers by Frolin Ocariza