Supporting interactive system testing with interaction sequences

This paper presents a novel approach to generate and execute graphical user interfaces from decision based UML sequence diagrams .This has been designed especially for efficient user interfaces where with less generation and verification effort are required different types of decision tables and UML sequence diagrams. The decision based sequence diagrams test sets did better at revealing integration level faults than only sequence diagrams. The experiment showed that model-based testing can be used to systematically generate test data and indicates that decision based UML models can play better roles in testing. Decision based Sequence Diagrams helps in deriving a reasonably complete reference model usable in industrial contexts, which can then be used for automatically deriving the test cases from user interfaces without going through the lines of codes of a program. Here the paper concentrates on generation of test cases for integration-level testing based on UML interaction diagr...

1999

Interactive systems can be analysed and developed in terms of functionality, presentation and behaviour. In this paper we demonstrate how testing information for interactive systems can be derived from formal speci cations of these aspects. The paper uses the Multi-modal Airline Travel Information System (MATIS) speci cation of Duke and Harrison. Their speci cation considers these aspects separately using both Z and CSP. Di erences between the aspects, supported by the di ering notations, carry through into the testing information we derive. This demonstrates that partial or viewpoint speci cations of interactive systems are useful for test derivation. This work is a step towards our goal of developing a framework for speci cation-based testing of interactive systems.

Lecture Notes in Computer Science, 2001

International Journal of Software Engineering and Knowledge Engineering, 2016

User inputs are critical for the security, safety, and reliability of software systems. This paper proposes a new concept called user input contracts, which is an integral part of a design-by-contract supplemented development process, and a model-based testing approach to detect violations of user input contracts. The approach generates test cases from an input contract integrated with graph-based model of user interface specification and applies them to the system under consideration. The paper presents a proof-of-concept tool that has been developed and used to validate the approach by experiments. The experiments are conducted on a web-based system for marketing tourist services to analyze input robustness of system under consideration with respect to user input contracts.

Proceedings First IEEE International Conference on Formal Engineering Methods

Today's software systems usually feature Graphical User Interfaces (GUIs). GUIs have become an important and accepted way of interacting with today's software. They can be a crucial point in the users' decisions to use or not use the system. However, GUI testing is difficult, extremely time-consuming, and costly, with very few tools and techniques available to aid in the testing process. This dissertation addresses the GUI testing problem. The goal is to introduce more systematization and automation into the GUI testing process by applying specification-based testing methods. The use of formal specifications allows the automatic generation of test cases containing not only the input data but also the outcomes expected. Specification-based testing methods have been applied for API testing but are insufficiently developed for GUI testing. Some of the specific challenges posed by GUI testing are addressed in this research work. The starting phase of the GUI testing process proposed is the construction of the GUI model. Then test cases are generated from the model and are executed on the GUI implementation. The results obtained from the GUI are compared with the results derived from the specification (test oracle). Whenever there is a conformance error it is reported. A set of guidelines are proposed for GUI modelling. For scalability and reusability reasons, GUI models are organized as a set of modules or classes. Besides modelling the atomic user actions and their effect on the GUI state, it is also possible to model composite actions (sequences of atomic actions), views (e.g., navigation map), and use case scenarios. Test cases are automatically generated in a two-step process: a FSM is built by a bounded exploration of the GUI model first; secondly, test sequences are generated from the FSM according to some coverage criteria (e.g., full transition coverage). The exploration process calculates the set of methods available in each state (those whose pre-condition holds) and calls them with parameter values taken from domains supplied by the tester. Test cases are sequences of operations that model user actions interleaved with operations to check the outcomes of those actions. The quality/adequacy of the generated FSM is assessed according to the degree of coverage of the model elements (actions, scenarios and views) as well as additional test conditions supplied by the tester. In order to reduce the number of test cases, it was developed an algorithm to reduce the FSM by removing redundant states and transitions with respect to the coverage goals defined. Conceptually, during test execution test cases are run in both levels, specification and implementation, in a "lock-step" mode and their results are compared after each step. This requires the definition of a mapping between abstract actions defined in the specification and concrete actions on concrete GUI objects in the implementation. To automate this process it was developed a GUI Mapping Tool that allows the tester to interactively relate the abstract actions with concrete GUI Abstract vi objects. The tool also generates automatically the code of a set of methods that simulate the concrete user actions on the GUI, and binds such methods to the abstract actions for test execution. The approach proposed in this research is illustrated and validated by two case studies performed on two software applications: the Notepad application that ships with the Microsoft Windows operating system, and the Address Book example application freely available with the open-source Eclipse platform. In spite of being used for several years, two errors were found in the Notepad application related to uncommon sequences of user actions. Since the source code of the Address Book application is available, a mutation testing technique was applied to assess the defect detection capability of the test cases generated automatically. All defects injected were detected. Overall, the approach proposed represents a significant improvement over the current GUI testing approaches based on Capture/Replay tools, since they only automate the execution and recording of the test cases. La qualité/adéquation du FSM produit est évaluée selon le degré d'assurance des éléments modèles (actions, scénarios et vues) aussi bien que conditions d'essai additionnelles fournies par l'essayeur. Afin de réduire le nombre de cas d'espèce, il Resumé viii a été développé un algorithme pour réduire le FSM par l'enlèvement états et transitions superflus en ce qui concerne les buts d'assurance définis. Conceptuellement, pendant l'exécution d'essai des cas d'espèce sont courus dans les deux niveaux, spécifications et l'exécution, en mode de «serrure-étape» et leurs résultats sont comparées après chaque étape. Ceci exige la définition de tracer entre les actions abstraites définies dans les spécifications et les actions concrètes sur les objets concrets de GUI dans l'exécution. Pour automatiser ce processus c'a été développé un GUI traçant l'outil qui permet à l'essayeur de rapporter interactivement les actions abstraites avec les objets concrets de GUI. L'outil produit également automatiquement du code d'un ensemble de méthodes qui simulent les actions concrètes d'utilisateur sur le GUI, et lie de telles méthodes aux actions abstraites pour l'exécution d'essai. L'approche proposée dans ce travail est illustrée et validée par deux études de cas réalisées sur deux applications de logiciel: l'application de bloc-notes qui se transporte avec le logiciel d'exploitation de Microsoft Windows, et l'exemple de carnet d'adresses application librement disponible avec la plateforme d'éclipse d'ouvrir-source. Malgré être employé pendant plusieurs années, deux erreurs ont été trouvées dans l'application de bloc-notes liée aux ordres rares des actions d'utilisateur. Depuis le code source de l'application de carnet d'adresses est disponible, une méthode d'essai de mutation a été appliquée pour évaluer les possibilités de détection de défaut des cas d'espèce produits automatiquement. Tous les défauts injectés ont été détectés. D'une façon générale, l'approche proposée représente une amélioration significative au-dessus des approches de essai courantes des GUI basées sur la Captation/Rejoue des outils, puisqu'ils automatisent seulement l'exécution et l'enregistrement des cas d'espèce.

Electronic Notes in Theoretical …, 2008

Reducing the likelihood of human error in the use of interactive systems is increasingly important: the use of such systems is becoming widespread in applications that demand high reliability due to safety, security, financial or similar considerations. Interactive systems are also becoming increasingly ubiquitous and being used in new and more complex situations. Consequently, the use of formal methods in verifying the correctness of interactive systems should also include analysis of human behaviour in interacting with the interface and with the wider socio-technical system. The aim of the FMIS series of workshops is to bring together researchers in computer science, cognitive psychology, and areas of HCI, from academia and industry who are interested in both formal methods and interactive system design. The 1st International Workshop on Formal Methods for Interactive Systems (FMIS 2006) was previously held in Macau, SAR China in October 2006. FMIS addresses issues of how formal methods can be applied to interactive system design. Topics of interest include, for example, the development of formal tools, techniques and methodologies based on cognitive psychology results, the development and use of formal user models, case studies applying formal methods to interface design, and formal analysis of the design of the wider socio-technical systems. The scope of HCI issues covered extends to all aspects of applying formal methods to interactive systems, including usability, user experience, human error, etc. and also theory where there is a demonstrable link to interactive system design. Application areas within scope include for example: mobile devices, embedded systems, safety-critical systems, high-reliability systems, shared control systems, digital libraries, eGovernment, pervasive systems, augmented reality, ubiquitous computing, and computer security applications. This Pre-proceedings includes 11 papers accepted for presentation at the 2nd International Workshop on Formal Methods for Interactive Systems (FMIS 2007) held on 4th September 2007 in Lancaster UK, as a satellite event of the 21st British HCI Group Annual Conference on Human Computer Interaction (HCI 2007). It includes papers on topics such as formal models of aspects of cognition, ways to model and reason about human error and error recovery, use of transition systems in formal modelling, visualisation of formal models, modelling of physicality and model-based testing and refinement. All papers submitted to the workshop were reviewed by 3 referees followed by a meta-review process. From the 18 submissions the Program Committee selected 7 papers for long presentation and a further 4 papers for short presentation. Revised versions of the papers selected for long presentation will appear in the post-proceedings of the conference to be published by Elsevier in the series Electronic Notes in Theoretical Computer Science.

2011

UNU-IIST is jointly funded by the government of Macao and the governments of the People's Republic of China and Portugal through a contribution to the UNU Endowment Fund. As well as providing two-thirds of the endowment fund, the Macao authorities also supply UNU-IIST with its office premises and furniture and subsidise fellow accommodation.

2016

Interactive cockpits have been used since the early 00's in many aircraft cockpits, but the use of interactivity still remains limited to non-critical functions even in the most recent aircrafts. Indeed, engineering such interactive systems is still a challenge and their engineering has not reach the Design Assurance Level required for critical functions. In interactive cockpits, interaction takes place through graphical input devices and keyboards (such as the Keyboard Cursor Control Unit in Airbus family) while the behavior of the User Interface (UI) must be compliant with the specifications defined in ARINC 661 standard. The tool-supported three-fold approach presented in this paper proposes means for increasing the assurance level of interactive systems. The approach includes a formal description technique for describing each component of an interactive system (detection and prevention of development faults), a command and monitoring technique dedicated to interactive system...

16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05), 2005

In this paper we present a method for automatically testing interactive multimodal systems 1 . The proposed approach was originally dedicated to synchronous programming which is mainly used for real-time systems. Nevertheless, the behaviour of real-time systems, consisting of cycles starting by reading an external input and ending by issuing an output, is to a certain extent similar to the one of interactive systems. So considered, this paper investigates the use of the Lutess testing environment dedicated to synchronous software for automatically testing multimodal interactive (not necessarily real-time) systems. More precisely, we focus on test data generation based on operational profiles. The main benefit of this approach is that it increases the ability to automatically test an interactive system over long input sequences, according to various use profiles.

1997

Integrated Formal Methods, 2004

A formal, yet user-friendly, test description language could increase the possibilities for automation in the testing phase while at the same time gaining widespread acceptance. Scenario languages are currently one of the most popular formats for describing interactions between possibly distributed components. The question of giving a solid formal basis to scenario languages such as MSC has also received a

Software Quality Journal, 1995

A non-invasive approach for capture and playback (C&P) can be a very useful tool for testing applications endowed of a graphic user interface in local and/or distributed environments, and in general for testing applications without modifying their run-time environment. In the software lifecycle, the phases ofC&P are performed after the application design. Since these are close to the delivery deadline,

Lecture Notes in Computer Science, 2018

Interaction sequences can be used as an abstraction of an interactive system. We can use such models to consider or verify properties of a system for testing purposes. However, interaction sequences have the potential to become unfeasibly long, leading to models which are intractable. We propose a method of reducing the state space of such sequences using the self-containment property. This allows us to hide (and subsequently expand) some of the model describing parts of the system not currently under consideration. Interaction sequences and their models can therefore be used to control the state space size of the models we create as an abstraction of an interactive system.

Springer eBooks, 2003

Proceedings of the …, 1982

We present a case study of an eXperimental integrated interactive System, XS-1, being implemented on small computers. The primary goal of this project is to provide experimental support for design principles that have emerged from a critique of the behavior of today's ...