Integration testing for robotic systems

2018

Nowadays robotic systems are a combination of complex software and hardware components providing sophisticated functionalities. Robotic control systems evolving in an uncertain environment are generally developed case by case for specific deployment platforms. For complex and realistic systems, simulation plays a central role during design by providing testing facilities. In this paper, we propose to model robotic architectures using the modeldriven framework AMSA. In order to facilitate simulation and testing, we propose to incorporate different behaviors in the model through parameterizations. For testing purpose, we define scenarios as a sequence of parameter modifications. From architectural and scenario models, code is generated for the robotic middleware ROS. During simulation, scenarios are used to evaluate different controller behaviors for different contexts. The approach has been experimented for the MDE challenge: a rover must follow another leader rover. The approach all...

IEEE Transactions on Industrial Informatics, 2018

ArXiv, 2019

It is hard to test autonomous robot (AR) software because of the range and diversity of external situations (terrain, obstacles, humans, peer robots) that AR must deal with. Common measures of testing adequacy may not address this diversity. Explicit situation coverage has been proposed as a solution, but there has been little empirical study of its effectiveness. In this paper, we describe an implementation of situation coverage for testing a simple simulated autonomous road vehicle, and evaluate its ability to find seeded faults compared to a random test generation approach. In our experiments, the performance of the two methods is similar, with situation coverage having a very slight advantage. We conclude that situation coverage probably does not have a significant benefit over random generation for the type of simple, research-grade AR software used here. It will likely be valuable when applied to more complex and mature software.

DAAAM International Scientific Book, 2021

Development of the intelligent control software for mobile robots usually requires a lot of changes; the problem is to find evolutionary way of the process. This article describes the study of the problems of testing, integration of parts of hardware and software for mobile service robots (Amur-307 type). The complex of software tools for simplification, integration, testing and speed up debugging of compilation and reliability errors are described. The approach is using emulation of hardware, sensors and electric drivers, additional software modules, which allows creating a complete virtual analogue of robotic complex and speeding up software development, using unit tests.

SDL 2017: Model-Driven Engineering for Future Internet, 2017

Testing is a common technique to assess quality of systems. Regression testing comes into view, when changes are introduced to the system under test and re-running all tests is not practical. Numerous techniques have been introduced to select tests only relevant to a given set of changes. These are typically based on source code, however, modelbased development projects use models as primary artifacts described in various domain-specific languages. Thus, regression test selection should be performed directly on these models. We present a method and a case study on how model-based regression testing can be achieved in the context of autonomous robots. The method uses information from several domain-specific languages for modeling the robot's context and configuration. Our approach is implemented in a prototype tool, and its scalability is evaluated on models from the case study.

2015

Autonomous robots (AR) can get themselves into a wide range of situations, and they do not have a human to look after them in fine detail all the time. When we test autonomous robots, we must therefore care deeply about the range and diversity of the situations in which we have simulated and tested them – we must make sure that the situation coverage of our testing is adequate. Situation coverage measures can be implemented quantitatively, and so unlock a range of automated testing strategies. There are epistemic challenges to justifying the confidence we should attach to test results driven by situation coverage, but they are not fundamentally more difficult than those faced by other coverage criteria.

Proceedings of the 32nd International DAAAM Symposium 2021, 2021

This article describes the study of the problems of testing and integration of the group of mobile service robots (Amur-307 type) hardware and software complex using the virtualization support library. The architecture of the software solution and its features, that simplify integration testing and speed up debugging of compilation and reliability errors are described. An approach to emulation of hardware, sensors and electric driver of robots with manipulators specific to this software and hardware complex is also outlined, which allows creating a complete virtual analogue of it and speeding up software development (especially in terms of routing a robotic complex) using unit tests.

In this paper, we propose a mechanism allowing to evaluate the schedulability of a robotic software architecture, and then validate its real-time properties. The robotic software architecture is described through a Domain Speci c Language (DSL), MAUVE, that allows to model communicating components. The evaluation of schedulability of the architecture consists in rst computing the Worst-Case Execution Time (WCET) of the elementary functions of the components. Then the Worst Case Response Time (WCRT) of the component is computed from the elementary WCET and the component models, allowing to validate the schedulatiblity of the architecture. We illustrate our methodology on the evaluation of a control architecture for a ground mobile robot.

Software Quality Journal, 2008

This paper considers the problem of integration testing the components of a timed distributed software system. We assume that communication between the components is specified using timed interface automata and use computational tree logic (CTL) to define communication-based coverage criteria that refer to send-and receive-statements and communication paths. The proposed method enables testers to focus during component integration on such parts of the specification, e.g. behaviour specifications or Markovian usage models, that are involved in the communication between components to be integrated. A more specific application area of this approach is the integration of test-models, e.g. a transmission gear can be tested based on separated models for the driver behaviour, the engine condition, and the mechanical and hydraulical transmission states. Given such a state-based specification of a distributed system and a concrete coverage goal, a model checker is used in order to determine the coverage or generate test sequences that achieve the goal. Given the generated test sequences we derive a partial test-model of the components from which the test sequences were derived. The partial model can be used to drive further testing and can also be used as the basis for producing additional partial models in incremental integration testing. While the process of deriving the test sequences

International Journal of Software Engineering & Applications (IJSEA), 2024

In the rapidly evolving realm of software development, Robotic Process Automation (RPA) has emerged as a pivotal technology that holds the potential to redefine the landscape of automation testing. This scholarly article delves into the integration of RPA within the realm of automation testing, elucidating its capacity to augment efficiency and precision. The primary aim of this investigation is to evaluate the efficacy of RPA tools in automating repetitive and time-consuming testing procedures that have traditionally been conducted by human testers. Through a comparative analysis of diverse RPA tools and a series of empirical case studies, we substantiate how RPA can substantially diminish manual exertion while concurrently expanding coverage and dependability in test scenarios. Our findings reveal that RPA, when effectively implemented, not only expedites the testing cycle but also contributes to the production of more robust and dependable software products. The article concludes by deliberating on the broader implications of these findings for the field of software testing, with an emphasis on the necessity for organizations to adapt to RPA-driven methodologies. This scholarly investigation lays the groundwork for future research endeavors aimed at optimizing RPA strategies and their integration with other emerging technologies in the realm of automation testing.

2006

is an open source package that provides a high-resolution, physics based simulation of robotic platforms. The package provides models of several common robotic platforms and sensors as well as sample worlds and a socket interface into a commonly used commercial-off-the-shelf (COTS) simulation package. Initially introduced to support the development of search and rescue robots, USARSim has proved to be a tool with a broader scope, from robot education to human robot interfaces, including cooperation, and more. During Robocup 2006, a new competition based on USARSim will be held in the context of the urban search and rescue competitions.

2002

As robotic technologies mature, we are moving from simple systems that roam our laboratories to heteroge- neous groups of systems that operate in complex non- structured environments. The novel and extremely com- plex nature of these autonomous systems generates a great deal of subsystem interdependencies that makes team, in- dividual system, and subsystem validation and perfor- mance measurement dicult. Simple

This paper considers the problem of integration testing the components of a timed distributed software system. We assume that communication between the components is specified using timed interface automata and use computational tree logic (CTL) to define communication-based coverage criteria that refer to send-and receive-statements and communication paths. The proposed method enables testers to focus during component integration on such parts of the specification, e.g. behaviour specifications or Markovian usage models, that are involved in the communication between components to be integrated. A more specific application area of this approach is the integration of test-models, e.g. a transmission gear can be tested based on separated models for the driver behaviour, the engine condition, and the mechanical and hydraulical transmission states. Given such a state-based specification of a distributed system and a concrete coverage goal, a model checker is used in order to determine the coverage or generate test sequences that achieve the goal. Given the generated test sequences we derive a partial test-model of the components from which the test sequences are derived. The partial model can be used to drive further testing and can also be used as the basis for producing additional partial models in incremental integration testing. While the process of deriving the test sequences could suffer from a combinatorial explosion, the effort required to generate the partial model is polynomial in the number of test sequences and their length. Thus, where it is not feasible to produce test sequences that achieve a given type of coverage it is still possible to produce a partial model on the basis of test sequences generated to achieve some other criterion. As a result, the process of generating a partial model has the potential to scale to large industrial software systems. While a particular model checker, UPPAAL, was used, it should be relatively straightforward to adapt the approach for use with other CTL based model checkers. A potential additional benefit of the approach is that it provides a visual description of the state-based testing of distributed systems, which may be beneficial in other contexts such as education and comprehension.

Software & Systems Modeling

Robots are becoming ubiquitous: from vacuum cleaners to driverless cars, there is a wide variety of applications, many with potential safety hazards. The work presented in this paper proposes a set of constructs suitable for both modelling robotic applications and supporting verification via model checking and theorem proving. Our goal is to support roboticists in writing models and applying modern verification techniques using a language familiar to them. To that end, we present RoboChart, a domain-specific modelling language based on UML, but with a restricted set of constructs to enable a simplified semantics and automated reasoning. We present the RoboChart metamodel, its well-formedness rules, and its process-algebraic semantics. We discuss verification based on these foundations using an implementation of RoboChart and its semantics as a set of Eclipse plug-ins called RoboTool. Keywords State machines • Formal semantics • Process algebra • CSP • Model checking • Timed properties • Domain-specific language for robotics Communicated by Dr. Jeff Gray.

Proceedings of the 10th Performance Metrics for Intelligent Systems Workshop on - PerMIS '10, 2010

Robots must possess certain sets of capabilities to suit critical operations such as emergency responses. In the mobility function, ground robots must be able to handle many types of obstacles and terrain complexities, including traversing and negotiating positive and negative obstacles, various types of floor surfaces or terrains, and confined passageways. Additional mobility requirements include the ability to sustain specified speeds and to tow payloads with different weights. Standard test methods are required to evaluate how well candidate robots meet these requirements. A set of test methods focused on evaluating the mobility function has been collected into a test suite. Likewise, in other functions such as sensing, communication, manipulation, energy/power, Human-System Interaction (HSI), logistics, and safety, corresponding test suites are required. Also needed are test suites for aerial and aquatic robots. Under the sponsorship of DHS, NIST researchers are collaborating with others to establish such a collection of test suites under the standards development organization ASTM International. Apparatuses must be set up to challenge specific robot capabilities in repeatable ways to facilitate direct comparison of different robot models as well as particular configurations of similar robot models.