1 National ICT Australia Ltd; Australian Technology Park, Sydney 1430 NSW, Australia; {barbara.ki... more 1 National ICT Australia Ltd; Australian Technology Park, Sydney 1430 NSW, Australia; {barbara.kitchenham, hiyam.al-kilidar, muhammad.alibabar, mike.berry, karl.cox, jacky.keung, felicia.kurniawata, mark.staples, he.zhang, liming.zhu}@nicta.com.au ...
In this paper, we present an approach to improve the software architecture evaluation process by ... more In this paper, we present an approach to improve the software architecture evaluation process by systematically extracting and appropriately documenting architecturally significant information from software architecture and design patterns; we are interested in only two pieces of information found in software patterns: general scenarios and architectural tactics. General scenarios distilled from patterns not only assist stakeholders in developing concrete scenarios during a scenario-based architecture evaluation, but can also help an architect select and calibrate a quality attribute reasoning framework. Architectural tactics in patterns are used as a means of manipulating independent parameters in the reasoning framework to achieve the desired quality. Moreover, we believe if we use general scenarios and tactics extracted from patterns in an architectural evaluation, the results of that evaluation can be used as an evidence to validate the pattern's claim with respect to the quality attributes. We demonstrate our approach by using EJB architecture usage patterns. We contend that this approach can be used to analyze and validate any architecture pattern.
Despite the recognition of the importance of capturing and reusing architecture knowledge, there ... more Despite the recognition of the importance of capturing and reusing architecture knowledge, there is no suitable support mechanism. We have developed a conceptual framework to provide appropriate guidance and tool support for making tacit or informally described architecture knowledge explicit. This framework identifies different approaches to capturing implicit architecture knowledge. We discuss different usages of the captured knowledge to improve the effectiveness of architecture processes. This paper also presents a prototype of a web-based architecture knowledge management tool to support the storage and retrieval of the captured knowledge. The paper concludes with open issues that we plan to address in order to successfully transfer this support mechanism for capturing and using architecture knowledge to the industry.
Scenario-based methods for evaluating software architecture require a large number of stakeholder... more Scenario-based methods for evaluating software architecture require a large number of stakeholders to be collocated for evaluation meetings. Collocating stakeholders is often an expensive exercise. To reduce expense, we have proposed a framework for supporting software architecture evaluation process using groupware systems. This paper presents a controlled experiment that we conducted to assess the effectiveness of one of the key activities, developing scenario profiles, of the proposed groupware-supported process of evaluating software architecture. We used a cross-over experiment involving 32 teams of three 3rd and 4th year undergraduate students. We found that the quality of scenario profiles developed by distributed teams using a groupware tool were significantly better than the quality of scenario profiles developed by face-to-face teams (p < 0.001). However, questionnaires indicated that most participants preferred the face-to-face arrangement (82%) and 60% thought the distributed meetings were less efficient. We conclude that distributed meetings for developing scenario profiles are extremely effective but that tool support must be of a high standard or participants will not find distributed meetings acceptable.
This paper describes a tool for managing architectural knowledge and rationale. The tool has been... more This paper describes a tool for managing architectural knowledge and rationale. The tool has been developed to support a framework for capturing and using architectural knowledge to improve the architecture process. This paper describes the main architectural components and features of the tool. The paper also provides examples of using the tool for supporting wellknown architecture design and analysis methods.
Software architecture (SA) evaluation is a quality assurance technique that is increasingly attra... more Software architecture (SA) evaluation is a quality assurance technique that is increasingly attracting significant research and commercial interests. A number of SA evaluation methods have been developed. Most of these methods are scenario-based, which relies on the quality of the scenarios used for the evaluation. Most of the existing techniques for developing scenarios use stakeholders and requirements documents as main sources of collecting scenarios. Recently, architectures of large software systems are usually composed of patterns and styles. One of the purposes of using patterns is to develop systems with predictable quality attributes. Since patterns are documented in a format that requires the inclusion of problem, solution and quality consequences, we observed that scenarios are, though as informal text, pervasive in patterns description, which can be extracted and documented for the SA evaluation. Thus, we claim that the patterns can be another source of collecting quality attributes sensitive scenarios. This position paper presents arguments and examples to support our claim.
Management of architecture knowledge is vital for improving an organization's architectural capab... more Management of architecture knowledge is vital for improving an organization's architectural capabilities. Despite the recognition of the importance of capturing and reusing architecture knowledge, there is no suitable support mechanism. We propose a conceptual framework for providing appropriate guidance and tool support for making tacit or informally described architecture knowledge explicit. This framework identifies different approaches to capturing implicit architecture knowledge. We discuss different usages of the captured knowledge to improve the effectiveness of architecting process. The report also presents a brief description of a prototype of a web-based architecture knowledge management tool to support the storage and retrieval of the captured knowledge. The report concludes with open issues that we plan to address in order to successfully transfer this support mechanism for capturing and using architecture knowledge to the industry.
Software engineering community has proposed several methods to evaluate software architectures wi... more Software engineering community has proposed several methods to evaluate software architectures with respect to desired quality attributes such as maintainability, performance, and so on. There is, however, little effort on systematically comparing such methods to discover similarities and differences between existing approaches. In this paper, we compare four well known scenario-based SA evaluation methods using an evaluation framework. The framework considers each method from the point of view of method context, stakeholders, structure, and reliability. The comparison reveals that most of the studied methods are structurally similar but there are a number of differences among their activities and techniques. Therefore, some methods overlap, which guides us to identify five common activities that can form a generic process model.
Many claims have been made about the consequences of not documenting design rationale. The genera... more Many claims have been made about the consequences of not documenting design rationale. The general perception is that designers and architects usually do not fully understand the critical role of systematic use and capture of design rationale. However, there is to date little empirical evidence available on what design rationale mean to practitioners, how valuable they consider it, and how they use and document it during the design process. This paper reports a survey of practitioners to probe their perception of the value of design rationale and how they use and document the background knowledge related to their design decisions. Based on 81 valid responses, this study has discovered that practitioners recognize the importance of documenting design rationale and frequently use them to reason about their design choices. However, they have indicated barriers to the use and documentation of design rationale. Based on the findings, we conclude that further research is needed to develop methodology and tool support for design rationale capture and usage. Furthermore, we put forward some specific research questions about design rationale that could be further investigated to benefit industry practice.
Recent research suggests that architectural knowledge, such as design decisions, is important and... more Recent research suggests that architectural knowledge, such as design decisions, is important and should be recorded alongside the architecture description. Different approaches have emerged to support such architectural knowledge (AK) management activities. However, there are different notions of and emphasis on what and how architectural activities should be supported. This is reflected in the design and implementation of existing AK tools. To understand the current status of software architecture knowledge engineering and future research trends, this paper compares five architectural knowledge management tools and the support they provide in the architecture life-cycle. The comparison is based on an evaluation framework defined by a set of 10 criteria. The results of the comparison provide insights into the current focus of architectural knowledge management support, their advantages, deficiencies, and conformance to the current architectural description standard. Based on the outcome of this comparison a research agenda is proposed for future work on AK tools.
1 National ICT Australia Ltd; Australian Technology Park, Sydney 1430 NSW, Australia; {barbara.ki... more 1 National ICT Australia Ltd; Australian Technology Park, Sydney 1430 NSW, Australia; {barbara.kitchenham, hiyam.al-kilidar, muhammad.alibabar, mike.berry, karl.cox, jacky.keung, felicia.kurniawata, mark.staples, he.zhang, liming.zhu}@nicta.com.au ...
In this paper, we present an approach to improve the software architecture evaluation process by ... more In this paper, we present an approach to improve the software architecture evaluation process by systematically extracting and appropriately documenting architecturally significant information from software architecture and design patterns; we are interested in only two pieces of information found in software patterns: general scenarios and architectural tactics. General scenarios distilled from patterns not only assist stakeholders in developing concrete scenarios during a scenario-based architecture evaluation, but can also help an architect select and calibrate a quality attribute reasoning framework. Architectural tactics in patterns are used as a means of manipulating independent parameters in the reasoning framework to achieve the desired quality. Moreover, we believe if we use general scenarios and tactics extracted from patterns in an architectural evaluation, the results of that evaluation can be used as an evidence to validate the pattern's claim with respect to the quality attributes. We demonstrate our approach by using EJB architecture usage patterns. We contend that this approach can be used to analyze and validate any architecture pattern.
Despite the recognition of the importance of capturing and reusing architecture knowledge, there ... more Despite the recognition of the importance of capturing and reusing architecture knowledge, there is no suitable support mechanism. We have developed a conceptual framework to provide appropriate guidance and tool support for making tacit or informally described architecture knowledge explicit. This framework identifies different approaches to capturing implicit architecture knowledge. We discuss different usages of the captured knowledge to improve the effectiveness of architecture processes. This paper also presents a prototype of a web-based architecture knowledge management tool to support the storage and retrieval of the captured knowledge. The paper concludes with open issues that we plan to address in order to successfully transfer this support mechanism for capturing and using architecture knowledge to the industry.
Scenario-based methods for evaluating software architecture require a large number of stakeholder... more Scenario-based methods for evaluating software architecture require a large number of stakeholders to be collocated for evaluation meetings. Collocating stakeholders is often an expensive exercise. To reduce expense, we have proposed a framework for supporting software architecture evaluation process using groupware systems. This paper presents a controlled experiment that we conducted to assess the effectiveness of one of the key activities, developing scenario profiles, of the proposed groupware-supported process of evaluating software architecture. We used a cross-over experiment involving 32 teams of three 3rd and 4th year undergraduate students. We found that the quality of scenario profiles developed by distributed teams using a groupware tool were significantly better than the quality of scenario profiles developed by face-to-face teams (p < 0.001). However, questionnaires indicated that most participants preferred the face-to-face arrangement (82%) and 60% thought the distributed meetings were less efficient. We conclude that distributed meetings for developing scenario profiles are extremely effective but that tool support must be of a high standard or participants will not find distributed meetings acceptable.
This paper describes a tool for managing architectural knowledge and rationale. The tool has been... more This paper describes a tool for managing architectural knowledge and rationale. The tool has been developed to support a framework for capturing and using architectural knowledge to improve the architecture process. This paper describes the main architectural components and features of the tool. The paper also provides examples of using the tool for supporting wellknown architecture design and analysis methods.
Software architecture (SA) evaluation is a quality assurance technique that is increasingly attra... more Software architecture (SA) evaluation is a quality assurance technique that is increasingly attracting significant research and commercial interests. A number of SA evaluation methods have been developed. Most of these methods are scenario-based, which relies on the quality of the scenarios used for the evaluation. Most of the existing techniques for developing scenarios use stakeholders and requirements documents as main sources of collecting scenarios. Recently, architectures of large software systems are usually composed of patterns and styles. One of the purposes of using patterns is to develop systems with predictable quality attributes. Since patterns are documented in a format that requires the inclusion of problem, solution and quality consequences, we observed that scenarios are, though as informal text, pervasive in patterns description, which can be extracted and documented for the SA evaluation. Thus, we claim that the patterns can be another source of collecting quality attributes sensitive scenarios. This position paper presents arguments and examples to support our claim.
Management of architecture knowledge is vital for improving an organization's architectural capab... more Management of architecture knowledge is vital for improving an organization's architectural capabilities. Despite the recognition of the importance of capturing and reusing architecture knowledge, there is no suitable support mechanism. We propose a conceptual framework for providing appropriate guidance and tool support for making tacit or informally described architecture knowledge explicit. This framework identifies different approaches to capturing implicit architecture knowledge. We discuss different usages of the captured knowledge to improve the effectiveness of architecting process. The report also presents a brief description of a prototype of a web-based architecture knowledge management tool to support the storage and retrieval of the captured knowledge. The report concludes with open issues that we plan to address in order to successfully transfer this support mechanism for capturing and using architecture knowledge to the industry.
Software engineering community has proposed several methods to evaluate software architectures wi... more Software engineering community has proposed several methods to evaluate software architectures with respect to desired quality attributes such as maintainability, performance, and so on. There is, however, little effort on systematically comparing such methods to discover similarities and differences between existing approaches. In this paper, we compare four well known scenario-based SA evaluation methods using an evaluation framework. The framework considers each method from the point of view of method context, stakeholders, structure, and reliability. The comparison reveals that most of the studied methods are structurally similar but there are a number of differences among their activities and techniques. Therefore, some methods overlap, which guides us to identify five common activities that can form a generic process model.
Many claims have been made about the consequences of not documenting design rationale. The genera... more Many claims have been made about the consequences of not documenting design rationale. The general perception is that designers and architects usually do not fully understand the critical role of systematic use and capture of design rationale. However, there is to date little empirical evidence available on what design rationale mean to practitioners, how valuable they consider it, and how they use and document it during the design process. This paper reports a survey of practitioners to probe their perception of the value of design rationale and how they use and document the background knowledge related to their design decisions. Based on 81 valid responses, this study has discovered that practitioners recognize the importance of documenting design rationale and frequently use them to reason about their design choices. However, they have indicated barriers to the use and documentation of design rationale. Based on the findings, we conclude that further research is needed to develop methodology and tool support for design rationale capture and usage. Furthermore, we put forward some specific research questions about design rationale that could be further investigated to benefit industry practice.
Recent research suggests that architectural knowledge, such as design decisions, is important and... more Recent research suggests that architectural knowledge, such as design decisions, is important and should be recorded alongside the architecture description. Different approaches have emerged to support such architectural knowledge (AK) management activities. However, there are different notions of and emphasis on what and how architectural activities should be supported. This is reflected in the design and implementation of existing AK tools. To understand the current status of software architecture knowledge engineering and future research trends, this paper compares five architectural knowledge management tools and the support they provide in the architecture life-cycle. The comparison is based on an evaluation framework defined by a set of 10 criteria. The results of the comparison provide insights into the current focus of architectural knowledge management support, their advantages, deficiencies, and conformance to the current architectural description standard. Based on the outcome of this comparison a research agenda is proposed for future work on AK tools.
Uploads
Papers by Muhammad Babar