SOFTWARE ENGINEERING AND PROJECT MANAGEMENT(BCSS01) MODULE 3 CHAPTER 1---AGILE DEVELOPMENT 1.1 WHAT IS AGILITY? Agile is a software development methodology to build software incrementally using short iterations of 1 to 4 weeks so that the development process is aligned with the changing business needs. An agile team is a nimble team able to appropriately respond to changes. Change is what software development is very much about. Changes in the software being built, changes to the team members, changes because of new technology, changes of all kinds that may have an impact on the product they build or the project that creates the product. Support for changes should be built- in everything we do in software, something we embrace because it is the heart and soul of software. An agile team recognizes that software is developed by individuals working in teams and that the skills of these people, their ability to collaborate is at the core for the success of the project. The aim of agile process is to deliver the working model of software quickly to the customer For example: Extreme programming is the best known of agile process 1.2 AGILITY AND THE COST OF CHANGE In conventional software development the cost of change increases non linearly as a project progresses (Fig Solid Black curve). An agile process reduces the cost of change because software is released in increments and change can be better controlled within an increment. Agility argue that a well-designed agile process “flattens” the cost of change curve shown in following figure (shaded, solid curve), allowing a software team to accommodate changes late in a software project without dramatic cost and time impact, When incremental delivery is coupled with other agile practices such as continuous unit testing and pair programming, the cost of making a change is attenuated(reduced). Although debate about the degree to which the cost curve flattens is ongoing, there is evidence to suggest that a significant reduction in the cost of change can be achieved. application, design, architecture etc. The verification process involves activities like reviews, walk-throughs and inspection.EERING AND PROJECT MANAGEMENT(BCSS01) Cou! of chenge Cost of change Using agile processes, Development cost 1.3 WHAT IS AN AGILE PROCESS? Any agile software process is characterized in a manner that addresses a number of key sumptions about the majority of software projects It is difficult to predict in advance which software requirements will persist and which will change. It is equally difficult to predict how customer priorities will change as the project proceeds, For many types of software, design and construction are interleaved. That is, both activities should be performed in tandem so that design models are proven as they are created. It is difficult to predict how much design is necessary before construction is used to prove the design. 3. Analysis, design, construction, and testing are not as predictable. Given these three assumptions, an important question arises: 1) How do we create a process that can manage unpredictability It lies in process adaptability. An agile process, therefore, must be adaptable. But continual adaptation without forward progress accomplishes little. Therefore, an agile software process must adapt incrementally. To accomplish incremental adaptation, an agile team requires customer feedback. An effective catalyst for customer feedback is an operational prototype or a portion of an operational system, Hence, an incremental development strategy should be instituted. Software increments must be delivered in short time periods so that adaptation keeps pace with change. This iterative approach enables the customer to evaluate the software increment regularly, provide necessary feedback to the sofiware team, and influence the process adaptations that are made to accommodate the feedback.SOFTWARE ENGINEERING AND PROJECT MANAGEMENT(BCSS01) 1.3.1 Agility Pi Agility principles for those who want to achieve agility: Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. Welcome changing requirements, even late in development. Agile processes hamess change for the customer's competitive advantage. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. Business people and developers must work together daily throughout the project. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation Working software is the primary measure of progress processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. Continuous attention to technical excellence and good design enhances agility. Simplicity—the art of maximizing the amount of work not done—is essential . The best architectures, requirements, and designs emerge from self- organizing teams . At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. Not every agile process model applies these 12 principles with equal weight, and some models choose to ignore (or at least downplay) the importance of one or more of the principles. 1.3.2 The Politics of Agile Development There is debate about the benefits and applicability of agile software development as opposed to more conventional software engineering processes (produces documents rather than working product). Even within the agile, there are many proposed process models each with a different approach to the agility, 1.3.3 Human Factors Agile development focuses on the talents and skills of individuals, molding the process to specific people and teams.” The key point in this statement is that the process molds to the needs of the people and team.SOFTWARE ENGINEERING AND PROJECT MANAGEMENT(BCSS01) Competence. In an agile development context, “competence” encompasses innate talent, specific software-related skills, and overall knowledge of the process that the team has chosen to apply. Skill and knowledge of process can and should be taught to all people who serve as agile team members. Common focus. Although members of the agile team may perform different tasks and bring different skills to the project, all should be focused on one goal—to deliver a working software increment to the customer within the time promised, To achieve this goal, the team will also focus on continual adaptations (small and large) that will make the process fit the needs of the team. Collaboration. Software engineering (regardless of process) is about assessing, analyzing, and using information that is communicated to the software team; creating information that will help all stakeholders understand the work of the team; and building information (computer software and relevant databases) that provides business value for the customer. To accomplish these tasks, team members must collaborate—with one another and all other stakeholders. Decision-making ability. Any good software team (including agile teams) must be allowed the freedom to control its own destiny. This implies that the team is given autonomy decision-making authority for both technical and project issues. Fuzzy problem-solving ability. Software managers must recognize that the agile team will continually have to deal with ambiguity and will continually be buffeted by change Mutual trust and respect. The agile team must become what DeMarco and Lister call a “jelled” team. A jelled team exhibits the trust and respect that are necessary to make them “so strongly knit that the whole is greater than the sum of the parts.” Self-organization. In the context of agile development, self-organization implies three things (1) the agile team organizes itself for the work to be done, (2) the team organizes the process to best accommodate its local environment, (3) the team organizes the work schedule to best achieve delivery of the software increment. Self-organization has a number of technical :, but more importantly, it serves to improve collaboration and boost team morale.SOFTWARE ENGINEERING AND PROJECT MANAGEMENT(BCSS01) 1.4 EXTREME PROGRAMMING (XP) Extreme Programming (XP), the most widely used approach to agile software development, emphasizes business results first and takes an incremental, get-something-started approach to building the product, using continual testing and revision. XP proposed by Kent beck during the late 1980's. 1.4.1 XP Values Beck defines a set of five values that establish a foundation for all work performed as part of XP— communication, simplicity, feedback, courage, and respect. Each of these values is used as a driver for specific XP activities, actions, and tasks In order to achieve effective communication between software engineers and other stakeholders, XP emphasizes close, yet informal collaboration between customers and developers, the establishment of effective metaphors for communicating important concepts, continuous feedback, and the avoidance of voluminous documentation as a communication medium. To achieve simplicity, XP restricts developers to design only for immediate needs, rather than consider future needs. The intent is to create a simple design that can be easily implemented in code. If the design must be improved, it can be refactored at a later time. Feedback is derived from three sources: the implemented software itself, the customer, and other software team members. By designing and implementing an effective testing strategy the software provides the agile team with feedback. XP makes use of the unit test as its primary testing tactic. As each class is developed, the team develops a unit test to exercise each operation accor to its specified functionality. Beck argues that strict adherence to certain XP practices demands courage. A better word might be discipline, An agile XP team must have the discipline (courage) to design for today, recognizing that future requirements may change dramatically, thereby demanding substantial rework of the design and implemented code. By following each of these values, the agile team inculeates respect among its members, between other stakeholders and team members, and indirectly, for the software itself. As they achieve successful delivery of software increments, the team develops growing respect for the XP process.EERING AND PROJECT MANAGEMENT(BCSS01) .2 The XP Process Extreme Programming uses an object-oriented approach as its preferred development paradigm and encompasses a set of rules and practices that occur within the context of four framework act 2 planning, design, coding, and testing. Following figure illustrates the XP process and notes some of the key ideas and tasks that are associated with each framework activity. mple design spike solutions CRC cords Prototypes \ por programming Releose “projec vleciy computed | | cOnfinvovs integration plance testing Fig : The Extreme Programming process Key XP activities are 1) Planning. The planning activity begins with listening—a requirements gathering activity * Listening leads to the creation of a set of “stories” (also called user stories) that describe required output, features, and functionality for software to be built Each story is written by the customer and is placed on an index card. The customer assigns a value (i.., a priority) to the story based on the overall business value of the feature or function. Members of the XP team then assess each story and assign a cost— measured in development weeks—to it. If the story is estimated to require more than three development weeks, the story into smaller stories and the assignment of value and cost occurs again, It is important to note that new stories can be written at any time.EERING AND PROJECT MANAGEMENT(BCSS501) The stories with highest value will be moved up in the schedule and implemented first 2) Design. XP design rigorously follows the KIS (keep it simple) principle. If a difficult design problem is encountered as part of the design of a story, XP recommends the immediate creation of an operational prototype of that portion of the design. Called a spike solution, XP encourages refactoring—a construction technique that is also a method for design optimization. Refactoring is the process of changing a software system in a way that it does not change the external behavior of the code and improves the internal structure. 3) Coding. After stories are developed and preliminary design work is done, the team does not move to code, develops a series of unit tests for each of the stories that is to be included in the current release (software increment). Once the unit test has been created, the developer is better able to focus on what must be implemented to pass the test Once the code is complete, it can be unit-tested immediately, and providing feedback to the developers. A key concept during the coding activity is pair programming. i.e... two people work together at one computer workstation to create code for a story. As pair programmers complete their work, the code they develop is integrated with the work of others. 4) Testing. The creation of unit tests before coding commences is a key element of the XP approach. The unit tests that are created should be implemented using a framework that enables them to be automated. This encourages a regression testing strategy whenever code is modified. As the individual unit tests are organized into a “universal testing suite” integration and validation testing of the system can occur on a daily basis. This provides the XP team with a continual indication of progress and also can raise warning flags early if things go awry. Wells states: “Fixing small problems every few hours takes less time than fixing huge problems just before the deadline.”SOFTWARE ENGINEERING AND PROJECT MANAGEMENT(BCSS01) XP acceptance tests, also called customer tests, are specified by the customer and focus on overall system features and functionality that are visible and reviewable by the customer. Acceptance tests are derived from user stories that have been implemented as part of a sofiware release. .4.3 Industrial XP Joshua Kerievsky describes Industrial Extreme Programming (IXP) in the following manner: “IXP is an organic evolution of XP. It is imbued with XP’s minimalist, customer-centric, test-driven spirit. IXP differs most from the original XP in its greater inclusion of management, its expanded role for customers, and its upgraded technical practices.” IXP incorporates six new practices that are designed to help ensure that an XP project works successfully for significant projects within a large organization Readiness assessment. Prior to the initiation of an IXP project, the organization should conduct a readiness assessment, The assessment ascertains whether (1) an appropriate development environment exists to support IXP, (2) the team will be populated by the proper set of stakeholders, (3) the organization has a distinet quality program and supports continuous improvement, (4) the organizational culture will support the new values of an agile team, and (5) the broader project community will be populated appropriately. Project community. Classic XP suggests that the right people be used to populate the agile team to ensure success, The implication is that people on the team must be well-trained, adaptable and skilled, and have the proper temperament to contribute to a self-organizing team, When XP is to be applied for a significant project in a large organization, the concept of the “team” should morph into that of a community. A community may have a technologist and customers who are central to the success of a project as well as many other stakeholders (e.g., legal staff, quality auditors, manufacturing or sales types) who “are often at the periphery of an IXP project yet they may play important roles on the project”. In IXP, the community members and their roles should be explicitly defined and mechanisms for communication and coordination between community members should be established. Project chartering. The IXP team assesses the project itself to determine whether an appropriate business justification for the project exists and whether the project will further the overall goals and objectives of the organization, Chartering also examines the context of the project to determine how it complements, extends, or replaces existing systems or processes, Test-driven management. An IXP project requires measurable criteria for assessing the state of the project and the progress that has been made to date. Test-driven management establishes a series of measurable “destinations” and then defines mechanisms for determining whether or not these destinations have been reachedSOFTWARE ENGINEERING AND PROJECT MANAGEMENT(BCSS01) Retrospectives. An IXP team conducts a specialized technical review after a software increment is delivered. Called a retrospective, the review examines “issues, events, and lessons-learned” across a software increment and/or the entire software release. The intent is to improve the IXP process. Continuous learning, Because learning is a vital part of continuous process improvement, members of the XP team are encouraged (and possibly, incented) to learn new methods and techniques that ean lead to a higher quality product. 1.4.4 The XP Debate Requirements volatility. The customer is an active member of the XP team, changes to requirements are requested informally. As a consequence, the scope of the project can change and earlier work may have to be modified to accommodate current needs. Conflicting customer needs. Many projects have multiple customers, each with his own set of needs. Requirements are expressed informally. User stories and acceptance tests are the only explicit manifestation of requirements in XP. specification is often needed to remove inconsistencies, and errors before the system is built Lack of formal design: when complex systems are built, design must have the overall structure of the software then it will exhibit quality. 1.5 OTHER AGILE PROCESS MODELS Other agile process models have been proposed and are in use across the industry. Among the most common are Adaptive Software Development (ASD) Serum Dynamic Systems Development Method (DSDM) Crystal Feature Drive Development (FDD) Lean Software Development (LSD) Agile Modeling (AM) Agile Unified Process (AUP)SOFTWARE ENGINEERING AND PROJECT MANAGEMENT(BCSS501) Adaptive Software Development (ASD) Adaptive Software Development (ASD) has been proposed by Jim Highsmith as a technique for building complex software and systems. The philosophical underpinnings of ASD focus on human collaboration and team self-organization. High smith argues that an agile, adaptive development approach based on collaboration is “as much a source of order in our complex interactions as discipline and engineering.” He defines an ASD “life cycle” that incorporates three phases, speculation, collaboration, and learning. comparent inplemontd/ Ist Fig: Adaptive software development During speculation, the project is initiated and adaptive cycle planning is conducted. Adaptive cycle planning uses project initiation information—the customer's mission statement, project constraints (¢.g., delivery dates or user descriptions), and basic requirements—to define the set of release cycles (software increments) that will be required for the project. Motivated people use collaboration in a way that multiplies their talent and creative output beyond their absolute numbers. This approach is a recurring theme in all agile methods. But collaboration is not easy. It encompasses communication and teamwork, but it also emphasizes individualism, because individual creativity plays an important role in collaborative thinking. It is, above all, a matter of trust. People working together must trust one another to (1) criticize without animosity, (2) assist without resentment, (3) work as hard as or harder than they do, (4) have the skill set to contribute to the work at hand, and (5) communicate problems or concems in a way that leads to effective action As members of an ASD team begin to develop the components that are part of an adaptive cycle, the emphasis is on “learning” as much as it is on progress toward a completed cycle.SOFTWARE ENGINEERING AND PROJECT MANAGEMENT(BCSS01) ASD teams lear in three ways: focus groups, technical reviews, and project postmortems. ASD’s overall emphasis on the dynamics of self-organizing teams, interpersonal collaboration, and individual and team learning yield software project teams that have a much higher likelihood of success. Serum Scrum is an agile software development method that was conceived by Jeff Sutherland and his development team in the early 1990s. Serum principles are consistent with the agile manifesto and are used to guide development activities within a process that incorporates the following framework activities: requirements, analysis, design, evolution, and delivery. Within each framework activity, work tasks occur within a process pattern called a sprint. The work conducted within a sprint is adapted to the problem at hand and is defined and often modified in real time by the Scrum team. The overall flow of the Scrum process is illustrated in following figure. SI) g — 4 ieee gen te ) What did you do 2) Bo you hove ony obstacle? Sprint Backlog: Backlog 3) Wrer wil you beens ext Fee ‘om 20 doy mowing? ossgned expanded Product Backlog: Priortzed prodoct features desied by the cuslomer New functionality is demonstrated ‘at end of sprint Scrum emphasizes the use of a set of software process patterns that have proven effective for projects with tight timelines, changing requirements, and business criticality. Each of these process patterns defines a set of development actions: Backlog—a prioritized list of project requirements or features that provide business value for the customer. Items can be added to the backlog at any time. The product manager assesses the backlog and updates priorities as required.SOFTWARE ENGINEERING AND PROJECT MANAGEMENT(BCSS01) Sprints—consist of work units that are required to achieve a requirement defined in the backlog that must be fit into a predefined time-box (typically 30 days). Changes (e.g., backlog work items) are not introduced during the sprint. Hence, the sprint allows team members to work in a short-term, but stable environment Scrum meetings—are short (typically 15 minutes) meetings held daily by the Scrum team. Three key questions are asked and answered by all team members © What did you do since the last team meeting? © What obstacles are you encountering? © What do you plan to accomplish by the next team meeting? A team leader, called a Serum master, leads the meeting and assesses the responses from each person, The Scrum meeting helps the team to uncover potential problems as early as possible. Also. these daily meetings lead to “knowledge socialization” Demos—deliver the software increment to the customer so that functionality that has been implemented can be demonstrated and evaluated by the customer. It is important to note that the demo may not contain all planned functionality, but rather those functions that can be delivered within the time-box that was established. Dynamic Systems Development Method (DSDM) The Dynamic Systems Development Method (DSDM) is an agile software development approach that “provides a framework for building and maintaining systems which meet tight time constraints through the use of incremental prototyping in a controlled project environment” The DSDM philosophy is borrowed from a modified version of the Pareto principle—80 percent of an application can be delivered in 20 percent of the time. It would take to deliver the complete (100 percent) application. DSDM is an iterative software process in which each iteration follows the 80 percent rule. That is, only enough work is required for each increment to facilitate movement to the next increment, The remaining detail can be completed later when more business requirements are known or changes have been requested and accommodated, ‘The DSDM life cycle that defines three different iterative cycles, preceded by two additional life eycle activities + Feasibility study—establishes the basic business requirements and constraints associated with the application to be built and then assesses whether the application is a viable candidate for the DSDM processSOFTWARE ENGINEERING AND PROJECT MANAGEMENT(BCSS01) Business study—establishes the functional and information requirements that will allow the application to provide business value; also, defines the basie application architecture and identifies the maintainability requirements for the application. Functional model iteration—produces a set of incremental prototypes that demonstrate functionality for the custome intent during this iterative cycle is to gather additional requirements by eliciting feedback from users as they exercise the prototype. Design and build iteration—revisits prototypes built during functional model iteration to ensure that each has been engineered in a manner that will enable it to provide operational business value for end users. In some cases, functional model iteration and design and build iteration occur concurrently. Implementation—places the latest software increment into the operational environment, It should be noted that (1) the increment may not be 100 percent complete or (2) changes may be requested as the increment is put into place. In either case, DSDM development work continues by returning to the functional model iteration activity. Crystal Alistair Cockburn and Jim Highsmith created she Crystal family of agile methods in order to achieve a software development approach that puts a premium on “maneuverability” during what Cockburn characterizes as “a resource limited, cooperative game of invention and communication, with a primary goal of delivering useful, working software and a secondary goal of setting up for the next game” The Crystal family is actually a set of example agile processes that have been proven effective for different types of projects. The intent is to allow agile teams to select the member of the crystal family that is most appropriate for their project and environment. Feature Driven Development (FDD) Feature Driven Development (FDD) was originally conceived by Peter Coad and his colleagues as a practical process model for object-oriented software engineering. Stephen Palmer and John Felsing have extended and improved Coad’s work, describing an adaptive, agile process that ean bbe applied to moderately sized and larger software projects. Like other agile approaches, FDD adopts a philosophy that (1) emphasizes collaboration among people on an FDD team; (2) manages problem and project complexity using feature-based decomposition followed by the integration of software increments, and (3) communication of technical detail using verbal, graphical, and text-based means.