Software Development: Iterative and Evolutionary Development

Iterative and Evolutionary Development

• Involves early programming and testing of a partial system, in repeating cycles.
• Relies on short quick development steps (or iterations), feedback and adaptation to clarify the requirements and design so successively enlarging and refining a system. 
• Normally assumes that the development starts before all requirements are defined in detail, feedback is used to clarify and improve the evolving specifications.
• Each iteration will include requirements, analysis, design, implementation, and test. 
• Iterative feedback and evolution leads towards the desired system. The requirements and design instability lowers over time.

Current research demonstrates that iterative methods are associated with higher success and productivity rates, and lower defect levels.

Timeboxing

A key idea is that iterations are timeboxed, or fixed in length.

• Most iterative methods recommend in iteration length between 2 – 6 weeks.
• If it seems that it will be difficult to meet the deadline, the recommended response is to de-scope

De-scoping: removing tasks or requirements from the iteration, and including them in a future iteration, rather than slipping the completion date.

Iterative and Evolutionary Development (also known as iterative and inceremental development; spiral development and evolutionary development)

Build-Feedback-Adapt Cycles

In complex changing systems, feedback and adaptation are key ingredients for success:

• Feedback from early development, programmers trying to read specifications, and client demos: to refine the requirements.
• Feedback from tests and developers : to refine the design and models.
• Feedback from the progress of the team tackling early features : to refine the schedule and estimates.

Benefits of Iterative development 

• Less project failure, better productivity, and lower defect rates 
• Early rather than late mitigation of high risks 
• Early visible progress 
• Early feedback, user engagement, and adaptation 
• Managed complexity: the team is not overwhelmed by “analysis paralysis” or very long and complex steps 
• The learning within an iteration can be methodically used to improve the development process itself, iteration by iteration. 

The Unified Process is a popular iterative software development process. 

Why a new methodology?

Process-oriented methods:

• Requirements of a project are completely frozen before the design and development process commences.
• Not always feasible
• Need for flexible, adaptable and agile methods, which allow the developers to make late changes in specifications.

Waterfall (Sequential) Lifecycle

• Promotes big up-front “speculative” requirements and design steps before programming.
• Historically promoted due to belief or hearsay rather than statistically significant evidence.
• Success/failure studies show that the waterfall has high failure rates.

Why the waterfall lifecycle fails?

The key false assumption:

• The specifications are predictable and stable and can be correctly defined at the start, with low change rates.
• But change is a constant on software projects.
• A typical software project experienced a 25% change.

  

Unified Process (UP) Best Practices

• Get high risk and high value requirements first
• Constant user feedback and engagement
• Early cohesive core architecture
• Test early, often, and realistically
• Apply use cases where needed
• Do some visual modeling with UML
• Manage requirements and scope creep
• Manage change requests and configuration

Unified Process Phases

Inception

• Inception is not a requirements phase; rather a feasibility phase, where just enough investigation is done to support a decision to continue or stop. –
• The life-cycle objectives of the project are stated, so that the needs of every stakeholder are considered. Scope and boundary conditions, acceptance criteria and some requirements are established.
• Approximate vision, business case, scope, vague estimates.

Inception - Activities

•  Formulate the scope of the project: Needs of every stakeholder, scope, boundary conditions and acceptance criteria established.
•  Plan and prepare the business case: Define risk mitigation strategy, develop an initial project plan and identify known cost, schedule, and profitability trade-offs.
• Synthesize candidate architecture: Candidate architecture is picked from various potential architectures
• Prepare the project environment

Inception - Exit criteria

• An initial business case containing at least a clear formulation of the product vision - the core requirements - in terms of functionality, scope, performance, capacity, technology base.
• Success criteria (example: revenue projection).
• An initial risk assessment.
• An estimate of the resources required to complete the elaboration phase.

Elaboration

• An analysis is done to determine the risks, stability of vision of what the product is to become, stability of architecture and expenditure of resources. 
• Refined vision, iterative implementation of core architecture, resolution of high risks, identification of most requirements and scope, more realistic estimates

Elaboration - Entry criteria

• The products and artifacts described in the exit criteria of the previous phase. 
• The plan approved by the project management, and funding authority, and the resources required for the elaboration phase have been allocated

Elaboration - Activities

• Define the architecture: Project plan is defined. The process, infrastructure and development environment are described. 
• Validate the architecture.  
• Baseline the architecture: To provide a stable basis for the bulk of the design and implementation effort in the construction phase.

Elaboration - Exit criteria 

• A detailed software development plan, with an updated risk assessment, a management plan, a staffing plan, a phase plan showing the number and contents of the iteration , an iteration plan, and a test plan
• The development environment and other tools 
• A baseline vision, in the form of a set of evaluation criteria for the final product.
• A domain analysis model, sufficient to be able to call the corresponding architecture ‘complete’. 
• An executable architecture baseline. 

Construction 

• The Construction phase is a manufacturing process. It emphasizes managing resources and controlling operations to optimize costs, schedules and quality. This phase is broken into several iterations. 
•  Iterative implementation of the remaining lower risk and easier elements, and preparation for deployment. 

Construction - Entry criteria 

• The product and artifacts of the previous iteration. The iteration plan must state the iteration specific goals
• Risks being mitigated during this iteration. 
• Defects being fixed during the iteration. 

Construction - Activities 

• Develop and test components: Components required satisfying the use cases, scenarios, and other functionality for the iteration are built. Unit and integration tests are done on Components. 
• Manage resources and control process. 
• Assess the iteration: Satisfaction of the goal of iteration is determined.

Construction - Exit Criteria 

• The same products and artifacts, updated, plus
• A release description document, which captures the results of an iteration 
• Test cases and results of the tests conducted on the products
• An iteration plan, detailing the next iteration 
• Objective measurable evaluation criteria for assessing the results of the next iteration(s).  

Transition 

• The transition phase is the phase where the product is put in the hands of its end users. It involves issues of marketing, packaging, installing, configuring, supporting the user. community, making corrections, etc. 
• Beta tests, deployment. 

Transition - Entry criteria

• The product and artifacts of the previous iteration, and in particular a software product sufficiently mature to be put into the hands of its users.

Transition - Activities  

• Test the product deliverable in a customer environment. 
• Fine tune the product based upon customer feedback 
• Deliver the final product to the end user 
• Finalize end-user support material.

Transition - Exit criteria 

• An update of some of the previous documents, as necessary, the plan being replaced by a “post-mortem” analysis of the performance of the project relative to its original and revised success criteria; 
• A brief inventory of the organization’s new assets as a result this cycle.