The waterfall lifecycle is the natural way of managing the development something innovative and complex. It, or something better, is particularly neccessary where the development is subject to a contracted project. Note innovative , for members of this class, writing a program to read a list of numbers, sort them, and print in descending order, contains neither novelty, complexity nor risk [or pick some program that you do think trivial!]. Hence, it requires little or no planning, analysis, design - over and above what can be done almost unconciously.
On the other hand, it is unlikely that a viable dissertation project could be completed without some form of planning and phasing of tasks.
We have already mentioned that using the waterfall model: the project proceeds according to clearly defined phases; a preceeding phase must be completed before the next starts; phase completion is judged by the outcome of the phase matching the requirements defined by the previous phase. This is natural and logical - how rational and careful people proceed: `look before you leap'. Not that all software developers, or project students!, are rational and careful; and it is easy for the inexperienced to misjudge the levels of complexity, novelty or risk.
This chapter gives a detailed analysis of the waterfall lifecycle and techniques associated with structured systems analysis and design. After identifying some shortcomings of these we introduce alternative lifecycles which in some way improve on these inadequacies.
You may wonder why we are devoting time and space to a process which is known to be inadequate, there are three good reasons:
You will have noted our introduction of the term process : we use process in the sense of [Pressman, 1997] we define an information systems development process as
...a framework for the tasks that are required to build high quality software
. In this context `software' and `information system' can be taken as equivalent.
The phases of a typical waterfall process are:
1.
Concept.
2.
Feasibility analysis.
3.
User Definition of System Requirements.
4.
Developer Definition of System Requirements.
5.
High-Level Design.
6.
Detailed Design.
7.
Implementation and coding.
8.
Integration and Test.
9.
System Test.
10.
Acceptance Test.
11.
Operations.
12.
Maintenance.
Now, we will analyse these in some detail - to show up some of the weaknesses, but first some discussion of formality in the process.
Some Definitions:
Deliverable
A document, piece of software, or piece of hardware to be delivered to user (buyer, customer, client) as part of contract; the secret, for large jobs, is to have intermediate deliverables - so that bad news (or good) is known early enough to take corrective action. Analogy: Continuous assessment for university courses; inspection of a house during building.
Milestone
One of a number of predefined points during the project which mark completion of part of the work or acceptance of a deliverable by the customer; can mark time at which go-ahead to a next phase is decided; also progress payment points. Analogy: Passing of years exams and assessments; laying of foundations of a house.
Input
Information, data, documentation etc. supplied by user to developer for use during project; e.g. sample employee data for use as test data.
Output
Any product of a phase.
Review
Inspection of a deliverable (in most significant cases involving user) to see if it meets requirements; success may mark the reaching of a milestone.
Baseline
Current agreed plan, specification, or design or system - usually reviewed; cannot be changed, except by express agreement of all parties. Baseline often corresponds to a deliverable.
Examples: Current wages software produces errors. Entrepeneur identifies opportunity to develop a computer based system to read children's bedtime stories. Club secretary fed-up keeping manual records.
Preliminary exploration of possible solutions/technologies/suppliers.
The user documents as much as he knows about the job the system must do. May specify also: schedule and cost constraints; special constraints, e.g. run on an specific platform; all supplementary requirements: documentation, maintenance, quality, standards compliance, intermediate reviews.
often users don't know what they want, and, even if they do, they cannot express it with any precision. See Prototyping and related processes.
In reality, this phase needs three subphases.
Developer analyses users requirement and, performs further investigation of requirements, produces developers version of requirements: System Requirements Document (SRD). The SRD is now a specification and part of the contract. Prepare project plan, including costing and schedule. Risk analysis.
Problem: Cost estimation.
We are now in the realm of business contracts and the SRD is now a specification and part of the contract and will probably become a technical annex to a business proposal from developer to user.
Starts once business proposal accepted by user.
Input: URD.
Decompose into subsystems/modules. Allocate to teams or individuals. Design tests for subsystems.
Other names are Architectural Design, Preliminary Design, Product Design.
Input: SRD
Outputs:
Milestone: Architectural Design Review.
Does the decomposition technique provide really decoupled modules? I.e. Can teams work independently? The answer to these questions have massive implications for maintenance and reuse . See Object-oriented Analysis and Design.
Decompose further into subsystems (or units or components) such that approx. one person can cope. Specify these subsystems. Design tests for (sub)subsystems.
Input: ADD.
Outputs:
Milestone: Detailed Design Review.
Code and test components - each programmer. Notice how far into project.
Outputs:
Components and modules are brought together to form higher level systems. And tested. Repeat until you have a working system.
You should now have a working system! The full system is tested, in the developers environment.
Assess the system against the rquirements defined in SRD. Probably done under users control and supervision, and in their environment. Final payment hinges on this.
The system gets used - hopefully!
Following [Pressman, 1997], we identify four types of development (change) neccessitated during maintenance.
Correction
Correct defects in the software: defects relating to incorrect requirements, or incorrectly specifications; defects from any of the construction phases - `bugs'.
Adaption
Adapt to changes in the original software and hardware platform. E.g. simpler: MS-DOS to Windows. Complex: stand-alone to client-server.
Enhancement
Customer identifies additional requirements.
Prevention
After many sets of changes, the software `deteriorates', or otherwise becomes difficult to maintain. See Reengineering, Legacy Systems.
Draw a rough bar-chart of this and estimate the schedule. You'll be surprised at the length of time - and the amount of dead-time.
Here we give the template for a highly formal System Requirements Document, based on [IEEE, 1993b,ESA, 1984,Mazza et al., 1996] that would be compatible with the formal waterfall process; we then emphasise the utility of the section headings as checklist items for less formal applications.
paragraphContents of the SRD
(i) Title Page:
Name and address of organisation.
Title of Document: System Requirements Definition for...
Version number - if there are going to different
versions.
Date.
Reference number.
Author/responsible (optional).
(ii) Abstract.
(optional).
(iii) Change control log.
(iv) Distribution log.
(v) Contents list.
List of sections.
List of Figures.
List of Tables.
List of Appendices.
1. Introduction.
- Introduce purpose of document.
- Scope of document.
- Structure of document.
2. Applicable and Reference Documents.
- List of documents -- like a bibliography.
3. General Description.
3.1 Perspective.
- brief sketch of business area; of users; finally zooming in to system
required.
- should attempt to give programmers etc. a feeling for the problem,
outside formal specification.
3.2 Related projects.
- e.g. software to replace an out of date package, or one that never
worked.
- a parallel project (say) being conducted by another S/W house.
3.3 Function and Purpose.
Thumbnail sketch of of what the system must do, not do, and the purpose
it will be used for.
3.4 System Environment.
- what the system must interface with,
- special considerations leading to reliability (say) or safety
requirements.
- relationship to other systems.
3.5 General constraints.
- e.g. software must run on a particular computer, use a particular
compiler, etc.
- e.g. system must be available by December 31 1998.
- e.g. wages system must be able to perform a particular function
between 9am and 11am on Thursday mornings.
4. Specific Requirements.
4.1 Functional Requirements.
The requirements that we normally think of -- what the system must do.
- list of tasks or processes,
- processes (maybe) specified by:
- inputs
- outputs
- detailed specification of processing
- in structured systems analysis the functional requirements are
specified by:
- the hierarchy of Data Flow Diagrams
- the Data Dictionary
- specification of lowest level DFD processes using Structured
English, or pseudo-code and other methods.
- emphasis on: UNAMBIGUOUS, QUANTITATIVE, TESTABLE requirements.
4.2 Performance Requirements.
- specifies the speed at which the system must operate,
- e.g. payroll must complete in 1 hour -- for 1000 employees or less.
- Sections 4.1 and 4.2 might need to be lumped together where there
were many separate processes in the functional requirements -- each
with its own performance requirement.
4.3 Interface Requirements.
- what the system must connect to.
- e.g. a software payroll system which must interface with the current
personnel information system.
- a personal computer system which must connect to the existing
corporate network.
4.4 Operational Requirements.
Requirements arising out of how, where the system will be used. E.g.
- qualification of users/operators,
- use in a hostile environment.
4.5 Resource Requirements.
- for a software system specifies:
- the hardware to be used,
- hardware limitations e.g. amount of memory.
- compiler to be used.
4.6 Verification Requirements.
- specifies tests to be done during development, test data, test
documentation to be delivered.
4.7 Acceptance Testing.
- specifies the tests that will be used as the basis for user
acceptance of the system.
4.8 Documentation Requirements.
- documentation to be delivered during development.
- docn. that will be part of system e.g. User Manuals, Maintenance
Manuals, etc.
4.9 Quality Requirements.
- may specify production and adherence to a Quality Assurance Plan,
- may require supplier to have achieved a Q.A. standard, e.g. BS 5750
- change control requirements; configuration management.
4.10 Safety Requirements.
- safety of data files in a Data Processing application. cf Data
Protection Act.
- safety of equipment and people in a real time control application,
cf. Year 2000 problem. cf. Product liability.
4.11 Reliability Requirements.
Can be formally specified by Mean Time Between Failures (MTBF), and Mean
Time To Repair (MTTR). But difficult for software, still a research area.
4.12 Maintainability Requirements.
Specifies attributes of the system that affect the ability to
- trace and repair defects
- make modifications.
e.g. for software:
- modularity,
- use of object-oriented programming,
- built-in self test,
- quality of documentation,
- readability of source code.
4.13 Schedule Requirements.
- e.g. must be available by December 31 1998.
This example, from [Campbell, 1998] trivialises the process, nevertheless it identifies the characteristics I want to examine.
Informal requirement: Read ten numbers; sum them and compute the average; output the result to the screen. Three processes:
Input ---> Process ---> Output
is how it would appear on a top-level data-flow-diagram. If neccessary, e.g. if there was detailed verification of input, it might be neccessary to explode the Input process, to arrive at a number of subprocesses, e.g. four processes:
Input1 ---> Input2 ---> Input3
---> Input4
For the sake of argument, we will assume that Process can be exploded into two: Sum, Average. Let us ignore for now the decomposition of Output .
When you get to a process which merits no further decomposition, you define its activity using some sort of pseudo-code, in a so-called mini-spec . Each statement in the mini-spec might correspond to one or more program statements, or maybe to a system function call.
The idea of structured systems analysis is that the complete system requirements are captured in the combined:
a
data flow diagrams,
b
the data dictionary which describes the (flowing) data,
c
mini-specs.
The hierarchy of data-flow diagrams can now be rather easily used to form a (program) structure chart . From the example above, we can produce a structure chart:
+--------+
| Top |
+---+----+
|
+------------------+------------+------------------+
| | |
+------+ +--+----+ +--+---+
| Input| |Process| |Output|
+--+---+ +-------+ +------+
|
+-+-------------+----------+--- etc...
| |
+--------+ +-------+
|Input1 | |Input2 | etc...
+--------+ +-------+
Structure Chart -- Design
Thus we have a top-down decomposition .
Implementation in code is now a straightforward matter. There is a main
program, which corresponds to Top:.
Procedure Top.
Declare variables.
begin
Input();
Process();
Output();
end;
The structure chart decomposition of Input
is reflected likewise in its code:
Procedure Input.
Declare local variables.
Input1();
Input2();
Input3();
Input4();
return;
end;
If Input1 etc. are at the
bottom of the hierarchy, their code is developed from whatever is specified in
their mini-spec s.
Typically, procedures at each layer are arranged in libraries .
- see above.
from [Campbell, 1998].
In the couple of years, 1987-89, before I took this lecturing job, I was technical manager of a project which developed attitude-and-orbit control software for the ISO (Infra-red Space Observatory) satellite for European Space Agency, see my web page for more details. That project was governed by a number of quality assurance standards, the most applicable of which was the ESA Software Engineering Standards, [ESA, 1984,Mazza et al., 1996], which, in turn, was based on IEEE [IEEE, 1993b] and US Department of Defence standards. This standard mandated the waterfall lifecycle.
I enforced the waterfall lifecycle enthusiastically - sometimes with a rod of iron. Our effective customer was Fokker Space & Systems - the Dutch aerospace company. Incompatibility number-one : there was no way on this earth that Fokker could, or would, agree the requirements by early 1988. I dug in, and was supported by ESA and the overall prime contractor, Aerospatiale of France - though neither understood the issues, and thought that software people were witch-doctors and as such best avoided. I also had the contract on my side. By the time I left in 1989, although we had done a work-around and had proceeded with an incomplete requirements, a lot of time and energy had been wasted.
In addition, we had selected Ada as the project language. And, as you know, one of Ada's main innovations was a package structure similar to the Modula-2 module . Thus there was the means to design the software on the basis of abstract-data-types . Enter incompatibility number-2 : the ESA standards explicitly, or implicitly via the waterfall lifecycle, demanded top-down decomposition ; the eventual design was actually closer to object-oriented than top-down.
Incompatibility number-3 : the attitude-and-orbit control system required to execute independent and concurrent tasks: read position sensors, perform housekeeping activities, compute control output - to the thruster jets, etc. The system had no top! , [Meyer, 1997].
ISO was launched in December 1995. To date the satellite has performed
perfectly, and the mission is judged a great success. If you want more details,
go to my home page, http:/www.infm.ulst.ac.uk/~jgc/;
click on my brief bio. and look for links to do with CAPTEC (http://www.captec.ie) and ISO .
I should point out that my tenure as project manager was not all that bad! We had the foresight to choose Ada as the development language - I think it was only launch delays that stopped CAPTEC being the first software house with Ada code in space; we rejected C as the development language; likewise, we fought with great ferocity to reject assembler.
See section 2.5 of [Pressman, 1997].
See section 2.6 of [Pressman, 1997].
See section 2.7 of [Pressman, 1997].
See section 2.8 of [Pressman, 1997], [Hall, 1990], [Bowen and Hinchley, 1995a], [Bowen and Hinchley, 1995b], [Potter et al., 1996].
In this chapter we have conducted a thorough examination of the traditional waterfall lifecycle/process. We have noted some of it shortcomings and pointed to processes that have been introduced to solve these.