• Account Manager
• Account Manager: IT&T Services...
• INFORMATION TECHNOLOGY MANAGER
• Project Manager - WMS
• Communications Infrastructure...

In praise of software engineering

Tom McBride, Information Age

17/06/2007 12:09:30

It isn't as if the world has suddenly changed so much as the cumulative extent of the gradual change is beginning to overwhelm us. Where we could get by with people simply applying what they had learned through experience and through what other people did as part of their job, this is possible with diminishing frequency.

Before going too far it would be useful to say just what software engineering is. After all, the term has been bandied about for years and used to mean everything from programming to the totality of activities involved in developing software.

While there is some consensus among those who study the subject and who teach it, coming up with a concise definition is another matter. Software engineering is not a thing, an object that is easy to describe.

Nor is it a software development methodology such as "agile development". Instead, software engineering is a collection of activities that, in concert, develop software that possesses the functionality and attributes it was meant to possess.

If that sounds ho-hum, think of the number of software projects that have delivered software that did not have the intended functionality and was not as reliable, usable, maintainable or scalable as was intended. If it doesn't have to work and doesn't have to be dependable, any fool can do it.

Just as in other, more established, branches of engineering the task is to design something that serves a specific purpose and can be constructed, so it is with software engineering. We expect a building to be built just as it was designed and to be as thermally efficient, resistant to earthquakes and fires, to house so many people performing their intended activities as was designed.

We don't expect the building to suddenly sprout another floor, or to lose one. We don't expect that the building's plumbing be inadequate for the number of toilets and taps nor the power supply to fail every time another device is plugged in.

We expect that the building will work and we expect that the engineer who designed it will have designed something that can be constructed to have the attributes it is supposed to have. Similarly, we employ software engineering to deliver the software with the functionality and attributes that we intended to deliver.

The software systems on which commercial organisations now depend are large and complex enough that more than a trivial amount of expertise is needed to design, develop, install, operate and maintain them. But their size and complexity is not the most significant change. Instead it is that organisations now depend on their software systems, those already in place and those yet to be implemented.

Organisations are finding that new information technology systems, and the software at the heart of them, are an integral part of creating the capability necessary to meet the challenges of a changing world. As much as we would all like the world to hold still while we catch our breath, it never has and never will.

Obviously the efficiencies of developing and modifying software systems are constantly being scrutinised. There is no reason why software development should be immune when everything else is expected to achieve ever higher levels of efficiency and productivity. However, if we don't know what makes software development work we run the danger of crippling the whole thing in our ignorance.

A common perception is that software development means programming. After all, that is where the system is actually constructed. While most people will admit that it is necessary to gather some requirements and to design the system, possibly even to test the system once it is developed, these activities seem to be viewed as less important than actual programming. Surely, the reasoning goes, if the requirements could just be explained to the programmer(s) they could develop a system without any further effort.

The programming-centric view of software development is rather like the assembly line view of car manufacture. After all, the assembly line is where the cars are built so all other activities must be secondary.

Not quite: the assembly line is certainly essential and visible, but it is useless without its vast web of suppliers and the supply chain logistics that connect them all. The assembly line depends on the smooth working of this integrated web to supply the correct sub-assemblies just as much as the suppliers depend on the assembly line to consume the sub-assemblies. In addition, all of the suppliers depend on receiving the correct designs and specifications in enough time that new or changed designs can be produced and supplied at the right time. There may have been a time, possibly 20 years ago, when it was reasonable to look at a car assembly line as the embodiment of car manufacture but no longer.

If attention shifts from programming to project management then software development can be seen in a different light. Attention shifts to programming management. Schedules, work breakdown structures, project plans all become more important. Project failure, or the risk of it, is seen as a failure of project management. Something that can be overcome with the application of more project management.

Schedules become tighter, project monitoring becomes stricter. This is all to the good if the problem with your development originates from a lack of project management. However, if the problem originates elsewhere then no amount of project management will fix it.

Yet another perspective is software development as acquisition. In this view, software components are sourced from a range of vendors, then integrated. Obviously this can become rather large and complex but it is still, at its heart, an acquisition problem. Some parts of the development are outsourced. System components such as database management systems are purchased.

Service providers may deal with specific tasks such as system integration or acceptance testing. Risk management may be extensive and the riskier items receive a great deal of attention. Most, if not all, potential problems are viewed as contractual problems that can be overcome by contractual means such as penalty clauses.

These different views of software development are all valid to some extent. But they all highlight some aspects of software development and ignore others.

Software engineering is not all that obvious or on display. At its simplest, anyone can do it. But software systems are no longer simple and their development can dissipate into chaos as fast as an underperforming football team.

It takes a lot of work just to keep track of all the requirements so that they are grouped sensibly, implemented correctly and in a timely release so that a customer gets functionality that works, rather than uncoordinated features that must wait for supporting components in a later release.

Anyone can receive software components and sub-systems from an outsourced developer and put them together when it all goes as well as we planned. But it doesn't go well. Contracted developers misunderstand requirements. What we thought would integrate and interoperate, doesn't. The changed requirements that we sent to all the people who had to act on them were misunderstood or not received.

And it isn't as if you can always send the stuff back and tell them to fix it. You don't always have the time. Yes, it is wrong and yes, something needs to be fixed. But who is going to find out that it is wrong? Who is going to investigate what the quickest most efficient and effective fix will be? Who will develop a strategy for integration testing so that these problems are identified quickly and effectively? And on and on.

If we have managed to produce the required software systems in the past, why are software engineers and software engineering skills so suddenly needed? Who has done all the necessary "software engineering" and why can't they continue doing it? A lot of the necessary software engineering skills come from sound programming practices that individuals acquire over time and simply extend to the current situation.

For example, configuration management is one of the more necessary activities that is usually learned in smaller, quieter circumstances. The transition from version control for a small project to configuration management in the large, for distributed development and maintenance of a collection of systems, is usually gradual.

Essentially, it seems that we grew up with it. Similarly, a lot of software engineering activities are planned and managed by project managers who have moved to project management from software development. These were the senior developers who saw project management as a career move and took with them all their knowledge of how software was developed.

The IT industry itself is only gradually realising the importance of software engineering. This can be seen in the gradual emergence of a separate and identifiable field of software engineering in many universities. They may have had "software engineering" listed in their offerings but many have been struggling to articulate just what they mean by software engineering. Nevertheless, there is a gradual emergence and separation from both computer science and information systems.

The first thing that anyone can do is to examine their own situation to determine if they have a potential, or actual, problem. The Guide to the Software Engineering Body of Knowledge (SWEBOK) lists 10 software knowledge areas: • requirements • design • construction • testing • maintenance • configuration management • engineering management • engineering process • engineering tools and methods • quality. Your organisation may not need expertise in all of those areas and certainly won't need to be expert in all areas, but simply determining how much expertise you need is a good start. From there, it is a comparatively easy matter of gaining more knowledge and expertise.

For the self motivated (or the impatient) there are several good books on software engineering that cover the subject and all its topics quite well. They are normally intended as text books for post-graduate degrees and while they might be considered light reading by some, most are more likely to consider them anything but light.

There are many industry seminars and training courses available in the more popular areas. One of the criticisms voiced in an ACS survey of training was that commercial training courses tended to be too vendor specific. Where it comes to software engineering, vendors tend to have training in specific tools rather than the overall subject of software engineering. Most Australian universities have degrees in software engineering and some have post-graduate degrees with a major software engineering component. For the most part the degrees on offer do cover the main knowledge areas, along with subjects that would more correctly be classified as computer science. However, post-graduate degrees usually require years rather than months or weeks.

Large complex projects that develop the ICT component of organisational capability are requiring more software engineering to ensure that the software itself is developed correctly. Software engineering expertise has been learned as part of, or an extension from, sound programming practices but is unlikely to be enough to deal with the challenges of these large, complex projects. There are few people with the necessary software engineering skills readily available, and they will probably be snapped up.

This leads to an inevitable shortfall of people with the required skills as organisations begin to understand one of the causes of their projects malaise. It usually takes some time for markets to respond to the demand for skills that are not easily acquired so it is unlikely that there will be a ready supply of software engineering skills.

So, all of you who have knowledge and skills in software engineering, prepare for your moment in the sun.

Dr Tom McBride is a lecturer at the Faculty of Information Technology at the University of Technology, Sydney, and a 25-year veteran of the software industry, project management and quality assurance. He is involved with software engineering standards development with both Standards Australia and ISO.