Thursday, 4 November 2010

Complexity in Software

I get to work with numerous clients in a variety of different industries at varying levels of maturity and I see a number of themes which lead to software complexity:
  • Complex business domain
  • Large legacy of software and systems which must be integrated
  • Mismatched developer experience
  • Policy
I wanted to point out why these areas cause complexity and how you can overcome them to deliver quality software.

Complex Business Domain

You typically can't change a complex business domain, except in a scenario where you see that there is an overly complex business process and there is an opportunity for you to propose simplification of the process using the software as a driver. If you can simplify the business process before you implement the software - even better because it will reduce the risk of implementation.

Having said that you can't change the complex business domain, you shouldn't verbatim create that complexity in the software, you should use your kitbag of techniques to simplify the design of the software such as using:
  • Domain model
  • Service layers
  • Software Layering
  • Object Oriented development
  • Service Oriented development
  • Componentisation
  • Aspect oriented software development
Use techniques like Single Responsibility Principal (SRP) for classes and methods as well as IoC, and make sure that you have the right unit and integration tests in place.

By using these techniques you can spread the complexity across a number of layers, components, classes or services - just be careful you introduce the right level of abstractions that you don't overcomplicate the software by using too many of the techniques or spreading the complexity too thinly.

Complex integration environment

Again, this is not something you can typically get away from, any business of some age or size will have a legacy of systems that will typically have evolved and you will just have to deal with. You can't expect them to throw away everything they have and start again - and if they did, you would just have another type of complexity on your hand - project management complexity.

To deal with a complex integration environment, your approach should be:
  • Understand the software you are integrating with early and very deeply
  • Make sure you understand how you need to integrate with it
  • Consider if an integration layer (e.g. BizTalk) will simplify the integration or make it more complex
  • Have well defined interfaces (not just data and operation, but all of the other non-functional aspects of software)
  • Have a good stub strategy
  • Integrate as early as possible
  • Test as early as possible
  • Make sure your developers are testing against real services
  • Make sure your continuous integration builds are testing against real services

Mismatched Developer Experience

Complexity is often introduced by developers working on the project not having the right level of knowledge of experience, and it isn't just about getting better software developers. Too inexperienced and the developers will use inappropriate technology to solve the problem which will introduce complexity through choosing lots of technologies or the wrong one. An example of this I found was using Word on a server to generate letters which caused layers of complexity which nearly derailed a multi-million pound development project.

Go out and hire some very good developers and you might find that they do and build in complexity by selecting too many technologies for the job, selecting the perfect technology for each task and leaving you with a bunch of technologies you don't need. On another project I was on had 3 IoC containers because different developers liked different technologies.

Choosing a range of developers that work together well and understand how to make pragmatic decisions will be invaluable to ensuring a solution which is matched to your needs, far more than highly skilled developers with large egos.

Another important decision when developing the software is ensuring that the people who will maintain the software can understand it - a high-powered development team are likely to move onto other projects and technologies leaving you with software that you can't understand or maintain.


Often organisations will implement policies to try to manage software development, if the organisation doesn't have much experience in software development or they have had poor experiences when developing software will introduce more and more layers of policy and control without improving the outcome of software development. One organisation I worked with had 80 quality gates, would take a long time to produce simple software and would often fail to deliver working software.

Other policies I have seen fail is selection of software for strategic platforms without properly understanding the technology or selecting technology because it was by the organisation favoured by senior management - regardless of whether that was right for the organisation or right-sized for the project.

Policy should support the development process and shouldn't be so detailed as to hamstring projects by forcing them to pass irrelevant quality gates with documentation which adds no value and is never looked at again or using technology which has to be shoehorned in to make it work in the way which supports the project.

Choose processes which support a competent development team and allow them to make the right decisions to deliver the software. Allow them to choose the right software to support the project and perform neutral evaluation. Documentation should have a purpose and help support the aim of developing high-quality software, not cover for poor management.

Get a management team who have a proven record in quality software delivery projects which are on time and meet the business need as well as being supportable in the long term. Often managers who have delivered projects which are sold as products will understand this better than manager who have been sheltered in IT departments for the whole of their careers.

The overwhelming message here is that policy cannot fix a broken project, poor technology selection or inexperienced developers, don't add layers just because of an issue which was experienced once.


So where are we with these observations and recommendations? Select the right people as your number one priority. Don't add policy to make up for bad management or poor technology selections, don't be dogmatic about your technology selections and be prepared to revise your decisions as you get more information - this could happen throughout the project. Make sure that you build continuous integration environments, have a good stub strategy and push the testing as early as possible and make sure your developers are doing it.

Make sure that you use good software development factoring so that you use the right mix of complexity, layering and componentisation techniques which ensure that the software isn't overly complex either by having too much logic in one place or spreading the logic over too many code files or classes.

Experience counts - make sure the people you have are experienced but have also demonstrated pragmatism in their previous projects.

No comments:

Post a Comment