PM Essence

Global Warming: The case of the sinking iceberg in Agile Development


By Vijay Anand, CAST
“What more and more executives are realizing that the workable software now seems to contain more structural defects than ever before, especially the ones that span across layers”
We are living in a dynamic, ever-changing world where everything needs to be continuously agile and adaptive else one is set up for failure. Software development is not an exception. More and more product development now seems to happen on Agile methodology where we develop working software with the underlying presumption that requirements can mostly never be frozen. And this seems to work well. One can see that product functionality continues to be delivered perfectly as per customer requirements, given the frequent interactions via user stories or as I call "functional cradling" and JIT testing. So the part that is visible in the iceberg a.k.a functional defects seems to go down in production. But what about the underlying iceberg structure that is invisible - the code and how it integrates together in the final product?
What more and more executives are realizing that the workable software now seems to contain more structural defects than ever before, especially the ones that span across layers - e.g. Architectural issues, proper framework use, complexity etc. As a result, this shift in defect ratio seem to move from typically 85:15 (Functional Defects: Structural defects) to now up to 70 - 30 in some cases.
Developers today are more pressured to deliver functionality than ever before, given the fluidity of the Agile environment and hence sometime have to make a choice on ignoring the underlying structural quality for the sake of delivering functionality. This then becomes - Technical debt that hounds the team later resulting in lot of rework - root cause analysis (RCA), re-development, re-testing etc. on problems that are more and more cryptic and involve multiple best-of-breed technologies e.g. Team has to shut down and restart weblogic server every 4 hours in production. This somehow seems analogous to how we are exploiting earth for natural resources today at the cost of starving our future generations tomorrow.
So what can teams do to deliver workable software that not only delivers the right functionality but the right structural quality?
1. Publish a list of best practices that a team has to adhere and respect from a structural quality perspective to the development teams. Prevention and education is the first best medicine. 
2. Plan for 10% of the sprint cycle time to fix structural issues that could come up as part of the product development life cycle
3. Use an automated structural quality solution every sprint cycle to measure the application structural quality holistically and not just code quality - Resiliency, Performance, Security, Architectural compliance, maintainability etc.
4. Integrate any structural issues that come as part of the sprint cycle to a system like JIRA and move it to the product backlog for fix.
5. Provide developers the flexibility to prioritize and fix these structural issues based on how they execute their user stories for a release. This part thus doesn't constrict them totally but helps them manage their time better over a life cycle.
6. Just before the product is ready to be shipped, a quality gate at that stage ensures that the product is complete not only from a functional quality standpoint but also from a structural quality perspective.
In fact, you should further refer to a between Forrester - Wipro-CAST that discusses amongst others the trade-off between quality and speed in Agile development and how best-of-breed system integrators and partners are managing this facet. It has very interesting insights.