Agile Software Engineering: XP and TDD I-EN

We have become totally and utterly dependent on software. Software is the lifeblood that makes us able to function as a society. That is a recurring theme in the workshops and classes I teach because many of the participants are in software development, and we always end up discussing and commenting on the skills that agile teams need to build high-quality solutions quickly and predictably, more or less predictably.

Let’s be clear: there are no shortcuts. The only way to go fast, especially in the long run, is to take care of the quality of the technical solution. Producing junk code does not make us go faster.

On the contrary, it increases the technical debt which causes us to go slower and slower, losing agility, like a toy car running out of batteries until it stops completely.

Software engineering, as a branch of specialization, generates a lot of controversies: is it a science or is it an art? It is a discipline that deals with software design, development, maintenance, and validation.

It can be said that software engineering is divided into several phases: requirements analysis, design, implementation, testing, and maintenance.

Some different methodologies and techniques and tools approach these phases in different ways to ensure quality and compliance with requirements.

Some approach the whole process with a lot of planning and a rather rigid method. Other voices argue that software design and development has a lot of art and a little engineering.

It is necessary to be pragmatic and find a balance between discipline, adaptability, agility, good design principles and standards, innovation, having a well-prepared team, and good practices.

In any case, quality is a fundamental aspect of good software engineering. But keep in mind that quality is treacherous: Those who don’t have it, crave it, those who have it sometimes neglect it, even though they know it is an advantage, and those who have always had it learn that it must be continually attended to and invested in.

Back to the future with Extreme Programming (XP)

In agile software engineering, I emphasize agile, not much has been invented lately, the best practices are still current and have their origin in Extreme Programming (XP).

XP’s main contribution to the world of software development is a collection of interdependent practices that are used to produce quality software while creating a sustainable environment and pace of work for teams so they can be more effective.

The following diagram corresponds to the XP Circle of Life and shows all these practices.

The Circle of Life is composed of three concentric rings. The outer one is formed by the practices closest to the business areas and is very similar to Scrum.

The intermediate ring groups the practices closest to the team, and how they self-organize and communicate.

The third ring, the most internal, defines the practices necessary to obtain the highest level of technical quality:

• Test Driven Development: ensures that the entire code base has automated testing.
• Refactoring: ensures continuous improvement and refinement of the solution.
• Pair-programming: share knowledge, review and collaborate to innovate and improve the technical solution.
• Simple Design: less is more. Create a maintainable design that emerges during development.

Automate with Test-Driven Development (TDD)

TDD is such a broad topic that entire books have been written about it. It is a very disciplined way of programming that consists of working in cycles that are composed of three very interrelated activities: writing code, testing (writing and executing tests), and refactoring.

Each cycle can be summarized in five very simple steps:

The cycles go very fast and are a continuous oscillation between writing the test and the code. Of course, when a person first encounters TDD they see it as an invasive exercise and a disruption of thought processes. But after a while, this practice becomes a habit and you can’t work any other way. It is addictive.

Testing is also a form of documentation. Tests are perhaps the most perfect way to document code because they show how to use and invoke the system. They are not ambiguous documentation: they can be executed and are synchronized at all times with the implementation, something that is complicated to achieve with traditional forms of documentation. Each test is a tiny, self-contained fragment that describes and validates how a piece of code works.

The steps in each cycle are really simple and the benefits are enormous. Given the impact that software has on our lives and the risks involved in having poor-quality software, we should consider having TDD written into law and made mandatory… why not?

Mercilessly refactoring so as not to choke on technical debt

Everything changes and evolves. Some things get better with time, like good wine. But software, in general, ages rather badly. The design and architecture of systems deteriorate over time and become more difficult to maintain.

The crux of the matter is the accumulated technical debt. If no action is taken, eventually the code becomes unmaintainable hell. Over time the deliveries of new features and fixes start to consume exponential effort, this is when many managers get desperate about the delays and decide to add more manpower to the team, and this makes the situation even worse. It’s that hard. It’s that sad. Over time, the situation gets worse.

One specific type of technical debt is Code Smells. Without necessarily being bugs, code smells are symptoms or indicators that something is not right: there is duplicate code, obsolete methods, or blocks that are too long, also an excessive use of design patterns, poor cohesion, or a high level of coupling.

Is it possible to avoid the progressive paralysis caused by these problems?

The antidote is the practice of Refactoring, which consists in improving the internal structure of the technical solution (code, design, architecture) while preserving the external behavior specified by the tests.

It is a practice proposed by XP that serves precisely to prevent the above paralysis situation. Continually refactoring the solution is another good habit of agile teams and as should be the case with TDD, it should become a law!

XP practices are interrelated. To refactor without risk we need a safety net: we need to have a full battery of automated tests that allow us to validate the impact of changes as we make them to ensure that we haven’t broken anything.

With the automated tests produced by TDD, it is possible to make changes and clean up code without fear. No limits. Quickly and professionally.

Changes made during refactoring range from a simple renaming of variables, methods, functions, and classes to improve code readability to a complete reorganization of code blocks. Large functions are split into several smaller ones.

Classes with many methods are reorganized as many more reusable classes, with greater cohesion and few methods. Dependencies are reversed and the level of coupling between elements is reduced.

And while all this is going on, the tests continue to work correctly because that is the premise of this practice: when we refactor the system must continue to work without errors.

The same approach of small incremental changes can be applied to upgrade the design and adjust the architecture or the platform. In these cases, we are talking more about a system restructuring and it is not always possible to keep the system running, in many cases, it is also necessary to restructure the tests themselves. These changes, which are deeper and more impactful, can take days, weeks, or months of work.

But the concept is the same in both cases: work in a continuous process to improve the system and thus keep technical debt at bay. The agility of the team is at stake. That’s why these activities should not be planned; they should be part of the team’s way of building software.

Simple Design: the beauty and attraction of the simple

Less is more. Simple Design is the goal we pursue with the Refactoring practice. But what is a simple design in software development? Kent Beck defines it as one that complies with these principles:

The more complex a technical design is, the greater the cognitive load on the team, and the amount of mental effort to perform a task or process information. The greater the cognitive load, the more work and time developers must devote to understanding and applying changes to the system.

This practice represents the search for the Holy Grail in technical design because it is about finding that delicate balance between the simple design of the solution (the above properties) and the complexity of customer and user requirements. It is the constant search that manifests itself through refinement and refactoring activities.

An important aspect of a simple design is that it emerges as the team builds the solution and learns about user needs. In many cases, it is best to wait until the last responsible moment to make certain technical decisions.

This incremental way of working is very different from the more traditional approach to software architecture that seeks to intentionally define and design the entire solution in advance with a crystal ball. This is riskier because many times decisions are made that are blind bets.

And we have come to the end of this article dedicated to agile software engineering. But we are not finished! In the next article, we will look at the last XP practice related to quality: Pair Programming and we will also review Behavior-Driven Development and some other aspects and tools that agile teams incorporate to ensure that quality is always on top.

In our training catalog, at SAFe Agile Software Engineering we address all these topics within a framework to scale the work of several teams collaborating to build complex solutions.

Seguir leyendo…