Start Measuring Success With These 5 Agile KPIs

Jacqueline Sanders-Blackman
Jacqueline Sanders-Blackman
Start Measuring Success With These 5 Agile KPIs
Share
Share on facebook
Share on twitter
Share on linkedin

As the saying goes, “You can’t improve what you don’t measure.” Metrics (operational), and more specifically Key Performance Indicators (KPIs) or Critical Success Factors (strategic), are about measuring progress toward your goal. Without them, you cannot know if your AS IS (current state) is making progress and whether you are on track to reach your TO BE (future state).

The thing that makes KPIs so elusive is that there are so many things that could be measured. KPIs can be strategic or operational. They can be based on output versus outcomes or process efficiency versus deliverables. They can be qualitative versus quantitative. They can focus on the leading versus the lagging. Or, they can measure the efficiency or the effectiveness.

Another thing that further complicates software development metrics is that the output is a widget and no two software outcomes are ever exactly the same (some may be similar but not the exact same). Even COTS projects have different configurations and are used differently in every environment.

Regardless of whether you are using Waterfall or Agile, many of these challenges can be addressed by learning about software development metrics. However, an Agile environment adds a new dimension that is not in your traditional menu of metrics. For more, read my Do Your Agile Project Metrics Measure Up? Why Traditional KPIs Won’t Work blog post.

As people go to great lengths to create agile environments, the need to quantify and measure success is a reasonable desire. Many organizations naturally think that measuring the following will determine their “agile” success:

  • Velocity
  • Team capacity
  • Product owner’s time with the team
  • Teams’ time in a co-located area
  • Backlog growth
  • Defect resolution rates
  • Daily stand-up attendance
  • Demos given
  • Retrospectives held
  • Testing occurring during the sprint
  • Software delivered at every sprint

At a minimum, it probably is important that you are accountable for everything in this list. However, these measurements alone could stunt your agile growth because they focus on “doing agile” and not “being agile”. These metrics don’t communicate if you’re promoting customer collaboration, building trust, sharing knowledge, providing cross-functional training, prioritizing on high value, producing high quality software, fostering interaction, being responsive to change, and nurturing an engaging work environment.

For too long, metrics have put an emphasis on outcomes that are the result of unhealthy working environments, burn out, attrition, turnover, and work quality deterioration. Let’s look at some agile metrics that help measure and monitor more than ceremonies and focus on how the work is being done… metrics that measure and promote a healthy agile environment.

KPIs for Being Agile

Metrics today need to encourage the type of behaviors that cultivate the “being agile” world many organizations say they are striving towards.

Being Agile

Behavior

Methods of Measurement

Individual Behaviors (Being Agile)Knowledge Sharing, Nurturing Others, Providing Opportunities for Team Members, Reaching Out, Challenges/Supports Others360 Team Evaluations

Team Awards & Recognition

Individual Points Gamification
Team Behaviors (Being Agile)Respect, Trust, Honesty, Engagement, Determination, Collaboration, Passion, Initiative, Stretch, Transparency, Creativity/Innovation, Continuous Improvement, Swarm/Hovering BehaviorNiko-Niko Calendar

Caught in the Act Award

Team Points Gamification

Team Health Survey
Team to Team Engagement
(Being Agile)
Customer Collaboration (Face Time), Expectation Baseline, Interaction, Initiative, Responding to Change/Complexity, Outreach to Other Teams, Shared ServicesCustomer 360 Team Feedback

Shared Services 360 Team Feedback

Unplanned Request Metrics

Community of Practice, Guilds, Squads, Tribes, Dojo Activities
Outcomes
(Business Value/ Business Drivers/Value Management Traceability)
Growth, Innovation, Revenue Creation, Retention, Keeping the Lights On (KLO), Reduce Cost, MVP, ROI, Cost of Delay, Diminishing ReturnsPost Implementation Evaluation

Business Value Burn Up

Feature Demos
The Methods of Measurement in the last column are illustrations of the Gilb Measurability Principle: “anything you need to quantify can be measured in some way that is superior to NOT measuring it at all.” In other words, a measurement does not have to be perfect or even very precise, as long as your intent is to get a quantitative handle on something that was previously purely qualitative; the important thing is to take that first step toward quantifying.

Measuring Team Members – Individual Behaviors

Create team awards and recognition. (If this is something that already existed in your non-agile environment, make sure to revise the criteria for the rewards and recognition to agile values.)

I recommend creating team awards around those who take time out of their individual task to help a team mate. In a 360 Team Evaluation, ask team members (and only those who work together) if their coworkers are team contributors via a survey. The chart below can help identify team contributor behaviors for which to place a nomination… and behaviors that are not so desired.

 

Distraction

Individual Contributor

Team Contributor

Team Disruptor

Level of Control

Out of controlIn controlShares controlDominates, dictator

Knowledge

Uninformed, out of the loopInformedShares informationWithholds information

Support

Needy, helpless, takes but doesn't give backSelf-reliantHelpful, reaches outTakes over or offers no help

Confidence


Anxious, uncertainSecure, confidentReinforces, installs confidence in others, coaches, mentorsArrogant, unapproachable

Ability

Barely enough, Just enoughCapableTrusts others' capabilitiesPushy, conceited, "I'm right" attitude

Problem Solving

FearfulRisk TakerOffers others challenges, Supports effortsReckless, Self-absorbed, My way, Disregards how action impacts the team

As a group, the team determines how often the survey will be conducted, and the subsequent awards will be given.

As straight forward as that might seem, I have seen teams sabotage the nomination. In one environment that I worked in, a box was set out for team members to nominate each other. Several sprints went by and not one name was in the box. Many team members possessed ill feelings toward the transformation and longed for the days of being recognized as an individual contributor. Because no one was nominated, it was clear that coaching was needed to help them see the value in team contributor characteristics.

It’s been several months since the team received coaching, and now they regularly nominate members of their team. It is worth noting that the team did decide on a more formal, annual 360 review. Each team member gets to rank and vote on the other team members. No one outside the team votes. Everyone on the team then sees purely what their team members think of them and can adjust their behaviors accordingly.

Measuring Team Behaviors – Team Health

Agile benefits are realized when you have stable, high-performing team. To create a successful agile environment and culture, you need to measure the team’s health often and be ready to adjust when the numbers say the team is unhappy or unhealthy.

Something like team health or well-being might generally been seen as entirely subjective and thus impossible to measure and track. However, a technique called The Happiness Index, or the Niko-Niko Calendar, is a practice to help identify trends in the team’s attitude and tone.

It is also a great way to get the team engaged and talking about their challenges, not just general status updates. Sometimes the rich conversation derived from collecting this metric provides more value than anything else, especially if it leads to continuous improvement over time.

Measuring Team Behaviors – Team Engagement

The following chart is the foundation for a survey which was originally administered as a game using a platform called Kahoots®. Teams were quizzed on their understanding of why they were being asked to perform the various agile ceremonies.

Ceremony

Supporting Value/Principle

Product Backlog GroomingCustomer Collaboration - create a shared understanding of potential work
Sprint PlanningWorking Software - gain consensus of the team’s capability and capacity
Daily Stand UpInteraction and Individuals - open communication, daily commitments, evaluate priorities, synchronize the work
SprintsRespond to Change - collaborate with the customer as a member of the team to jointly create a solution
Sprint ReviewCustomer Collaboration - showcase work, be accessible to the end users during the design process
Team RespectiveInteraction and Individual - inspect the team dynamics, be self-aware, identify improvements

Measuring Team-to-Team Engagement

As one might predict, those who still had some resistance and hadn’t bought into the agile approach often answered the questions wrong. The purpose was not to convince or brainwash everyone into believing the purpose and intent of the ceremonies; it was to help the Scrum Masters make the ceremonies more meaningful.

This measurement is to test how well the team of teams are collaborating in an agile environment. There is a series of 5 questions for each category:

  1. Are the sprint design changes being clearly communicated when they have an impact on others?
  2. Is the Scrum Master someone that you can reach out to or that reaches out to you for clarification?
  3. Is the staff open to cross-team collaboration within reason?
  4. Is the work space conducive to cross-team collaboration?
  5. Are the team skills as a whole in line with the work items assigned?

When coaching teams, I often point out that perception is important. If they disagree with the rankings, we talk about what they need to do to improve the problem area or improve the perception. Sometimes the team effort and contributions aren’t visible. This exercise may lead to a team-to-team discussion or may highlight roadblocks the Scrum Master needs to address.

Measuring Outcomes – Business Value

In Agile speak, the outcome is not defined by the product; rather, the product is the vehicle that leads to the outcome. There are a lot of non-agile ways to measure outcome, but a few of the most common are listed in the image below.

At the end of the day, if you can’t identify the outcome that benefits the business, then creating software for the sake of software is a waste of company money. Every member of the team should be fully versed and informed of how every item in every sprint contributes to an outcome that has business value.

The transparency element that is essential to the agile approach helps remove the disconnect between the contributions of each team member and the outcomes that the business finds valuable. To measure this type of metric, survey the team to be sure everyone has the same understanding. Some teams even use a point system to track and reward. Usually, there is a direct correlation between those who actively listen and ask questions.

Setting the Stage for Agile KPIs to Drive the Agile Mindset

My final cautionary note overall is that metrics can be used for good (to make the environment better) or for bad (to micro manage people). When a metric indicates negative trends, root cause analysis should be performed to understand the contributing factors. I strongly encourage a more organic approach toward agile metrics. You want to measure your success, not meter it based on a systematic pre-arranged list.

With creativity and openness to change, many of these agile metrics can work in your environment. Will all of them work? Perhaps not. But, try them. Stagger how you introduce them. Make sure people are being open-minded and give them a chance. Make sure everyone that is impacted understands the purpose and value of the metric. Ask for input upfront and feedback afterwards. One thing we do know is: if it doesn’t work, don’t waste more time doing the same thing. Learn fast and see what works in your environment.

Thank you,

Jacqueline

More on Agile and Agile Metrics

Agile Metrics and Value Management

Do Your Agile Project Metrics Measure Up? Why Traditional KPIs Won’t Work

Kanban vs Scrum Infographic

From a Dysfunctional to Cross-Functional Agile Team

More on Business Agility

Join our #AlwaysLearning Community

Jacqueline Sanders-Blackman

Jacqueline Sanders-Blackman

Jacqueline Sanders-Blackman is a Netmind Lead Expert and Senior Instructor who has provided training and mentoring throughout her career in the areas of business analysis, database administration, data analysis and process improvement. She possesses over 29 years' experience working on projects, using various methodologies and approaches including Agile, Extreme Programing, Waterfall, Object-oriented, UML, Six Sigma and CMMI. She has held roles of business analyst, project manager, program manager, tester, DBA, developer, process analyst. Her diverse background made Certified Scrum Master and Agile Coach a natural progression. She has spent the last 9 years building up her Agile experience and now has a track record of helping organizations through the challenging transition from waterfall to Agile practices. Jacqueline is a published author and contributor to industry related titles.  Jacqueline was recognized as a leader by Women in IT 2013, 2014 and BDPA 2015.

Almost done!

Please check your email to confirm your subscription.

Join our #AlwaysLearning Community

Onsite Training Request

Please provide the information below to help us to customize your solution. 

Contact Us

Netmind US
296 South Main Street, Suite 300
Alpharetta, GA 30009-1973
T. +1 (678) 366.1363
F. +1 (678) 366.1983

Office Hours:
Monday – Friday, 8:30-5:00EST

General Inquiries:
info@netmind.net

Sales Inquiries:
sales@netmind.net

Request Information