Lean Portfolio Management (LPM) improves the way organizations make decisions about project and product development. This method requires a significant mindset change across the whole organization, including among business executives.

Stop managing costs, start managing value with Lean portfolio management

Do you recognize this? Centralized decision making, Outdated outcomes, lack of strategic alignment, approval of low-value initiatives, overly long evaluation cycles 

This is what you want!?

1. Planning, funding, and the assessment of processes become more frequent and dynamic.
2. Decentralized deceision making and more engaged employees.
3. Time to market (or time to value) improves because versions of your products can be delivered and tested earlier than under traditional predictive methodologies.
4. Better Product fit. Working with Minimum Viable Product iterations to get a final outcome with a better match to the demand. Also, products or services can be killed before they consume too many resources only to deliver an end result that is already outdated

Then you should manage your portfolio in a Lean Agile approach

Traditional approach

• People are told what to work on.
• Specialists are moved from project to project. They can work on many projects at the same time.
• Requirements and detailed project plans are created before the work begins.
• Work looks very detailed even before it starts.
• Plans are usually followed by inflexible annual plans that are difficult to change.
• Funding and budgeting are tightly controlled by finance.

What is LPM: Lean Portfolio management in a onepager

Roles, events, artifacts

Equal to Scrum, Lean Portfolio Management has also roles, events, artifacts. This is what described in SAFe (© Scaled Inc)

Roles

Lean Portfolio Management (LPM) – This function represents the individuals with the highest level of decision-making and financial accountability for a SAFe portfolio
Epic Owners – Take responsibility for coordinating portfolio Epics through the portfolio Kanban system
Enterprise Architect – Enterprise architects work across value streams and ARTs to help provide the strategic technical direction that can optimize portfolio outcomes

Events

Portfolio Sync – The Portfolio Sync provides visibility into how well the portfolio is progressing toward meeting its strategic objectives and may include reviewing value stream and program execution and governance of other portfolio investments
Participatory Budgeting – This event enables LPM to collaborate with Business Owners and other relevant stakeholders to right-size the investments in value streams and helps manage the approval process of epics in the portfolio Kanban Strategic
Portfolio Review – This event enables LPM to create alignment and investment guidance to inform rapid, high quality, decentralized decisions; adapt to meet changing needs; and provide governance necessary to effectively respond to new and changing opportunities

Artifacts

Strategic Themes – Provide specific, itemized business objectives that connect the portfolio to the evolving enterprise business strategy
Portfolio Vision – The portfolio Vision is a description of the future state of a portfolio’s Value Streams and solutions and describes how they will cooperate to achieve the portfolio’s objectives and the broader aim of the Investments by Horizon – Assists the portfolio in ensuring managing near and long-term solution investments
Guardrails – describe the portfolio’s policies and practices for budgeting, spending, and governance
Business Epics – Captures and reflect the new business capabilities that can only be provided through cooperation among value streams. Each Epic is supported by the Epic Hypothesis Statement, and Lean Business Case for those epics that make it to the analysis state
Enabler Epics – Reflect the architectural, technology, and business process initiatives that are necessary to enable new Features and Capabilities
Portfolio Kanban – Provides a system for visualizing the flow of epics
Portfolio Backlog – The highest-level backlog in SAFe, it holds approved business and enabler epics that are required to create a portfolio solution set. This provides the competitive differentiation and/or operational efficiencies necessary to address the strategic themes and facilitate business success
Portfolio Canvas – The portfolio canvas defines the value streams that are included in a SAFe portfolio, the value propositions and the solutions they deliver, the customers they serve, the budgets allocated to each value stream, and other key activities and events required to achieve the portfolio vision

Om beslissingen te nemen, is het niet alleen noodzakelijk om te weten hoe waardevol iets is, maar ook hoe urgent het is. Cost of delay (CoD) geeft de economische impact weert van een vertraging in de oplevering van projecten. Met deze berekening kan je maximale economische impact maken. Sterkstaaltje omdenken!

Niet veel organisaties gebruiken deze methode terwijl het heel veel optimalisatie kan opleveren. CoD combineert de urgentie en waarde en komt uit het LEAN gedachtegoed en wordt nu ook veel in Agile en SAFe georiënteerde bedrijven gebruikt. Het is bedacht door Don Reinertsen en hij zei erover: “If you only quantify one thing, quantify the cost of delay.”

 The Cost of Delay helpt jou!

Mensen zijn geneigd naar de baten te kijken en onderschatten de negatieve factoren die een vertraging kan veroorzaken:  de hoeveelheid tijd en geld dat het kost om het project later af te ronden. In een flow georiënteerde organisatie moet je constant de prioriteiten updaten. De CoD helpt in:

1. Het leidt tot nauwkeurigere schattingen van de waarde en kosten van elk initiatief. Waarde is hierin een breed begrip
2. Het objectiveert naar de organisatie waarom welke keuzes gemaakt worden gebaseerd op economische impact
3. Naast de baten krijgt je inzicht in de kosten van vertraging. Betere inschatting van de kosten om het niet af te maken of niet te doen
4. Het uitlijnen van de CoD binnen de organisatie brengt autonomie. Zolang teams binnen de vangrails blijven kunnen ze autonoom keuzes maken

Hiermee kan je dus de resources inzetten op de belangrijkste zaken, in termen van waarde. Het is breder inzetbaar dan je wellicht denkt. Niet alleen in de productontwikkeling en software ontwikkeling waarbij je kijkt hoeveel het om de lancering van uw nieuwe dienst / product / feature uit te stellen maar ook breder bij organisatie beslissingen of bij het facilitaire bedrijf.

Weighted Shortest Job First (WSJF)

Het Scaled Agile framework heeft de Cost of Delay uitgebreid met een berekening van de kosten voor het uitvoeren van de taak, genaamd job duration. In dit plaatje zie je de impact door de gewogen kortste klus eerst te doen inzichtelijke wordt hoeveel meer waarde er gecreëerd kan worden.  het volgende plaatje zie je de impact als je eerst A, B, C doet

Het berekenen van de WSJF

Drie componenten zijn belangrijk in de WSJF

1. Klant waarde – Wat is de relatieve waarde voor de klant of het bedrijf? Wat is de impact die we kunnen maken
2. Tijd kritische waarde – Is er een vaste deadline of wat gebeurd er als we het later maken in termen van klanttevredenheid en aantal nieuwe en huidige klanten
3. Risico mitigerende waarde – kunnen risico’s worden teruggebracht

De tweede stap is het berekenen van de inspanning om iets te realiseren ofwel de job duration. In het begin is dat moeilijk te bepalen maar hoe meer je dat doet met bijvoorbeeld planning poker hoe hoger de kwaliteit en hoe meer voorspelbaarheid.

In een tabel kan je dan een vergelijking maken om te kijken wat je het eerst moet oppakken

Voorkom Scrum In Name Only! Herken je in 1 van deze typologieën, doe de checklist of jouw team Scrum echt in de vingers heeft. 

Vroeger spraken projectmanagers over Prince II in name only (PINO) als projecten enkel bijvoorbeeld de stages gates gebruikten of een document wel de naam gaven van een bepaald document in Prince II maar niet de inhoud.
We leven nu in het Agile tijdperk en daar zie je nu in toenemende mate SINO ontstaan Scrum in Name Only of Agile doen i.p.v. Agile zijn.   De mensen van Serious Scrum hebben er nog een bredere lijst van gemaakt. Welke herken jij?  

Check jezelf (en niet voor anderen zoals je manager)

Crisp heeft een Scrum checklist gemaakt als makkelijk overzicht om je team of teams te evalueren. Het is geen checklist voor management om af te vinken waar de teams staan. als set van vaste regels. Het is een set van aanknopingspunten.

Allerbelangrijkst im deze lijst te bekijken vanuit de Agile mindset: als kans om te leren en verbeteren en niet om terecht te wijzen of vinkjes om te halen. Het gesprek is belangrijker dan het vinkje. Je kan het bijvoorbeeld gebruikten voor de retrospective om het team samen te laten verkennen waar ze staan.

Credits aan Crisp, zie hierbij link naar alle versies in vele talen waaronder natuurlijk Engels.  

Making it tangible: What metrics can I use with the “Manifesto for Meaningful measuring” in mind? This is an overview of metrics you can use depending on each level in the organization and each maturity of the team. And remember: outcome is most important, with a special place for the north star metric 

In our previous articles we offered a “cookbook for meaningful metrics”, followed by an article “Acting on key Agile metrics with an Agile mindset” where we also introduced an Agile manifesto for meaningful measuring.  In this last article we want to bring it all together by combining organizational levels with maturity levels and finding the right balance. 

 On the horizontal axis we recognize maturity levels measuring: 

• Effort: focused on quality, stability 
• Output: focused on productivity & predictability 
• Outcome: focused on value & growth 

On the vertical axis we recognize organization levels measuring:  

• Team level: mostly a Scrum team but most items are also useful for a team that works Scrumban or Kanban  
• Program level: could be a value stream, SAFe program level or group of teams working on the same product (team of teams) 
• Organization level: in some organizations this is portfolio level, executive leadership team (ELT) or a Management Team

When we visualize this, this leads to the following graph: 

Though the metrics used in this graph are not a complete set, we think many of them are valid, if applied in the right context, with the right mindset and in the right levels of both maturity and organization. Therefore we’ve included the definitions of the metrics below. 

We hope these articles will help you to implement meaningful Agile Measurements, whilst being sensitive about the right context and maturity- & organizational levels. We also hope that the set of definitions above inspires you not just to blindly use what we have offered here, but to add your own when the context requires. Have a great meaningful measurement roll-out!
Part 1: Cookbook for meaningful metrics
Part 2: Manifesto for Agile meaningful measuring with the right mindset
Part 3: Overview of Agile metrics on each organization level and maturity phase
Written by David Warnink & Jeroen Stoter 

BONUS: Definition of the metrics in the graph

Description of all measurements on each level in the organisation and on the different maturity levels 

Team level

Measures the value of teams 

Teamlevel – Effort 

• Burndown: chart for tracking the completion of different tasks during a sprint. The X-axis refers to the time. The Y-axis represents the work left. burn-up chart shows how much work has been completed and the total scope of the backlog. 
• Velocity: determines the ability of a team to work through backlogs. As time passes, velocity tends to evolve. The X-axis counts the sprints. The Y-axis represents the amount of points. 

• Predictability (Planned to done ratio): how much work the team commits to doing at the start of the sprint versus how much they have completed at the end of the sprint. (deliver reliability)  

Teamlevel – Output

• Lead time: the period between the moment of making a request for delivering a product and the actual delivery. 
• Defects: all problems with the products (software) after releasing: Error, defect, Bug and Incidents. Also the calls from customers 
• Cycle time: Cycle time can be defined as how long it takes to produce a software release, from concept to completion. 

• Sprint costs: trend in the costs of sprint or the cost for a story point 
• Grow improvements: Improvements that the team is realizing to grow in their maturity using a grow- or maturity model  

Teamlevel – Outcome 

• Delivered value: the trend in delivering value for the customer over time.  
• Customer NPS (net promotor score): shows how much the customers are willing to recommend the product or service to others. Customer loyalty is an important leading indicator to determine the success for the future 

• Team Morale: measures the enthusiasm and persistence of the team engagement  

Program / value stream level 

Measures the value Program performance, team of teams or in SAFe the program level measured mostly in program increment. It is often a rolled up aggregation of team metrics  

Program level – Effort 

• Release burn down: chart is for tracking the completion of different tasks during a sprint. The X-axis refers to the time. The Y-axis represents the work left. burn-up chart shows how much work has been completed and the total scope of the backlog 

• Program velocity: determines the ability of a team to work through backlogs. As time passes, velocity tends to evolve. The X-axis counts the sprints. The Y-axis represents the amount of points 
• Program predictability: (Planned to done ratio): how much work the Program commits to doing at the start of the sprint versus how much they have completed at the end of the sprint. (deliver reliability) 

Program level – Output

• Cumulative flow diagram:  ensures consistency in workflow across the team. Graph to visualize amount of work waiting to be done (backlog), work in progress (started), and completed (validated). The X-axis represents time. The number of issues is on the Y-axis. Ideally, the diagram should be smooth from left to right 
• Program lead time: the period between the moment of making a request for delivering a product (feature) and the actual delivery. 
• Program cycle time: Cycle time can be defined as how long it takes to produce a software release, from concept to completion 
• Release costs: Trend in the costs of every release  
• DevOps Health radar: The health radar was built to help Agile Release Trains assess their ability to Release on Demand. 

Program level – Outcome 

• Delivered value: the trend in delivering value for the customer over time.  

• Customer NPS (net promotor score): shows how much the customers are willing to recommend the product or service to others. Customer loyalty is an important factor to determine the success for the future 
• Employee NPS (net promotor score): shows the employees level of motivation (mastery, purpose, autonomy)  

Organization 

Performance organization level or portfolio level. Depends on the scale of the organization. 

Organization level – Effort 

• Epic burn down (epic progress): Status view of all epics in a portfolio. Chart is for tracking the completion of different tasks during a sprint. The X-axis refers to the time. The Y-axis represents the work left. burn-up chart shows how much work has been completed and the total scope of the backlog 
• Investing horizon: Overview of the investment horizons in short – middle – long term in the current backlogs (on the horizon levels described by McKinsey).  

Organization level – Output 

• Lead time: the period between the moment of making a request for delivering a product and the actual delivery 

• R&D over revenue costs:  This metric compares the overall R&D cost with the revenue that results from it.  The ratio between euro’s earned and time spent indicates the rewards that the efforts turn out. The ratio gives an indication of how successful the investment of resources in products are over a certain time. The ratio between actual revenue on contract-, maintenance- and SaaS licenses as mentioned in the Product P&L and actual internal R&D hours spent on that product in any full year. 

• Business agility: helps portfolios evaluate their progress toward business agility 
• LPM self-assessment: Structured, periodic self‐assessment to continuously measure and improve portfolio processes. Aligns strategy and execution by applying Lean and systems thinking approaches to strategy and investment funding, Agile portfolio operations, and governance  

Organization level – Outcome 

• North star metric: is the key measure of success for the organization. It defines the relationship between the customer problems that the organization is trying to solve and the revenue that the business aims to generate by doing so whilst acting as an overarching “mother”-metric, for all organizational levels to relate (their metrics) to. 

• Delivered value: the trend in delivering value for the customer over time.  
• Customer NPS (net promotor score): shows how much the customers are willing to recommend the product or service to others. Customer loyalty is an important factor to determine the success for the future 
• Employee NPS (net promotor score): shows the employees level of motivation (mastery, purpose, autonomy)  

Specific DevOps metrics 

We’ve tried to be generic for all product development organizations with the examples above, but have included a set of specific DevOps measurements for Software Organizations. 

Within a software driven organization using DevOps there some additional metrics available 

• Continuous Delivery pipeline efficiency:. Measures efficiency of steps in terms of touch and wait time, i.e., analysis, backlog, build, validate, deploy, release, etc Some of the information can be sourced automatically from tools, especially Continuous Integration, and Continuous Deployment, while other data requires manually recording in a spreadsheet. 
• Deployment frequency: tracks the frequency of deployments. The objective of DevOps is to develop and deploy smaller deployments more frequently, as reducing the size of deployments and the amount of changes for each cycle makes it easier to test and release the deployment. 
• Total test duration: the time it takes to run the automated tests 

• Requirements coverage: measures what features are tested, and how many tests are aligned with a user story or requirement. 
• Unit test coverage: Shows how much of your source is tested. It helps you assess the quality of your test suite. Combination of function, statement, branches, condition & line coverage 
• Path coverage: measures the linearly independent paths covered by the tests 
• % automated tests: the percentage of automated test related to all tests 
• Test execution: measures total tests executed as part of a build. 

• Recovery over time: measures the number of rollbacks that occurred either physically or by turning off feature toggles. 

Metrics in an Agile organization are not magic in itself. The magic is in the behavior of people and especially leadership acting on the metric. Our Agile Manifesto for meaningful metrics helps in making the preferred behavior visible by focusing on dialogue, the trend and the coherence in the numbers. 

Following our first article on Agile metrics “Cookbook for meaningful Agile metrics”, we promised to break a lance for an Agile Manifesto for meaningful measuring. We believe that Agile metrics are an essential component of a high performing Agile organization. Agile metrics help Agile teams and their management measure the effort, output and outcome. Value delivered to customers – instead of measuring “what” or “how much” we are doing, is the most powerful.  

By measuring, agile metrics help keep the different levels in the organization in check whether there are any problems or opportunities for improvement. They also give a clear assessment of the current state.  

We have seen companies where management wants to use for example cost per story point as a standard measurement across teams; or management tries a one-size fits-all and forces all the teams use the same point measurements.  

This invariable leads to anti-patterns which always prove to be harmful. Remember, like velocity, it’s never valid to compare the cost per story points between teams. Every team is composed of unique individuals with their own set of capabilities, work habits, strengths, and weaknesses. As a consequence, teams will invariably have different results in their estimates, velocity, and cost per story point. Fixed reports that are not tailored towards the specific needs of the team, it’s maturity and organizational level will not help all teams to learn, grow and improve. This does not imply that such a metric cannot be helpful for one specific team if the context is right! 

This is where we want to introduce a Manifesto for Agile meaningful measuring; while there is value in the items on the right, there is more value in the items on the left. 

Agile Manifesto for meaningful measuring 

Dialogue about the why over Judging People 

To enable learning and growth, a safe environment to make mistakes and learn from them is key for teams to open up and face the opportunities to improve. Data without context can turn out to be nothing more than noise and distraction. The role of a leader is to support the teams to understand the direction and facilitate them and help remove their impediments. 
Nothing beats a good conversation with the team; a metric can be a starting point, not an accountability gate for the team to pass 

Coherence and Trends over Maximizing one metric 

Viewing all metrics in conjunction is important to avoid increasing fixation on improving one number. This creates more space to research the core problems in order to arrive at solutions that positively improve multiple metrics. The last number of the metric does not say much, following trends gives room to react to the pattern Don’t sail blindfolded on your dials; be aware of context and as such not just one metric will tell the full story. 

Realtime and Transparent over Fixed Reporting 

Transparency is one of the pillars in an Agile organization; this also applies to measuring. Therefore make all metrics public, irrespective of their level. This is a clear indication of transparency and underpins what you are steering towards as manager. Working with the Obeya room methodology can help. With the current techniques this can be done in real time and on a daily basis, aimed at supporting the employees. It prevents you from steering on monthly reports afterwards because you are then too late or the picture is not up to date. The value of a fixed report is therefore low in a complex, fast organization. 

Outcome over Effort 

A North star metric can be a great metric for all organizational levels to define and measure their own contribution to the overall goal. When a team or organizational level matures, the basic “hygiene” is in order, the “right to play” has been established. Growing through “Output” towards “Outcome” oriented performance and related metrics is a rewarding journey for all stakeholders.  
In general, we prefer leading indicators over lagging indicators. 

Bonustip

To close this episode, here are five tips how a good metric looks like (source) 

1. The metric is used by the team – Agile metrics should not be imposed or measured by management, they should be used voluntarily by agile teams to learn and improve. 
2. The metric is surrounded by conversation – Metrics should not just be numbers, they should be the starting point of a conversation about process and roadblocks affecting the team. 
3. The metric is part of a specific experiment – Metrics should be used to answer a specific question about agile processes, not just measured for the sake of measurement. 
4. The metric is used in tandem with other metrics – Even a great metric, if used alone, might lead to tunnel vision, and incentivize teams to maximize that metric at the expense of all else. Using several metrics together provides a balanced picture of agile activity. 
5. The metric is easy to calculate and understand – Metrics that are overly complex or not fully understood, even if they provide good insights about a team’s work, are not useful in guiding day-to-day activities. 

In the next episode, we will bring it all together by linking organizational levels to maturity.

Part 1: Cookbook for meaningful metrics
Part 2: Manifesto for Agile meaningful measuring with the right mindset
Part 3: Overview of Agile metrics on each organization level and maturity phase

Written by David Warnink and Jeroen Stoter

Agile metrics are an essential component of a high performing Agile organization. A great metric challenges teams to be better if surrounded in the right mindset. Metrics are there in different phases effort, output, outcome and on different levels in the organization.  

Creating agile organizations is the default answer to today’s increasing complexity and product development velocity; so much that required transitions become a goal in itself, rather than a means to an end. As such some best practices are implemented at will, without proper context. 

The challenge we see, is that throughout such transitions, it is imperative that for all sorts of stakeholders, there is value to be delivered captured in relevant metrics, to assess progress. 

As the transformation journey progresses, the metrics need to support the maturity phase of the organization at hand. If this sounds complicated, we do this in practice all the time. If you think about the ability to for example “light matches” is not something we want to measure with young children. But when they grow up, we love it when they want to cook us a meal, by lighting the stove as a first step. So the metric “ability to light a fire” is not one with a relevant meaning in the early stages of maturity. 

And then there is the element of stakeholders, often assessing “value” on different axes. From a market or customer perspective, the ability to adopt a product early to gain efficiencies may be more important than whether the team that delivers is planning sufficient improvements to remain efficient themselves, or even technologically relevant. And other stakeholders include internal staff or teams, portfolio, process improvements, innovation. 

We believe there are no shortcuts here, an organization is a living species, and its parts, its people, need to walk the transformation path for them to learn and progress in the process. There is no on-size fits all “check-list”, but we believe there are three generic phases: 

• Effort 
• Output 
• Outcome 

In the effort phase, it is about “the right to play”, more focused internally, requiring metrics to demonstrate a level of stability. Examples include “predictability” on a portfolio level, or “epic burndown” on a team level. 

In the output phase, as maturity is growing, customer focus is increasing. Metrics include “Defects”, or even “Business Agility”. 

When the rhythm is there, the next logical step is focusing on outcome. NPS and Employee engagement are typical metrics in this phase. 

One needs to understand the introduction of a metric that is suitable for the outcome phase, does not serve well if the basics are not in good order. You don’t want to measure the child on his ability to light a match.  

So what does a great metric look like? A great metric challenges teams to be better. Though such metric can set a big hairy and audacious goal (BHAG) – the goal needs to be achievable to be motivating; on an organization level, an overarching North-Star metric to which the underlying metrics contribute, can serve as a means for teams to contribute to the overall organization goals. 

Metrics are not blind dials to sail by; leaders need to provide context and safety for teams to learn and grow. By having a conversation standup or Obeya, supporting the teams to find and solve their impediments. If leaders use metrics to judge, this will lead to undesired behavior of the teams.  

So here are five tips: 

1. Recognize the maturity phase and apply metrics that fit that phase; 
2. Don’t sail blindfolded on your dials; be aware of context and as such not just one metric will tell the full story; 
3. Emphasize the opportunity to learn by having the right conversations; metrics are not a managers tool, they are for teams to support their maturity growth; 
4. Nothing beats a good conversation with the team; a metric can be a starting point, not an accountability gate for the team to pass.  
5. Teams need to feel safe to be able to learn and grow 

The result will be beneficial to all stakeholders.  

In a next episode, we will break a lance for an Agile Manifesto for meaningful measuring.

Part 1: Cookbook for meaningful metrics
Part 2: Manifesto for Agile meaningful measuring with the right mindset
Part 3: Overview of Agile metrics on each organization level and maturity phase

Written by David Warnink & Jeroen Stoter