Yes, for sure, when you think about KPIs, there are many factors that you can use to think about when determining where to start tracking KPIs.

Here are some of the processes I use, and I think are industry standard.

Align the KPI to the strategic business goals. Second, I like to test the KPIs using SMART framework. Specific, Measurable, Achievable, and Time-Bound. When you do that, it makes the team more connected to the KPI and then at least know what they are trying to hit.

Of course, you will want some examples including:

Budget variance: Compares the actual budget to the projected budget to see if the actual expenses or revenue meet expectations.

Cost Performance Index (CPI): Measures the performance of the project team and how much the project is ahead or behind schedule.

Schedule Performance Index: Shows how much and if a project is veering off schedule.

One of the best ways I have done this is to have a data-driven approach to running my PMOs. What do I mean by that, well, everything that you are surrounded in a PMO is data and if you want to have KPIs in your PMOs and your projects than you have to hyper focused on data. So, how do you do that will running a PMO, well one of the things I have done for years across several PMOs is a concept of pulling my PMO leadership team together each week and look at the data of the PMO. That means we look at Program and Project data, we look at their risks, their issues, their schedules, lessons learned…etc. and we do this week in and week out as regular operation process to run the PMO. From there, we work with the respective Program and Project Managers and talk to them about what we are seeing in the data and where can we help and support where applicable.

I have seen a lot of challenges when companies try and implement KPIs and I think the biggest things that they run into is the bad behavior. I think when companies try and implement KPIs too early or at a point when the companies are not at the right maturity level, then they face a ton of different challenges. For example, we have project managers and people on project teams trying to hit metrics for the sake of hitting metrics. So, in those cases, we see watermelon metrics, we see PMs hiding the truth, we see all sorts of things happening here.

You have to create a culture of openness and having your PMs be focused on being 3rd party truth tellers, rather than hiding what is really happening on a project to hit a metric.

Some of the common challenges I have seen though with KPIs include:

Lack of Clarity in Objectives – Many organizations struggle to define clear business objectives, making it difficult to create relevant KPIs from those metrics. I struggled with that a lot at one company I was at, and we just continue to struggle and struggle in that area.

Data Quality Issues – Organizations often face challenges with inconsistent or inaccurate data, which can undermine the reliability of KPIs. Garbage in and Garbage out and when we have some teams doing KPIs like a check box exercise, then they will not spend time putting in good quality data.

Overcomplication – Some organizations create too many KPIs and want to over index on getting all these KPIs setup and don’t have maturity in place in the org to hit those them. This leads to confusion and difficulty in focusing on what truly matters and more important bad behavior, which I have seen a ton of times.

Yes, I have many of those scenarios, including one from when I was working at Microsoft as a PMO manager. I noticed that despite strong project planning and execution, many projects were failing to meet delivery targets. Delays were impacting production schedules, customer satisfaction, and project budgets. We were all waterfall at the time, but felt like we needed to get some KPI help and do a root cause analysis and identify the root causes.

KPI Analysis Approach:

I employed a Lean Sig Sigma expert, and we gathered historical project data and selected key performance indicators (KPIs) focused on schedule adherence, resource utilization, and defect rates in project deliverables. We chose a few projects that represented typical challenges and analyzed these KPIs in depth. We were using Project Online and PowerBi in this PMO, so we had all this data, and pulling it together was pretty easy.

Key Insights:

Schedule Variability: One of the things we found was that projects often started on time but consistently fell behind by the testing phase. That pushed Testing into a bind of having to do things in a compressed timelines. While planning was thorough, teams tended to underestimate the time needed for testing and factor in what the time it took in to fix bugs.

Resource Bottlenecks: A second insight showed that certain critical resources—especially specialized testing and DBA staff were in short supply, leading to delays. The resource utilization data from our timesheets highlighted these bottlenecks, showing that team members were overcommitted across multiple projects. This was a problem many teams were facing, and we were certainly facing it as well.

High Defect Rates: During this process, we also found that our defect rate was much higher. We found due to the compressed timelines, the testing was rushed and in some cases handled by less experienced team members just to hit the timelines. This meant teams spent extra time addressing errors later in the process, creating a cycle of rework for the development teams. Which they were not happy about and caused lots of tension.

Implementation and Improvement:

So, what did we do? I worked with my leadership teams across the different disciplines, and we agreed on for future projects:

More Realistic Scheduling: The team added more time for QA and testing. We also added time in our schedule where we would test, go back to develop, test, develop and created those timelines into our project schedule. So, everyone understood that our schedule would include those back-and-forth cycles.

Resource Allocation Adjustments: To resolve resource bottlenecks, the PMO redefined the resource allocation process and established a centralized process for establishing an organization-wide project priority list. With that list, project resources knew where to spend their time on projects. If a higher priority project needed resources, people would pivot to help. So, there were still resource assignments across multiple projects, but now they knew which ones were most important.

Early Testing Focus: To minimize defects, they adopted these multiple test-dev-test cycles within the project schedules that allowed the highest quality errors to be fixed before testing continued. The time added was saved with higher quality products and higher customer satisfaction.

Results:

Within six months, the KPI adjustments led to substantial improvements. Project completion times improved by 20%, rework costs decreased by 15%, and customer satisfaction scores increased. By refining how they approached QA, resource allocation, and scheduling, we saw sustained improvements in project delivery and minimized disruptions.

As a PMO leader, the target is constantly changing, and it’s something you need to stay on top of for your PMO. Here are some of the strategies that I have used in before and still use today:

First, continually stay on top of where your organization is going. As the business climate changes, you need to focus on which metrics need to change in your PMO execution to align with those goals.

Consume industry metrics and ongoing advancements. As the industry changes, you might need to adjust and adapt your metrics to start tracking your programs and projects.

Regularly review your own KPI metrics and program and project execution to those metrics. What is working, what is not working and what course corrections do you make based on those findings.

These are just some of the strategies I use to ensure my KPIs stay relevant in my PMO.

The most important thing you can do here is to not make it an environment of not hitting KPIs on your programs and projects. For example, if project teams feel like they are going to be penalized for hitting a metric, what they will do is they will start to hide data and not tell the true story. In that situation, what you get are “watermelon” projects and not hearing the full story. The team is so focused on hitting a KPI metric, they lose focus less on the delivery of the project.

So, what are some ways that I found to allow that to happen. Here is what I use:

Setting Clear Expectations with project teams on KPIs – The first step is to define KPIs that are aligned with both the project objectives and the strategic goals of the organization. Ensure project teams that hitting KPIs are goals and that there is no penalized if they don’t hit them.

Promoting Transparency – I’ve implemented PowerBI dashboards and regular reporting mechanisms where KPIs are tracked in real-time. This transparency ensures that everyone on the project team and stakeholders can see the current performance status and how it compares to the targets. It promotes a culture of accountability as each team member knows their contributions are visible to both their leadership and stakeholders.

Encouraging Continuous Improvement – Beyond accountability, KPIs are a tool for continuous improvement for your programs and projects. For example, in one of my projects, we used KPIs to monitor key processes like on-time delivery and defect rates. When performance fell short, we held lessons learned sessions to identify areas for process improvements across both those areas. This feedback loop was critical to fostering a culture where the team was constantly looking for ways to improve.

I am sure there are others you can do, but I have done this across several PMOs and companies and found these to be the most successful.

I think through the questions I responded to earlier; I have talked about how I have diagnosed and put some processes in place to find the root causes, but to summarize, let me reiterate some of these key steps.

First review the data and look at what is underperforming and by how much. Secondly, perform a root cause analysis on why the KPI is underperforming. This is where Dashboards and reporting on this KPI data is critical and any historical data to determine if there is a point where something went wrong.

Finally, I would put some correct actions in place to fix the KPIs that are underperforming and then closely watch and review those KPIs. It is the ongoing, close monitoring of underperforming KPIs that allows you to react if you see that same thing happening.

There are two parts to this question; let me address both of them. Let’s tackle the technology to improve KPI tracking first, as that is the easiest to address. I think what we have seen with tools like Power BI and Tableau, when connected to a data source like Project Online, has been incredible for KPI tracking and reporting. I have been using the combination of Project Online and PowerBI for years and years now and I can say I it is an amazing set of tools for KPI reporting. It has been incredible.

I think a lot of people say that and mean the ‘waterfall’ or ‘predictive’ method of managing a project. I would suggest that construction, manufacturing, and healthcare continue to use those ‘old’ methods, and therefore, I don’t feel they are old at all. When it comes to KPIs, I believe that construction projects do KPI tracking better than any other methodology in the world.

Your second question was about data utilization on projects, and that is something I have discussed for years. My background in Oracle Data Administration and SQL Server Administration has given me a strong foundation and laser focus on the data from our projects. I have always believed that understanding your projects is critical to running a successful project. Some key lessons from using data in your projects include informed decision-making based on data rather than gut feelings, predictive analytics, and continuous performance monitoring.