Product Metrics

Previous articles in this blog have looked at the usage of metrics for measuring software developer productivity and software delivery performance. This article dives deeper into the topic of measuring software success with product metrics.

• Product metrics measure software success, i.e. the value / acceptance / usage of a product / service by its customers / users.
  
• DORA metrics measure the software delivery performance of the company / staff providing the service.
  
• SPACE is a framework to assess software developer productivity. 
  [A future article will look at the concept of Developer Experience as another avenue to assess developer productivity.]

A recent LinkedIn Post by Aatir Abdul Rauf provides a good overview by summarizing eight resources on the topic:

1. A Guide to Product Metrics (by Mixpanel)
2. North Star & Guardrail Metrics (Daniel Lee)
3. Defining Product Metrics (Hani Azam)
4. 11 Metrics for Product-led Growth (Katryna Balboni)
5. SaaS Product Metrics (by Userpilot)
6. Ecommerce Metrics (by SendInBlue)
7. Marketplace Metrics (Juho Makkonen)
8. AARRR, GAME, Pirate Metrics (Anthony Murphy)

According to Hani Azam, product metrics help companies

• have clarity, alignment and prioritization in what to build,
• decide how to build the product once they’ve prioritized what to build, and
• determine how successful they are and hold them accountable to an outcome

He distinguishes between four types of product metrics:

1. North Star
• This is the single most important measure of success for a company. 
• A North Star Metric (NSM) should be:
•  a direct reflection of a company’s mission,
• an indicator of how a company brings value to its customers,
• the only one of its kind,
• the answer to the following question:
  what is the one metric that best represents the desired outcome of your company?
2. Primary/Goal Metric
• Primary metrics depict the desired outcome of a particular product, team or initiative. 
• This is unlike the NSM, which represents the desired outcome of the company as a whole.
3. Supporting/Tracking/Input Metrics
• Indicators that the NSM (or primary metric) is moving in the right direction. 
• Particularly useful as leading indicators to your NSM.
• Tell you where your efforts to move your NSM may be falling short.
4. Counter Metrics/Guardrails
• Counter metrics represent other outcomes that the business cares about, which may be negatively affected by a positive change in the primary metric (or NSM). 
• Serve as guardrails.
  They exist to make sure that in the pursuit of your primary metric, you are not doing harm to another aspect of the business.

Hani provides the following examples for North Star Metrics (NSM) of various companies:

He also comments on the nature of good metrics:

Good metrics should be meaningful, measurable, and moveable / adaptable.

The last part of his article describes how to come up with supporting and counter metrics based on the observation that 

the best way to come up with these metrics is by tracing a users journey with your product.

The author refers to a common approach for breaking down the user journey:

AARRR (or Pirate Metric) framework - introduced by Dave McClure - breaks down the user journey by Acquisition, Activation, Retention, Referral and Revenue. 

Hani expanded the framework from AARRR to AAAERRR as follows: 

1. Awareness: How many people are aware your brand exists?
2. Acquisition: How many people are interacting with your product?
3. Activation: How many people are realizing the value of your product?
4. Engagement: What is the breadth and frequency of user engagement?
5. Revenue: How many people are paying for your product?
6. Retention/Renewal: How often are your people coming back?
7. Referral: How many customers are becoming advocates?

Please consult the original article for a more detailed description of the framework with concrete examples of applicable metrics.