Insights and Tips on Data for Product Managers!

🔢 D for Data

(4th post in the A2Z in Product Management series)

Being a product analyst turned product manager, these are some of my insights and tips on the topic:

  1. Understand the difference between data-driven and data-informed, and be data-informed.

  2. Sometimes it’s difficult to get supporting data for a problem that you want to solve and consequently find it difficult to build a case for it. In this scenario, you can still build a case through qualitative insights (usually from User Research) and insights from any similar problems in the past. If there’s nothing still, look at the market and the competitors.

  3. Try to have a clear understanding of how the problem you’re solving or product you’re building is related to your company’s North Star metric. Without this understanding, you may optimize for the wrong outcomes and have no effect or a negative effect on the North Star metric.

  4. Product goals and product metrics go hand in hand, and these are one of the most important things that should be clearly thought through before you start building the product.

  5. Define what success looks like in terms of metrics. Is it a 5% improvement over the baseline or 2x? Share this with important stakeholders and keep everyone aligned on this. This helps you measure the product's success easily post-release and ensures everyone is on the same page about it.

  6. The most common mistake people make is just measuring target metrics without having an all-around view of the impact. Always think of what all can get affected (both positively and negatively) by what you’re building and define your metrics accordingly. E.g., adding more ads to your product can increase revenue, but it can also reduce user retention.

  7. Develop a framework that works for you, keeping the above points in mind, and try to stick to it for every product or feature that you build. It’s easier than having to think from scratch every time, and also makes it easier for your analyst to set up tracking.

  8. Always keep an eye on metrics that affect user experience like loading times, errors, app crashes, ANRs, etc., on an ongoing basis and also with any new changes that you release.

  9. Remember, as a PM, you’re the one responsible for successful product outcomes. If your developer hasn’t added the right analytics logging or if your analyst is not tracking the right metrics, it’s on you. The best way to ensure everything gets covered is to ensure everyone knows 'Why' and 'How' clearly, so that they can also contribute and suggest if something’s missing from your end.

  10. It’s very important for PMs to have a working knowledge of various analytics tools like Mixpanel, Amplitude, etc. It makes you self-sufficient and increases your decision-making speed without having to wait for analysts to help you. If you can also use SQL, then you’re gold.

  11. One more common mistake new PMs make is thinking A/B tests are just about a % improvement over the control group. There are a lot of other parameters (sample size, confidence intervals, p-value…) to look at to know if what you’re seeing is significant and will have the same impact when rolled out to all users.

Thanks for reading, and

do add your thoughts/insights/suggestions on the same!