Product managers don’t need to be data scientists, but they must know enough data science to ask the right questions and interpret the answers.
Data is the fuel that powers modern product decisions. The actual job of a PM is not just to collect data, but to translate it into insight — to know what question to ask, how to measure it, and how to act on the answer.
Without clear objectives and measurable success criteria, data is noise. You must start by defining what problem you want to solve and what success looks like in quantitative terms.
This lesson walks you through the core components of data science that every PM should master — from framing hypotheses and running A/B tests to understanding predictive analytics and the tools that unlock data’s value.
Define your objective and success metrics before collecting data
The first step in any data-driven product initiative is to get crystal clear on your objective.
What is the problem you are trying to solve?
What do you want to accomplish?
How will you know if you succeeded?
Talvinder emphasizes this foundation repeatedly: “Define your objective: what is the problem? What do you want to accomplish? Define success. How do you know that you met your objective? How does success look like? How do you measure it quantitatively?”
For example, if your product team believes new users take too long to understand how to use the app, your objective might be: "Reduce time to first key action by 20% within 4 weeks." Your success metric could be the average time between the login event and the first meaningful action event in your analytics tool.
Without this clarity, you might collect data endlessly without knowing which signals matter.
Hypothesis-driven product management: the PM as a scientist
Product management is an experiment-driven discipline. You observe, hypothesize, test, and learn.
Talvinder explains the process with a simple use case:
- Observation: New users take too long to start using the product.
- Hypothesis: Introducing an onboarding video will reduce this time.
- Action: Implement the video feature.
- Measurement: Compare funnels before and after the video release.
This is classical hypothesis testing — a core statistical principle. The PM’s job is to design experiments that confirm or reject hypotheses with data, not just intuition.
Hypothesis testing and p-values
A/B testing is the most common way to run hypothesis tests in product. It compares two variants (control and treatment) to see which performs better on a key metric.
Talvinder points to resources like the YouTube video on p-values to understand the statistical significance behind the differences you observe.
In essence, hypothesis testing quantifies the probability that your observed effect is due to chance versus a real impact of your change.
A/B Testing: Let data drive decisions, not opinions
A/B testing is non-negotiable for modern PMs. Talvinder shares the story of Obama’s 2008 campaign:
“Obama raised $60 million more by running a simple experiment — testing different email subject lines and call-to-action text. Variations using the word ‘Change’ consistently outperformed others.”
This demonstrates the power of data-driven optimization: small changes, validated by rigorous testing, can yield massive impact.
Why A/B testing matters
- Kills the HiPPO syndrome (Highest Paid Person’s Opinion). Subjective debates end with objective data.
- De-risks product decisions by validating impact on a small scale before full rollout.
- Reveals behavioral truths beyond what surveys or feedback can capture.
- Enables continuous, iterative product improvement.
Talvinder underscores: “Most product ideas fail to deliver expected improvements. A/B testing makes failures cheap learning opportunities, not expensive disasters.”
Basic components of A/B testing
- Define a clear hypothesis.
- Identify the key metric to optimize.
- Randomly assign users to control or treatment groups.
- Run the test long enough to reach statistical significance.
- Analyze results and decide the next step.
Predictive analytics: forecasting the future with data from the past
Beyond descriptive analytics and A/B testing lies predictive analytics — using historical data to forecast future outcomes.
Talvinder explains:
“Predictive analytics are gaining popularity. Customer lifetime value (CLTV) is an example — predicting how much a customer will spend over time. Next best offer and sales forecasts are other common use cases.”
Predictive models use statistical techniques like regression and decision trees, requiring strong data inputs and assumptions.
The biggest barrier is often lack of good data:
- Clean, consistent, and integrated customer data across channels.
- Unique customer IDs and comprehensive purchase histories.
- Relevant demographic and behavioral attributes.
Talvinder notes: “If you’ve built a single customer data warehouse, you have an incredible asset for predictive analytics.”
Data collection: what data do you need and how do you get it?
Defining your objective and hypothesis leads to questions about data:
- What variables support your objective?
- What data sources can provide these variables?
- When and how do you collect the data?
Talvinder lists the key steps:
“Define and carry out strategies to collect data — how do you get it? Where do you get it? When do you get it?”
Common data sources include:
- Product analytics tools (Mixpanel, Segment, Amplitude) capturing user events.
- CRM and sales systems.
- Customer feedback and surveys.
- External data sources, if relevant.
You must also understand the types of data — quantitative (numbers, counts, times) and qualitative (interviews, open feedback).
Tools every PM should know for data analysis
Talvinder is clear: you don’t need to be a data scientist, but you must know the tools that turn data into decisions.
“Google Sheets and Excel work to some extent. If you belong to the ‘I don’t need anyone’ camp, you may learn R or Python for basic analysis.”
Beyond spreadsheets, visualization and BI tools like Tableau, QlikSense, and Looker help you explore data patterns.
But data analysis starts with getting data out of databases:
“How will you get the data? Mostly MySQL or MongoDB. To generalize, from relational or non-relational databases. You need to know how to query databases.”
Talvinder promises an introduction and comparisons between these tools, emphasizing they fall into four categories:
- Discovery
- Analysis
- Qualitative
- Quantitative
- Reporting
Descriptive analytics: instant answers to what happened
Descriptive analytics answers questions like “What happened?” and “What is happening now?”
Talvinder explains:
“Descriptive analytics usually happen instantly or near-instantly. A/B testing and hypothesis testing belong here.”
For example, you might measure:
- Conversion rates on a landing page.
- Drop-off rates in a signup funnel.
- Average session duration.
These metrics provide a baseline for understanding product performance and spotting issues.
Predictive analytics vs descriptive analytics: different jobs, both essential
Talvinder distinguishes:
- Descriptive analytics tells you what happened. It uses tools like A/B testing to get near-instant answers.
- Predictive analytics forecasts what will happen, using statistical models and historical data.
Both are essential. Descriptive analytics helps you validate hypotheses and optimize current features. Predictive analytics helps you anticipate user behavior and plan ahead.
Examples of data science in action: from Netflix to Indian startups
Talvinder points to famous case studies that illustrate the impact of data science on product:
-
Obama’s $60 million fundraising boost by simple A/B testing of email content.
Read the full article -
Netflix Prize: A $1 million competition to improve their recommendation algorithm by 10%.
Wikipedia page
Thrillist article
Why Netflix didn’t use the winning entry -
Target’s predictive model that inferred a teen girl was pregnant before her father did, based on purchasing patterns.
Forbes article
These examples show how data science transforms product decisions from guesswork into measurable impact.
The PM’s role: asking the right questions and interpreting the answers
Talvinder is clear about what PMs need to do:
- Understand the basics of statistics, hypothesis testing, and data collection.
- Be able to translate business questions into measurable hypotheses.
- Know how to interpret results and communicate them effectively.
- Collaborate with data scientists and analysts without needing to be one.
He quotes Nate Silver’s advice: “Getting your hands dirty with the dataset is far better than spending too much time reading.”
Field Exercise: Define your product objective and success metric (10 min)
Pick a recent or upcoming product initiative. Write down:
- The specific problem you want to solve.
- How you will measure success quantitatively.
- What data you need to collect.
- How you will collect it (tools, sources).
If you cannot do this clearly, pause and revisit your objective before moving forward.
Field Exercise: Design a simple A/B test (15 min)
Choose a product feature or page you want to improve.
- Formulate a hypothesis (e.g., “Changing the button color will increase click-through rate”).
- Define the success metric (e.g., CTR on the button).
- Outline how you would split users into control and treatment groups.
- Describe how you would interpret the results.
Where to go next
- Learn to conduct effective user research: User Research Methods
- Master product metrics and KPIs: Metrics and KPIs
- Understand experimentation and hypothesis testing: Experimentation and A/B Testing
- Get introduced to data querying and visualization tools: Data Tools for PMs
PL alumni now work at Razorpay, Swiggy, Flipkart, PhonePe, and dozens of other leading Indian tech companies.