"Daily active users dropped 10% yesterday. What happened?" Some version of this question shows up in almost every product manager and data analyst interview loop. So does its sibling: "How would you measure success for this feature?" These questions test whether you can think in systems, separate real problems from noise, and pick metrics that actually reflect value. This guide gives you a structure for both types and five worked examples.
Two types of metrics questions
- Root cause: a number moved. Find out why. (DAU dropped, conversion fell, latency spiked, revenue rose oddly.)
- Goal setting: pick the right metrics for a product or feature. (How would you measure success for Instagram Stories? What's the north star for a food delivery app?)
The two are linked. You can only diagnose a drop well if you understand how the metric is built and what drives it.
A structure for root cause questions
- Clarify the metric. How is DAU defined? Over what time window did it drop? Compared to what: yesterday, last week, the same day last year?
- Check whether it's real. Tracking bugs, logging changes, a new app version that stopped firing events, a data pipeline delay. A surprising number of real world drops are data problems.
- Check the calendar. Holidays, weekends, seasonality, major events, daylight saving time.
- Segment. Break the metric down by platform, app version, country, new versus returning users, acquisition channel and feature. A real drop is usually concentrated somewhere.
- Look inside first, then outside. Internal: releases, experiments, outages, pricing or policy changes, marketing campaigns ending. External: competitors, app store changes, news, regulation.
- Form a hypothesis, then say how you'd confirm it. Name the data you'd pull and what result would prove or disprove it.
- Recommend next steps. Roll back, fix, communicate, or monitor.
Example 1: DAU dropped 10% yesterday
Clarify: "DAU is unique users who opened the app at least once that day? And it's 10% below the same weekday last week, not just below yesterday?" (Comparing to the same weekday rules out normal weekly cycles.)
Is it real? "First I'd check whether events are arriving normally. Did we ship an app version yesterday? Did logging volume drop in a way that doesn't match server traffic? If server requests are flat but logged opens dropped, it's a tracking bug."
Calendar: "Was yesterday a holiday in a big market?"
Segment: "Suppose the drop is entirely on Android, on the newest app version, across all countries. That points strongly to the release."
Hypothesis: "The new Android version either crashes on launch for some devices or broke the event that counts an open. I'd check crash rates by version and device, and compare server side session counts with client events."
Next steps: "If it's crashes, halt the rollout and ship a fix. If it's logging, fix the event and annotate the dashboard so nobody misreads the dip later."
Example 2: Checkout conversion fell from 3.2% to 2.6%
Break conversion into its funnel: product view to add to cart, cart to checkout start, checkout start to payment, payment to success. "Suppose cart and checkout start rates are steady, but payment success fell sharply for one card network. That points to a payment provider issue, not demand." Check provider error codes, then route that network to a backup processor while it's fixed.
A good answer also asks whether traffic mix changed. If a new ad campaign brought lots of low intent visitors, conversion falls even if nothing broke. Segment by channel to check.
Example 3: Signups up 40%, but revenue flat
"Before celebrating, I'd check signup quality. Are the new signups from one channel, one country, or one referral promo? Are they activating?" A common cause: a promo or bot traffic inflating signups without real users. Look at activation rate and day 7 retention for the new cohort compared with the old one. If they're far lower, the growth isn't real, and you might be paying for it.
A structure for goal setting questions
- State the product's goal in one sentence: what value it creates for users and for the business.
- Map the user journey: discover, try, get value, come back, pay or share.
- Pick a north star: one metric that rises when users get real value. It should be a rate or count of a valuable action, not a vanity number.
- Add supporting metrics for each step of the journey.
- Add guardrails: metrics that must not get worse, like latency, complaints, unsubscribes or other products' usage.
- Name the tradeoffs: where metrics conflict and how you'd decide.
Example 4: How would you measure success for a "save for later" button in a shopping app?
Goal: help shoppers come back and buy items they weren't ready to buy yet.
North star: purchases that come from saved items within 30 days, per active user.
Supporting: share of product views that end in a save; share of saved items revisited; time from save to purchase.
Guardrails: overall conversion rate (saving shouldn't replace buying now), and add to cart rate.
Tradeoff: "If saves cannibalize immediate purchases, total revenue could fall even as saves rise. I'd run an A/B test and watch revenue per user, not just saves."
Example 5: What's the north star for a food delivery app?
Candidates often say "orders". Better: orders delivered on time and as expected, per week. It captures demand and the quality that keeps customers coming back. Supporting metrics: new customer first orders, repeat order rate, restaurant and courier supply in busy hours. Guardrails: delivery time, refund rate, courier earnings per hour.
Tips if you're interviewing for a data role
- Say how you'd pull the data, often in SQL. Interviewers love "I'd group sessions by app version and day and compare the share of sessions with zero events".
- Talk about statistical noise. A 2% daily swing may be normal. Know the metric's usual variance before calling a drop.
- Mention Simpson's paradox: an overall rate can fall even when every segment improved, because the mix shifted.
More root cause scenarios to practice
- Average session length doubled overnight.
- App store rating fell from 4.6 to 4.1 in two weeks.
- Email open rates fell 25% this month.
- Support tickets rose 30% after a release.
- Average order value rose 12%, but orders fell 8%.
- Page load time is up 40% for one country.
- Free trial signups are up, but trial to paid conversion is down.
- Weekly active creators fell, but viewers are flat.
For each one, write down the first three things you'd check and what result would point to each cause. Session length doubling, for instance, is often a tracking bug where sessions stop ending properly.
Worked example: email open rates fell 25%
Is it real? Mail privacy features in some email apps load images automatically, which inflates or distorts opens. A change in how a major provider handles that can move open rates without any change in real reading. Check clicks, which are more reliable, and check whether the drop is concentrated in one email client.
Internal changes: a new sending domain, a template change that broke the tracking pixel, a list import that added cold contacts, or more frequent sends causing fatigue.
Deliverability: a spike in spam complaints can push mail into spam folders. Check bounce and complaint rates by provider.
Conclusion shape: "If clicks are flat and the drop is all in one client, it's a measurement change. If clicks fell too and complaints rose after a list import, it's a deliverability problem: pause sends to that segment and clean the list."
Build a metric tree before the interview
For any product you might be asked about, sketch how its main metric breaks down. For an online store: revenue equals visitors times conversion rate times average order value. Each of those splits further: visitors by channel, conversion by funnel step, order value by items per order and price per item. With a tree in your head, root cause questions become a walk down the branches.
How you'd pull it in SQL
Data interviews often follow up with "how would you check that?" A typical answer for the DAU drop:
SELECT app_version, event_date, COUNT(DISTINCT user_id) AS dau
FROM events
WHERE event_date BETWEEN '2026-10-20' AND '2026-11-03'
GROUP BY app_version, event_date
ORDER BY event_date, app_version;
Splitting by version and day shows immediately whether the drop lines up with a release.
Questions
How do I answer 'DAU dropped 10%'?
Clarify the definition and comparison, check whether the drop is real or a tracking issue, check the calendar, segment by platform, version, country and user type, then form and test a hypothesis and recommend next steps.
What is a north star metric?
One metric that rises when users get real value from the product, such as orders delivered on time per week for a delivery app.
What are guardrail metrics?
Metrics that must not get worse while you improve your main metric, like latency, refund rate or unsubscribes.
What is Simpson's paradox?
When an overall rate moves in one direction while every segment moves in the other, because the mix of segments changed.
Are metrics questions asked in data analyst interviews?
Yes. Data analyst loops often include a metrics or root cause case, usually with a follow up about how you'd pull the data in SQL.