Instrumenting a Product: The Day-One Metrics Baseline

Instrumenting a Product: The Day-One Metrics Baseline

If you can't answer 'better or worse this week' in sixty seconds, you're on vibes.

4 min read

This is the play for establishing the metrics floor before launch, not after. The test is simple: if you cannot answer "is the product getting better or worse this week" in sixty seconds, you are operating on vibes. Instrumentation is part of "done." A feature that ships without analytics is half-shipped.

When to use this play#

Use it at the start of any product where you will need to know whether it is working, which is to say, any product worth building. The cost of instrumenting on day one is small; the cost of reconstructing a baseline after launch is large and the early data is gone forever.

How to run it#

1. Pick one north-star metric. Choose the single number that best represents user value created. It is not signups and not page views; it is the thing that means a user got what they came for. One product, one north star.

2. Pick one tool per concern. Resist analytics sprawl. Choose a tool for product analytics, a tool for errors, a tool for performance, and stop. Building custom analytics in-house is almost never the right call; it is a product unto itself that distracts from yours.

3. Stand up the six-category baseline stack. Cover all six so you see the whole funnel and the system underneath it:

  • Acquisition: traffic source and signup conversion.
  • Activation: the first time a user completes the core action, the "aha" moment.
  • Retention: DAU, WAU, and MAU plus cohort curves.
  • Engagement: key actions per session and how often users return.
  • Revenue: MRR or ARR, churn, expansion, and LTV.
  • Operational: latency, error rate, and cost per request.

4. Adopt an event-naming standard. Name events noun_verb in past tense, lowercase with underscores, and give each event exactly one canonical name. For example, order_placed or report_generated. One name per event, everywhere.

5. Attach required properties to every event. Each event carries:

  • user_id, hashed, never raw PII.
  • timestamp in UTC.
  • source.
  • environment.
  • app_version.

6. Default your cohorts. Use weekly cohorts for the first 90 days, then monthly. Track day-1, day-7, and day-30 retention.

7. Make dashboards visible. Put the dashboards in front of the whole team and the sponsor, not in a private corner. Shared visibility is what turns metrics into decisions.

Common traps#

  • Treating analytics as a post-launch add-on. The early baseline cannot be backfilled; instrument before launch or lose it.
  • Tracking vanity metrics. Signups and page views feel good and tell you little. Anchor on the north star.
  • Analytics sprawl. Five overlapping tools means five sources of disagreement. One tool per concern.
  • Building analytics in-house. Almost never worth it. Buy the boring tool and spend the effort on your product.
  • Leaking PII into events. Hash user identifiers, respect consent, and keep raw PII out of the analytics pipeline.

Experiment guardrails#

When you run experiments, hold them to a standard:

  • Write the hypothesis first, in the form "we believe X will change Y by Z."
  • Define the success metric and the duration before you start, not after you peek.
  • Document the results, including null results. A well-run experiment that found nothing is still knowledge worth keeping.

Signals it's working#

  • Anyone on the team can answer "better or worse this week" in under a minute.
  • Events have consistent names and properties, so analysis does not start with cleanup.
  • Experiments end with documented outcomes rather than abandoned dashboards.

How it ends#

This play does not really end; it becomes the floor everything else stands on. Once the baseline is live, instrumentation is simply part of "done" for every feature that follows.