Metrics
Just as biologists measure vital signs to assess organism health, our framework uses Metrics - the quantifiable measurements that objectively evaluate performance across Units and Variants.
What is
Just as biologists measure vital signs to assess organism health, our framework uses `Metrics` - the quantifiable measurements that objectively evaluate performance across Units and Variants.
## What is a Metric?
A Metric is a precisely defined measurement that captures a specific aspect of Unit behavior or system performance:
- **Quantifiable**: Represented as a numerical value
- **Comparable**: Consistent across Units and Variants
- **Temporal**: Anchored to specific timeframes or events
- **Actionable**: Directly tied to business or product outcomes
## Key Characteristics
1. **Metric Architecture**
- Primary metrics directly measure experiment hypotheses
- Secondary metrics capture related impacts and side effects
- Guardrail metrics protect against negative consequences
- Leading indicators predict long-term outcomes
- Lagging indicators confirm ultimate impacts
2. **Measurement Framework**
- Point-in-time snapshots capture specific moments
- Time-boxed metrics measure within defined periods (1d, 7d, 30d)
- Cumulative metrics track lifetime totals
- Rate metrics measure frequency or velocity
- Ratio metrics compare related measurements
3. **Implementation Strategy**
- Event-based tracking for user interactions
- Time-series data for temporal analysis
- Aggregation pipelines for computational metrics
- Statistical transformations for derived insights
- Metadata enrichment for contextual understanding
4. **Analysis Capabilities**
- Segmentation by Unit characteristics
- Comparison across Variants
- Statistical significance testing
- Distribution analysis beyond averages
- Correlation and causation assessment
## Critical Metric Categories
1. **Acquisition Metrics**
- Visit-to-signup conversion rate
- Cost per acquisition
- Traffic source effectiveness
- Landing page performance
2. **Activation Metrics**
- Feature discovery rate
- First value moment achievement
- Onboarding completion rate
- Initial engagement depth
3. **Engagement Metrics**
- Active usage frequency
- Feature utilization rates
- Content creation/consumption
- Session duration and depth
4. **Retention Metrics**
- Return rate (1d, 7d, 30d)
- Churn prediction indicators
- Reengagement effectiveness
- Long-term retention curves
5. **Revenue Metrics**
- Conversion to paid
- Average revenue per user
- Lifetime value
- Expansion revenue
## Evolutionary Intelligence
Metrics create the fitness function for product evolution:
1. Well-defined metrics establish clear selection criteria
2. Balanced metric sets prevent local optimization
3. Long-term metrics guide strategic direction
4. Short-term metrics enable rapid iteration
5. Metric hierarchies connect tactical actions to strategic outcomes
The ultimate goal is a comprehensive measurement system that accurately captures both immediate impacts and long-term value, enabling data-driven decisions that optimize for genuine user and business outcomes.By Eduarda Ferreira