Defining Models
The experimentation framework is called the "Lab", and serves as our means of split testing, characterization, and will eventually be its own consumer facing product with an API. Our backend is django
The experimentation framework is called the "Lab", and serves as our means of split testing, characterization, and will eventually be its own consumer facing product with an API. Our backend is django, so use django models for example schemas.
Here are the models:
- `Product`: Something we're producing and want to improve. Unique by `user` and `key`. Also includes `status` (enabled, disabled, removed).
- `Unit`: A single unit of production or unit under test. Foreign key to `product`.
- `Step`: A step in a production process. Unique by `product` and `key`.
- `Experiment`: Products undergo experiments. Unique by `product` and `key`. Also includes `status` (enabled, disabled, removed).
- `Variant`: Experiments have variants. Unique by `experiment` and `key`. Also includes `status` (enabled, disabled, removed).
- `Assignment`: An experiment variant assigned to a `unit`. Unique by `experiment` and `unit`. One unit can be assigned to multiple experiments.
- `Metric`: A measurable characteristic of a product. In engineering contexts, metrics are also specs. Unique by `product` and `key`. Can have lower limit, nominal, upper limit, and unit.
- `Measurement`: A single measurement for a particular `unit` for a `spec`. Unique by `metric`, `unit`.On March 6, 2025, Mike Morton initiated a Socra titled "Defining Models," focusing on the development of an experimentation framework known as "Lab." This framework is designed to facilitate split testing and characterization, ultimately evolving into a consumer-facing product with an API. The backend utilizes Django, employing Django models to create a structured schema for various components of the framework.
Key models were defined to support the experimentation process:
- **Product**: Represents items being produced and improved, uniquely identified by user and key, with a status indicating its operational state (enabled, disabled, or removed).
- **Unit**: Denotes individual production items or test units, linked to a specific product.
- **Step**: Outlines stages in the production process, uniquely identified by product and key.
- **Experiment**: Associates products with unique tests, identifiable by product and key, also reflecting their operational status.
- **Variant**: Represents different variations within an experiment, similarly tracked by experiment and key.
- **Assignment**: Links a variant to a unit, ensuring unique assignments for each unit across multiple experiments.
- **Metric**: Defines measurable characteristics of a product, with specifications for lower and upper limits, as well as units of measurement, uniquely identified by product and key.
- **Measurement**: Records individual measurements for a unit against a specific metric, ensuring uniqueness through metric and unit identifiers.
This structured approach not only lays the groundwork for future experimentation in product development but also establishes a robust framework for measurement and analysis, critical for enhancing consumer-facing products.By Mike Morton