Dimensional attributes issue
When creating a Unit, we automatically assign a variant for each active experiment. However, we do not have a structure for attributes/traits of an object.
For example, a Unit may represent a visit
When creating a Unit, we automatically assign a variant for each active experiment. However, we do not have a structure for attributes/traits of an object.
For example, a Unit may represent a visitor to my website, and a dimensional attribute of the unit would be Device type.
Device type is something inherent to the Unit, not something we control. For this reason, it functions differently than an experiment. But the statistical methods and relationships are essentially identical to Experiment:
- Experiment has Variants, and we create an Assignment tying a unit to an experiment variant
- Dimensional attributes also have variants, and we'll need to create assignments tying a unit to a dimensional attribute variant based on data we collect.
Dimensional attributes also function differently than measurments, in that they are inherent to the Unit and not something we are measuring. Additionally, dimensional attributes are often non-numeric categorical.
So, we need to figure out the best way to add in dimensional attributes.
Given the context that the lab intends to evolve a product over time, similar to how an organism evolves over time via traits and natural selection, it would seem to me that nature does not distinguish between the concept of an experiment vs. a dimensional attribute. Instead, everything is a trait.
Options:
- rename Experiment to Trait, add a `type` field to Trait (controlled vs. uncontrolled). Re-think how we use weights in variants.
- Add a new model, called Dimension or trait, with the same structure as experiments, variants, and assignments.
**Case-study: Ad campaigns**
Ad campaigns are a type of experiment in which we have control over the experiment and variant, but do not have control over assignment.
When a user clicks on an ad from google, we do not choose which ad they see/click on. Google does this. We can only observe the ad from which they originate from upon parsing the google click ID (GCLID).
Additionally, some users do not originate from an ad for which we have an experiment, meaning they should not receive an assignment for this trait at all.
For this reason, experiments/traits can either be controlled by us (we randomly select a variant and assign) or uncontrolled (assignment is determined by some other means).
For this reason, experiment is fundamentally ALSO a trait, and experiments must implement a `type` field with controlled vs. uncontrolled.
TODO:
- [x] Add `type` field on `experiment`: `random`, `observed`, or `adaptive`**Socra Summary: "Add Dimensional Attributes"**
On February 21, 2025, Mike Morton initiated the Socra titled "Add Dimensional Attributes," focusing on enhancing the structure of Units to incorporate dimensional attributes akin to traits in evolutionary product development. The aim was to better understand how Units, such as website visitors, could include inherent characteristics like Device type, which function differently from controlled experiments.
The discussion highlighted the need for a systematic approach to assign dimensional attributes and their variants, drawing parallels between experiments and these attributes. It was emphasized that while experiments have controlled and uncontrolled variants, dimensional attributes are inherently non-numeric and categorical.
A case study on ad campaigns was presented, illustrating the distinction between controlled experiments (where assignment is selected) and uncontrolled traits (where assignment is dictated externally, such as by Google). This led to the conclusion that experiments should be treated as traits, necessitating the addition of a `type` field to differentiate between controlled and uncontrolled assignments.
The Socra concluded with a task to implement the `type` field on experiments, marking a significant step toward integrating dimensional attributes into the overall product development process.
Key Tags:
- Dimensional Attributes
- Experiments
- Traits
- Variants
- Assignments
- Device Type
- Ad Campaigns
- Controlled vs. Uncontrolled
- Statistical Methods
- Evolutionary Product DevelopmentBy Mike Morton