chore(schema): add base schema for fenix_derived#9706
Conversation
This comment has been minimized.
This comment has been minimized.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
This PR adds a new dataset-level base-schema file fenix_derived.yaml with names, descriptions, and (mostly) types for 84 fields shared across fenix_derived tables. These base schemas are consumed almost entirely via !include-field-description, so the description text is what actually propagates into downstream schema.yaml files. The descriptions are generally clear and detailed. Two points below: some descriptions bake in a point-in-time claim about current nullness that will go stale, and the type key is applied inconsistently across fields.
| A value of 0 indicates no session starts were observed in the window. | ||
| type: INT64 | ||
| - name: days_sent_metrics_ping_bits | ||
| description: An integer bitfield representing the days within a rolling window on which the client sent a metrics ping. This field is currently |
There was a problem hiding this comment.
suggestion: These descriptions encode a transient observation about the current data state ("currently fully null, indicating it is not yet populated"), which will become misleading once the field is populated. Since this base schema is reused across many fenix_derived tables via !include-field-description, a stale claim propagates everywhere. Describe what the field is meant to represent and drop the point-in-time nullness claim. The same applies to is_large_device (lines 169-170), marketing_notifications_allowed (line 193), notifications_allowed (line 233, "currently null for all observed rows"), and meta_attribution_app (line 202, "null for the vast majority of clients").
There was a problem hiding this comment.
removed null references in descriptions
| @@ -0,0 +1,331 @@ | |||
| fields: | |||
| - name: activated | |||
There was a problem hiding this comment.
nitpick: activated omits type while 81 of the 84 fields include it; app_version (line 54) and first_run_date (line 145) are the other two without it. Based on their descriptions these are BOOLEAN, STRING, and DATE respectively. Add the missing type for consistency (or drop type throughout, as the sibling firefox_desktop_derived.yaml and subscription_platform_derived.yaml base schemas do).
There was a problem hiding this comment.
dropped the fields
There was a problem hiding this comment.
@phil-lee70 i'm working on the agent re-naming this to dataset_schema.yaml - should we go ahead and change it here?
There was a problem hiding this comment.
sure thing! i'll change it in this PR
This comment has been minimized.
This comment has been minimized.
chelseyklein
left a comment
chelseyklein
left a comment
There was a problem hiding this comment.
These look good. left one comment about possibly changing base schema name to dataset_schema.yaml - nonblocking
Integration report
|
3 participants
Base Schema
Generated
fenix_derived.yamlwith descriptions for 84 shared fields.