EnVar microschema · class
Data Layout
DataLayout
How the companion data file (CSV / parquet) is laid out and how this record's values are located inside it. Separates the file-layout concern from variable identity: VariableIdentity.variable_name says what the variable is; DataLayout says which column (and, for long format, which rows) carry its values. One per record.
Where it sits
Composed intoEnvironmentalExposureRecord
Fields
Wide means one column per measurement type (a tmax column, a vp column, and so on); long means one row per measurement, with a shared value column and a label column saying which measurement each row is. Long is also known as "tidy" format.
Whether the companion file is wide (one column per variable) or long (one value column, variables discriminated by row). Required.
Examples
wide — one column per variable, e.g. a tmax column in a Daymet extract
long — tidy format with a shared value column, e.g. tract-level PM2.5
Allowed values
wide One column per variable; column name identifies the variable
long One shared value column; the variable is discriminated by a row value in a va...
Covered by — 5 standards
Why it matters & mappings
Every other column binding in this class is read relative to the orientation: in a wide file value_column names a variable-specific column, in a long file it names a shared column whose rows must be filtered. Without this flag a consumer cannot interpret the bindings and so cannot reliably locate this record's values in the companion file.
See also: https://doi.org/10.18637/jss.v059.i10
This is simply the name of the spreadsheet column where the actual numbers live — for example a column headed tmax holding daily maximum temperatures, or a generic column headed value in a long table.
Name of the column carrying this record's exposure values, e.g. tmax in a wide file or value in a long file. Required.
Examples
tmax — wide file — the variable's own column carries the values
value — long file — the shared value column
Covered by — 5 standards
Why it matters & mappings
The record deliberately carries no inline observation result — the values live only in the companion CSV/parquet file. Without this column binding, a validator or downstream pipeline has no way to find the values the sidecar describes, so the sidecar is unverifiable and the data unusable.
In a long table there is a column — often literally called variable — that says what each row measures (tmax, vp, ...). This slot names that column. Wide files do not have one, which is why this only applies to long orientation.
For long orientation: name of the column that discriminates variables (e.g. variable). Not applicable to wide files.
Example
variable
Why it matters & mappings
In a long file many variables share one value column and are told apart only by a discriminator column. Without naming that column, a consumer cannot separate this record's variable from every other variable in the file, so the value binding is ambiguous for all long-format data.
See also: https://doi.org/10.18637/jss.v059.i10
This is the label to filter on: keep only the rows where the variable column says, for example, tmax, and you have exactly this record's measurements. It usually matches the variable's short name, but does not have to.
For long orientation: the value in variable_column that selects this record's rows (e.g. tmax). Often, but not necessarily, equal to VariableIdentity.variable_name.
Example
pm25_annual — selects the PM2.5 rows in a long tract-level file
Why it matters & mappings
Knowing which column discriminates variables is not enough — a consumer also needs the label value that selects this record's rows. Without the key, every row of a long file is a candidate and this record's values cannot be filtered out of the shared value column.
See also: https://doi.org/10.18637/jss.v059.i10
This names the ID column that says who (or where) each row belongs to — a patient identifier like subject_id, or a place identifier like a census-tract code.
Name of the column carrying the opaque subject / cohort identifier the record-level subject refers to (e.g. subject_id).
Examples
subject_id — patient-level extract
tract_id — census-tract-level extract
Covered by — 5 standards
Why it matters & mappings
The whole point of an exposure sidecar is to be joined back to a health-data layer. Without knowing which column carries the subject or cohort identifier, the values cannot be attached to the people or places the record-level subject refers to, and the linkage step becomes guesswork.
This names the column that gives each measurement its "when" — a date column for daily data, or a year column for annual summaries.
Name of the column carrying the observation date / timestamp (e.g. date).
Examples
date — daily data
year — annual aggregate
Covered by — 5 standards
Why it matters & mappings
Without knowing which column carries the date or timestamp, each value cannot be placed in time, so exposure values cannot be aligned with clinical events — which day's exposure goes with which health record — and any lag or window analysis is impossible.
Some datasets include a "plus or minus" column next to each value — an estimate of how far off each number might be. This slot names that column so the uncertainty travels with the data instead of being lost.
Name of the column carrying per-value uncertainty (e.g. pm_se, tmax_stderr). Its semantics (uncertainty type, units) live in the Uncertainty microschema. Null with reason for products whose per-value uncertainty exists upstream but is not surfaced.
Example
tmax_stderr
Covered by — 5 standards
Why it matters & mappings
Products like Daymet ship per-value standard errors that most pipelines silently drop. Without this binding, downstream analyses cannot find or propagate the measurement error attached to each value, and dropped uncertainty is indistinguishable from uncertainty that never existed.
A blank cell or a 0 can mean three very different things: there was never any data here, we could not compute a value here, or the value is genuinely zero. Each needs different treatment. This names the column that tags every value with which kind it is, so nobody mistakes "no data" for "zero exposure".
Name of a per-value column carrying a null-semantics indicator drawn from NullSemanticsEnum (present / structural_null / derived_null / true_zero; see envar_common). Bound here in DataLayout — mirroring the existing per-value value_uncertainty_column / quality_flag_column pattern — because the distinction is per-value: whether a given blank or 0 means "no source data", "aggregation impossible", or "measured zero" varies row by row and so must be carried alongside the values, not declared once for the dataset. Null with reason when the producer does not distinguish null kinds. The vocabulary itself lives in the Uncertainty / shared layer (NullSemanticsEnum).
Example
null_kind — per-value column tagging each cell present / structural_null / derived_null / true_zero, next to the value column
Covered by — 5 standards
Why it matters & mappings
A blank or a 0 is ambiguous, and the three kinds of "missing/zero" have OPPOSITE correct handling — structural_null and derived_null must be dropped (the latter also excluded from denominators) while true_zero must be kept as a real zero; conflating them biases the health-effect estimate. Nulls should be excluded from percentile denominators for exactly this reason, so binding a per-value null-kind column is what lets a consumer handle each cell correctly instead of reading missingness as low or zero exposure.
Some data files helpfully include the same measurement twice, in two units — like a column in Kelvin next to one in Celsius. This names that second column so it is accounted for rather than mysterious.
Name of a second column carrying the same values in the source's native units (VariableIdentity.native_units_ucum), when the companion file ships both, e.g. Amadeus's value_kelvin next to value_celsius. value_column stays the binding for the values in units_ucum; this binds the native-unit twin.
Example
value_kelvin — Amadeus gridMET output ships Kelvin and Celsius side by side
Why it matters & mappings
When a producer ships both unit representations, binding only one leaves an unexplained column in the data file and hides the cheapest available cross-check (recomputing one column from the other via unit_conversion_formula).
When there is no uncertainty column, this slot says why — for example "the upstream product does not provide one" versus "it exists upstream but was not extracted".
Reason value_uncertainty_column is null.
Example
not_provided_by_source
Allowed values
not_provided_by_source Source product does not produce this information
available_but_not_extracted Source produces this information but the current pipeline does not surface it
upstream_data_not_propagated An upstream tool emitted this information but the current pipeline dropped it...
under_investigation We are working on populating this slot
not_applicable This slot does not apply to this variable / record
Why it matters & mappings
A bare null cannot be audited: it could mean the source has no per-value uncertainty, the pipeline dropped it, or the producer forgot to record it. Stating the reason makes the absence deliberate and lets reviewers tell "unavailable" from "lost", which matters for reproducing the extraction.
A quality flag column is like a traffic light next to each number — good, suspect, or bad — recorded by the data producer. This slot names that column; what the flag codes mean is described in the Uncertainty microschema.
Name of any per-value QA flag column. CF ancillary_variables analogue; the flag vocabulary lives in the Uncertainty microschema.
Example
tmax_qc — per-value QA flag column accompanying a tmax value column
Why it matters & mappings
Without knowing where the per-value QA flags live, consumers cannot filter or down-weight values the producer already marked as suspect, so known-bad measurements flow silently into analyses.
See also: https://cfconventions.org/
When there is no quality-flag column, this slot says why it is absent — for example because the source dataset simply does not publish quality flags.
Reason quality_flag_column is null.
Example
not_provided_by_source
Allowed values
not_provided_by_source Source product does not produce this information
available_but_not_extracted Source produces this information but the current pipeline does not surface it
upstream_data_not_propagated An upstream tool emitted this information but the current pipeline dropped it...
under_investigation We are working on populating this slot
not_applicable This slot does not apply to this variable / record
Why it matters & mappings
Without a stated reason, a missing QA-flag column is ambiguous: the source may publish no flags, or the pipeline may have dropped them. The reason turns a silent gap into a documented decision that a reviewer can check against the upstream product.
When there is no null-kind column, this slot says why — for example because the source never separates "no data" from "real zero" in the first place.
Reason null_semantics_column is null.
Example
not_provided_by_source — the producer does not distinguish kinds of missing / zero
Allowed values
not_provided_by_source Source product does not produce this information
available_but_not_extracted Source produces this information but the current pipeline does not surface it
upstream_data_not_propagated An upstream tool emitted this information but the current pipeline dropped it...
under_investigation We are working on populating this slot
not_applicable This slot does not apply to this variable / record
Why it matters & mappings
A bare null is ambiguous: the producer may not distinguish null kinds at all, or the pipeline may have dropped the indicator. Stating the reason makes the absence deliberate and auditable rather than a silent gap.
Full field reference — every slot, cardinality & inheritance
| Field | Name | Tier | Cardinality / Range | Description |
|---|---|---|---|---|
| Table Orientation | table_orientation |
core | 1 TableOrientationEnum |
Whether the companion file is wide (one column per variable) or long (one `va... |
| Value Column | value_column |
core | 1 String |
Name of the column carrying this record's exposure values, e |
| Variable Column | variable_column |
conditionally core | 0..1 String |
For long orientation: name of the column that discriminates variables (e |
| Variable Key | variable_key |
conditionally core | 0..1 String |
For long orientation: the value in variable_column that selects this record... |
| Subject Column | subject_column |
recommended | 0..1 String |
Name of the column carrying the opaque subject / cohort identifier the record... |
| Time Column | time_column |
recommended | 0..1 String |
Name of the column carrying the observation date / timestamp (e |
| Native Value Column | native_value_column |
optional | 0..1 String |
Name of a second column carrying the same values in the source's native uni... |
| Per-Value Uncertainty Column | value_uncertainty_column |
recommended | 0..1 String |
Name of the column carrying per-value uncertainty (e |
| Reason Uncertainty Column Is Missing | value_uncertainty_column_missing_reason |
optional | 0..1 MissingReasonEnum |
Reason value_uncertainty_column is null |
| Quality Flag Column | quality_flag_column |
optional | 0..1 String |
Name of any per-value QA flag column |
| Reason Quality Flag Column Is Missing | quality_flag_column_missing_reason |
optional | 0..1 MissingReasonEnum |
Reason quality_flag_column is null |
| Null-Semantics Column | null_semantics_column |
recommended | 0..1 String |
Name of a per-value column carrying a null-semantics indicator drawn from `Nu... |
| Reason Null-Semantics Column Is Missing | null_semantics_column_missing_reason |
optional | 0..1 MissingReasonEnum |
Reason null_semantics_column is null |
Conditional rules on this class
| Rule Applied | Preconditions | Postconditions |
|---|---|---|
| slot_conditions | {'table_orientation': {'equals_string': 'long'}} |
{'variable_column': {'required': True}, 'variable_key': {'required': True}} |
Diagram & LinkML source
classDiagram
class DataLayout
click DataLayout href "../../classes/DataLayout/"
DataLayout : native_value_column
DataLayout : null_semantics_column
DataLayout : null_semantics_column_missing_reason
DataLayout --> "0..1" MissingReasonEnum : null_semantics_column_missing_reason
click MissingReasonEnum href "../../enums/MissingReasonEnum/"
DataLayout : quality_flag_column
DataLayout : quality_flag_column_missing_reason
DataLayout --> "0..1" MissingReasonEnum : quality_flag_column_missing_reason
click MissingReasonEnum href "../../enums/MissingReasonEnum/"
DataLayout : subject_column
DataLayout : table_orientation
DataLayout --> "1" TableOrientationEnum : table_orientation
click TableOrientationEnum href "../../enums/TableOrientationEnum/"
DataLayout : time_column
DataLayout : value_column
DataLayout : value_uncertainty_column
DataLayout : value_uncertainty_column_missing_reason
DataLayout --> "0..1" MissingReasonEnum : value_uncertainty_column_missing_reason
click MissingReasonEnum href "../../enums/MissingReasonEnum/"
DataLayout : variable_column
DataLayout : variable_key
name: DataLayout
annotations:
domain_of_use:
tag: domain_of_use
value: environmental_exposure
description: 'How the companion data file (CSV / parquet) is laid out and how this
record''s values are located inside it. Separates the file-layout concern from variable
identity: `VariableIdentity.variable_name` says what the variable is; `DataLayout`
says which column (and, for long format, which rows) carry its values. One per record.'
title: Data Layout
from_schema: https://w3id.org/linkml/microschemas/envar
rank: 1000
slot_usage:
table_orientation:
name: table_orientation
required: true
value_column:
name: value_column
required: true
attributes:
table_orientation:
name: table_orientation
annotations:
tier:
tag: tier
value: core
justification:
tag: justification
value: 'Every other column binding in this class is read relative to the orientation:
in a wide file `value_column` names a variable-specific column, in a long
file it names a shared column whose rows must be filtered. Without this
flag a consumer cannot interpret the bindings and so cannot reliably locate
this record''s values in the companion file.'
explanation:
tag: explanation
value: Wide means one column per measurement type (a `tmax` column, a `vp`
column, and so on); long means one row per measurement, with a shared `value`
column and a label column saying which measurement each row is. Long is
also known as "tidy" format.
covered_by:
tag: covered_by
annotations:
omop_gaia:
tag: omop_gaia
annotations:
extent:
tag: extent
value: absent
status:
tag: status
value: asserted
where:
tag: where
value: no column
note:
tag: note
value: The OMOP external_exposure table is implicitly long, but no
orientation flag is declared as a field — it is a structural property
a consumer must infer.
degauss:
tag: degauss
annotations:
extent:
tag: extent
value: absent
status:
tag: status
value: asserted
where:
tag: where
value: no column
note:
tag: note
value: DeGAUSS's daymet CSV is wide (a `tmax` column) but declares
no orientation field.
amadeus:
tag: amadeus
annotations:
extent:
tag: extent
value: absent
status:
tag: status
value: asserted
where:
tag: where
value: no column
note:
tag: note
value: Amadeus's gridmet_tmmx.csv is wide but declares no orientation
field.
cher:
tag: cher
annotations:
extent:
tag: extent
value: partial
status:
tag: status
value: asserted
where:
tag: where
value: table_column_dictionary.column_tag (Index / Value distinguishes
table structure)
note:
tag: note
value: 'C-HER''s per-column tags (Index vs Value) expose table structure,
though not a wide/long orientation flag. Conservative: partial.'
codata:
tag: codata
annotations:
extent:
tag: extent
value: out_of_layer
status:
tag: status
value: asserted
where:
tag: where
value: no column
note:
tag: note
value: File orientation is an instance-layer layout detail; CODATA's
conceptual layer does not describe physical table shape.
description: Whether the companion file is wide (one column per variable) or long
(one `value` column, variables discriminated by row). Required.
title: Table Orientation
examples:
- value: wide
description: one column per variable, e.g. a `tmax` column in a Daymet extract
- value: long
description: tidy format with a shared `value` column, e.g. tract-level PM2.5
from_schema: https://w3id.org/linkml/microschemas/envar
see_also:
- https://doi.org/10.18637/jss.v059.i10
owner: DataLayout
domain_of:
- DataLayout
range: TableOrientationEnum
required: true
value_column:
name: value_column
annotations:
tier:
tag: tier
value: core
justification:
tag: justification
value: The record deliberately carries no inline observation result — the
values live only in the companion CSV/parquet file. Without this column
binding, a validator or downstream pipeline has no way to find the values
the sidecar describes, so the sidecar is unverifiable and the data unusable.
explanation:
tag: explanation
value: This is simply the name of the spreadsheet column where the actual
numbers live — for example a column headed `tmax` holding daily maximum
temperatures, or a generic column headed `value` in a long table.
covered_by:
tag: covered_by
annotations:
omop_gaia:
tag: omop_gaia
annotations:
extent:
tag: extent
value: full
status:
tag: status
value: verified
where:
tag: where
value: external_exposure.value_as_number (Cel)
evidence:
tag: evidence
value: EnVar/examples/heat/COMPARISON.md §D (Value in human-friendly
°C — ✅ for GAIA)
note:
tag: note
value: The OMOP CDM binds the value to the stable external_exposure.value_as_number
column.
degauss:
tag: degauss
annotations:
extent:
tag: extent
value: full
status:
tag: status
value: verified
where:
tag: where
value: cohort_addresses_geocoded_daymet CSV `tmax` column
evidence:
tag: evidence
value: EnVar/examples/heat/COMPARISON.md §D (Value in human-friendly
°C — ✅ for DeGAUSS)
note:
tag: note
value: DeGAUSS writes the value to a stable `tmax` column on every
row.
amadeus:
tag: amadeus
annotations:
extent:
tag: extent
value: full
status:
tag: status
value: verified
where:
tag: where
value: gridmet_tmmx.csv `value_celsius` column
evidence:
tag: evidence
value: EnVar/examples/heat/COMPARISON.md §D (Value in human-friendly
°C — ✅ for Amadeus)
note:
tag: note
value: Amadeus writes the converted value to a stable `value_celsius`
column.
cher:
tag: cher
annotations:
extent:
tag: extent
value: full
status:
tag: status
value: asserted
where:
tag: where
value: table_column_dictionary column tagged Value (column_tag = Value)
note:
tag: note
value: C-HER identifies the value-carrying column first-class via
the Value column tag.
codata:
tag: codata
annotations:
extent:
tag: extent
value: out_of_layer
status:
tag: status
value: asserted
where:
tag: where
value: no column
note:
tag: note
value: Binding a value to a physical column is instance-layer; CODATA's
conceptual layer does not name file columns.
description: Name of the column carrying this record's exposure values, e.g. `tmax`
in a wide file or `value` in a long file. Required.
title: Value Column
examples:
- value: tmax
description: wide file — the variable's own column carries the values
- value: value
description: long file — the shared value column
from_schema: https://w3id.org/linkml/microschemas/envar
owner: DataLayout
domain_of:
- DataLayout
range: string
required: true
variable_column:
name: variable_column
annotations:
tier:
tag: tier
value: conditionally_core
justification:
tag: justification
value: In a long file many variables share one value column and are told apart
only by a discriminator column. Without naming that column, a consumer cannot
separate this record's variable from every other variable in the file, so
the value binding is ambiguous for all long-format data.
explanation:
tag: explanation
value: In a long table there is a column — often literally called `variable`
— that says what each row measures (`tmax`, `vp`, ...). This slot names
that column. Wide files do not have one, which is why this only applies
to long orientation.
description: 'For long orientation: name of the column that discriminates variables
(e.g. `variable`). Not applicable to wide files.'
title: Variable Column
examples:
- value: variable
from_schema: https://w3id.org/linkml/microschemas/envar
see_also:
- https://doi.org/10.18637/jss.v059.i10
owner: DataLayout
domain_of:
- DataLayout
range: string
variable_key:
name: variable_key
annotations:
tier:
tag: tier
value: conditionally_core
justification:
tag: justification
value: Knowing which column discriminates variables is not enough — a consumer
also needs the label value that selects this record's rows. Without the
key, every row of a long file is a candidate and this record's values cannot
be filtered out of the shared value column.
explanation:
tag: explanation
value: 'This is the label to filter on: keep only the rows where the variable
column says, for example, `tmax`, and you have exactly this record''s measurements.
It usually matches the variable''s short name, but does not have to.'
description: 'For long orientation: the value in `variable_column` that selects
this record''s rows (e.g. `tmax`). Often, but not necessarily, equal to `VariableIdentity.variable_name`.'
title: Variable Key
examples:
- value: pm25_annual
description: selects the PM2.5 rows in a long tract-level file
from_schema: https://w3id.org/linkml/microschemas/envar
see_also:
- https://doi.org/10.18637/jss.v059.i10
owner: DataLayout
domain_of:
- DataLayout
range: string
subject_column:
name: subject_column
annotations:
tier:
tag: tier
value: recommended
justification:
tag: justification
value: The whole point of an exposure sidecar is to be joined back to a health-data
layer. Without knowing which column carries the subject or cohort identifier,
the values cannot be attached to the people or places the record-level `subject`
refers to, and the linkage step becomes guesswork.
explanation:
tag: explanation
value: This names the ID column that says who (or where) each row belongs
to — a patient identifier like `subject_id`, or a place identifier like
a census-tract code.
covered_by:
tag: covered_by
annotations:
omop_gaia:
tag: omop_gaia
annotations:
extent:
tag: extent
value: full
status:
tag: status
value: verified
where:
tag: where
value: external_exposure.person_id; person.person_id; location.location_id
evidence:
tag: evidence
value: EnVar/examples/heat/COMPARISON.md §A (Person identifier — ✅
for OMOP/GAIA)
note:
tag: note
value: OMOP carries the subject identifier first-class on external_exposure.person_id
(and person/location).
degauss:
tag: degauss
annotations:
extent:
tag: extent
value: full
status:
tag: status
value: verified
where:
tag: where
value: '`id` column on every CSV row'
evidence:
tag: evidence
value: EnVar/examples/heat/COMPARISON.md §A (Person identifier — ✅
for DeGAUSS)
note:
tag: note
value: DeGAUSS carries the person identifier in the `id` column on
every row.
amadeus:
tag: amadeus
annotations:
extent:
tag: extent
value: full
status:
tag: status
value: verified
where:
tag: where
value: '`person_loc_id` / `person_id` column on every row'
evidence:
tag: evidence
value: EnVar/examples/heat/COMPARISON.md §A (Person identifier — ✅
for Amadeus)
note:
tag: note
value: Amadeus carries the person identifier on every row.
cher:
tag: cher
annotations:
extent:
tag: extent
value: partial
status:
tag: status
value: asserted
where:
tag: where
value: table_column_dictionary column tagged Index / Foreign Reference
(participant linkage)
note:
tag: note
value: 'C-HER can tag a subject/participant column via Index or Foreign
Reference, but not as a dedicated sidecar subject binding. Conservative:
partial.'
codata:
tag: codata
annotations:
extent:
tag: extent
value: out_of_layer
status:
tag: status
value: asserted
where:
tag: where
value: no column
note:
tag: note
value: The subject/cohort column binding is instance-layer layout,
outside CODATA's conceptual scope.
description: Name of the column carrying the opaque subject / cohort identifier
the record-level `subject` refers to (e.g. `subject_id`).
title: Subject Column
examples:
- value: subject_id
description: patient-level extract
- value: tract_id
description: census-tract-level extract
from_schema: https://w3id.org/linkml/microschemas/envar
owner: DataLayout
domain_of:
- DataLayout
range: string
time_column:
name: time_column
annotations:
tier:
tag: tier
value: recommended
justification:
tag: justification
value: Without knowing which column carries the date or timestamp, each value
cannot be placed in time, so exposure values cannot be aligned with clinical
events — which day's exposure goes with which health record — and any lag
or window analysis is impossible.
explanation:
tag: explanation
value: This names the column that gives each measurement its "when" — a `date`
column for daily data, or a `year` column for annual summaries.
covered_by:
tag: covered_by
annotations:
omop_gaia:
tag: omop_gaia
annotations:
extent:
tag: extent
value: partial
status:
tag: status
value: asserted
where:
tag: where
value: external_exposure attr_start_date / attr_end_date (window-wide,
not per-day)
note:
tag: note
value: 'GAIA carries date columns but collapses every row to the variable''s
window-wide attr_start_date/attr_end_date — per-day external_exposure
does not come out. Conservative: partial.'
degauss:
tag: degauss
annotations:
extent:
tag: extent
value: full
status:
tag: status
value: verified
where:
tag: where
value: one row per (person × day) in cohort_addresses_geocoded_daymet
CSV (per-day date)
evidence:
tag: evidence
value: EnVar/examples/heat/COMPARISON.md §D (One row per person ×
day — ✅ for DeGAUSS)
note:
tag: note
value: DeGAUSS emits a per-day date on every row (24 rows, per-day).
amadeus:
tag: amadeus
annotations:
extent:
tag: extent
value: full
status:
tag: status
value: verified
where:
tag: where
value: one row per (person × day) in gridmet_tmmx.csv (per-day date)
evidence:
tag: evidence
value: EnVar/examples/heat/COMPARISON.md §D (One row per person ×
day — ✅ for Amadeus)
note:
tag: note
value: Amadeus emits a per-day date on every row (24 rows, per-day).
cher:
tag: cher
annotations:
extent:
tag: extent
value: partial
status:
tag: status
value: asserted
where:
tag: where
value: table_column_dictionary column tagged Temporal – Primary
note:
tag: note
value: 'C-HER tags time columns first-class via the Temporal – Primary
tag, but not as a portable sidecar time binding. Conservative: partial.'
codata:
tag: codata
annotations:
extent:
tag: extent
value: out_of_layer
status:
tag: status
value: asserted
where:
tag: where
value: no column
note:
tag: note
value: The time-column binding is instance-layer layout, outside CODATA's
conceptual scope.
description: Name of the column carrying the observation date / timestamp (e.g.
`date`).
title: Time Column
examples:
- value: date
description: daily data
- value: year
description: annual aggregate
from_schema: https://w3id.org/linkml/microschemas/envar
owner: DataLayout
domain_of:
- DataLayout
range: string
native_value_column:
name: native_value_column
annotations:
tier:
tag: tier
value: optional
justification:
tag: justification
value: When a producer ships both unit representations, binding only one leaves
an unexplained column in the data file and hides the cheapest available
cross-check (recomputing one column from the other via `unit_conversion_formula`).
explanation:
tag: explanation
value: Some data files helpfully include the same measurement twice, in two
units — like a column in Kelvin next to one in Celsius. This names that
second column so it is accounted for rather than mysterious.
description: Name of a second column carrying the same values in the source's
*native* units (`VariableIdentity.native_units_ucum`), when the companion file
ships both, e.g. Amadeus's `value_kelvin` next to `value_celsius`. `value_column`
stays the binding for the values in `units_ucum`; this binds the native-unit
twin.
title: Native Value Column
comments:
- 'Added after the reverse gap survey (docs/reverse-gap-survey.md, 2026-07): the
Amadeus pipeline''s native CSV carries `value_kelvin` and `value_celsius` twin
columns, and `DataLayout` could bind exactly one of them — the native-unit copy
had no home and its very existence went unrecorded. Rider on the native-unit
conversion record in envar_variable (`native_units_ucum`, `unit_conversion_formula`).'
examples:
- value: value_kelvin
description: Amadeus gridMET output ships Kelvin and Celsius side by side
from_schema: https://w3id.org/linkml/microschemas/envar
owner: DataLayout
domain_of:
- DataLayout
range: string
value_uncertainty_column:
name: value_uncertainty_column
annotations:
tier:
tag: tier
value: recommended
justification:
tag: justification
value: Products like Daymet ship per-value standard errors that most pipelines
silently drop. Without this binding, downstream analyses cannot find or
propagate the measurement error attached to each value, and dropped uncertainty
is indistinguishable from uncertainty that never existed.
explanation:
tag: explanation
value: Some datasets include a "plus or minus" column next to each value —
an estimate of how far off each number might be. This slot names that column
so the uncertainty travels with the data instead of being lost.
covered_by:
tag: covered_by
annotations:
omop_gaia:
tag: omop_gaia
annotations:
extent:
tag: extent
value: absent
status:
tag: status
value: asserted
where:
tag: where
value: no column
note:
tag: note
value: No per-value uncertainty column on external_exposure; the OMOP
stack carries no measurement-error column for the value.
degauss:
tag: degauss
annotations:
extent:
tag: extent
value: absent
status:
tag: status
value: asserted
where:
tag: where
value: no column
note:
tag: note
value: DeGAUSS's daymet CSV carries the value only, no per-value standard-error
column.
amadeus:
tag: amadeus
annotations:
extent:
tag: extent
value: absent
status:
tag: status
value: asserted
where:
tag: where
value: no column
note:
tag: note
value: Amadeus's gridmet_tmmx.csv carries no per-value uncertainty
column.
cher:
tag: cher
annotations:
extent:
tag: extent
value: partial
status:
tag: status
value: asserted
where:
tag: where
value: table_column_dictionary column tagged Meta value (info about
a Value column)
note:
tag: note
value: 'C-HER''s Meta value tag could carry a per-value uncertainty
column referencing its Value column, but none is asserted here.
Conservative: partial.'
codata:
tag: codata
annotations:
extent:
tag: extent
value: out_of_layer
status:
tag: status
value: asserted
where:
tag: where
value: no column
note:
tag: note
value: A per-value uncertainty column binding is instance-layer layout,
outside CODATA's conceptual scope.
description: Name of the column carrying per-value uncertainty (e.g. `pm_se`,
`tmax_stderr`). Its semantics (uncertainty type, units) live in the Uncertainty
microschema. Null with reason for products whose per-value uncertainty exists
upstream but is not surfaced.
title: Per-Value Uncertainty Column
examples:
- value: tmax_stderr
from_schema: https://w3id.org/linkml/microschemas/envar
owner: DataLayout
domain_of:
- DataLayout
range: string
value_uncertainty_column_missing_reason:
name: value_uncertainty_column_missing_reason
annotations:
tier:
tag: tier
value: optional
justification:
tag: justification
value: 'A bare null cannot be audited: it could mean the source has no per-value
uncertainty, the pipeline dropped it, or the producer forgot to record it.
Stating the reason makes the absence deliberate and lets reviewers tell
"unavailable" from "lost", which matters for reproducing the extraction.'
explanation:
tag: explanation
value: When there is no uncertainty column, this slot says why — for example
"the upstream product does not provide one" versus "it exists upstream but
was not extracted".
description: Reason `value_uncertainty_column` is null.
title: Reason Uncertainty Column Is Missing
examples:
- value: not_provided_by_source
from_schema: https://w3id.org/linkml/microschemas/envar
owner: DataLayout
domain_of:
- DataLayout
range: MissingReasonEnum
quality_flag_column:
name: quality_flag_column
annotations:
tier:
tag: tier
value: optional
justification:
tag: justification
value: Without knowing where the per-value QA flags live, consumers cannot
filter or down-weight values the producer already marked as suspect, so
known-bad measurements flow silently into analyses.
explanation:
tag: explanation
value: A quality flag column is like a traffic light next to each number —
good, suspect, or bad — recorded by the data producer. This slot names that
column; what the flag codes mean is described in the Uncertainty microschema.
description: Name of any per-value QA flag column. CF `ancillary_variables` analogue;
the flag vocabulary lives in the Uncertainty microschema.
title: Quality Flag Column
examples:
- value: tmax_qc
description: per-value QA flag column accompanying a `tmax` value column
from_schema: https://w3id.org/linkml/microschemas/envar
see_also:
- https://cfconventions.org/
owner: DataLayout
domain_of:
- DataLayout
range: string
quality_flag_column_missing_reason:
name: quality_flag_column_missing_reason
annotations:
tier:
tag: tier
value: optional
justification:
tag: justification
value: 'Without a stated reason, a missing QA-flag column is ambiguous: the
source may publish no flags, or the pipeline may have dropped them. The
reason turns a silent gap into a documented decision that a reviewer can
check against the upstream product.'
explanation:
tag: explanation
value: When there is no quality-flag column, this slot says why it is absent
— for example because the source dataset simply does not publish quality
flags.
description: Reason `quality_flag_column` is null.
title: Reason Quality Flag Column Is Missing
examples:
- value: not_provided_by_source
from_schema: https://w3id.org/linkml/microschemas/envar
owner: DataLayout
domain_of:
- DataLayout
range: MissingReasonEnum
null_semantics_column:
name: null_semantics_column
annotations:
tier:
tag: tier
value: recommended
justification:
tag: justification
value: A blank or a `0` is ambiguous, and the three kinds of "missing/zero"
have OPPOSITE correct handling — structural_null and derived_null must be
dropped (the latter also excluded from denominators) while true_zero must
be kept as a real zero; conflating them biases the health-effect estimate.
Nulls should be excluded from percentile denominators for exactly this reason,
so binding a per-value null-kind column is what lets a consumer handle each
cell correctly instead of reading missingness as low or zero exposure.
explanation:
tag: explanation
value: 'A blank cell or a `0` can mean three very different things: there
was never any data here, we could not compute a value here, or the value
is genuinely zero. Each needs different treatment. This names the column
that tags every value with which kind it is, so nobody mistakes "no data"
for "zero exposure".'
covered_by:
tag: covered_by
annotations:
omop_gaia:
tag: omop_gaia
annotations:
extent:
tag: extent
value: partial
status:
tag: status
value: asserted
where:
tag: where
value: gaia_catalog meta_etl_*.json nodata = ["float4", "-9999"] (dataset-level
sentinel, not per-value null-kind)
note:
tag: note
value: 'GAIA declares a dataset-level nodata sentinel for daymet but
no per-value column tagging present / structural_null / derived_null
/ true_zero. Conservative: partial.'
degauss:
tag: degauss
annotations:
extent:
tag: extent
value: absent
status:
tag: status
value: asserted
where:
tag: where
value: no column
note:
tag: note
value: DeGAUSS emits no null-kind indicator column and no missing-value
sentinel in its CSV output.
amadeus:
tag: amadeus
annotations:
extent:
tag: extent
value: partial
status:
tag: status
value: asserted
where:
tag: where
value: thredds_dataset.xml _FillValue / missing_value = 32767 (dataset-level
sentinel, not per-value null-kind)
note:
tag: note
value: 'Amadeus declares a fill/missing-value sentinel that marks
structural nulls, but no per-value column distinguishing the three
null kinds. Conservative: partial.'
cher:
tag: cher
annotations:
extent:
tag: extent
value: absent
status:
tag: status
value: asserted
where:
tag: where
value: no column
note:
tag: note
value: C-HER handles null semantics in its validation rules, but binds
no per-value null-kind column in table_column_dictionary.
codata:
tag: codata
annotations:
extent:
tag: extent
value: out_of_layer
status:
tag: status
value: asserted
where:
tag: where
value: no column
note:
tag: note
value: A per-value null-semantics column binding is instance-layer
layout, outside CODATA's conceptual scope.
description: 'Name of a per-value column carrying a null-semantics indicator drawn
from `NullSemanticsEnum` (`present` / `structural_null` / `derived_null` / `true_zero`;
see envar_common). Bound here in `DataLayout` — mirroring the existing per-value
`value_uncertainty_column` / `quality_flag_column` pattern — because the distinction
is per-value: whether a given blank or `0` means "no source data", "aggregation
impossible", or "measured zero" varies row by row and so must be carried alongside
the values, not declared once for the dataset. Null with reason when the producer
does not distinguish null kinds. The vocabulary itself lives in the Uncertainty
/ shared layer (`NullSemanticsEnum`).'
title: Null-Semantics Column
examples:
- value: null_kind
description: per-value column tagging each cell present / structural_null /
derived_null / true_zero, next to the value column
from_schema: https://w3id.org/linkml/microschemas/envar
owner: DataLayout
domain_of:
- DataLayout
range: string
null_semantics_column_missing_reason:
name: null_semantics_column_missing_reason
annotations:
tier:
tag: tier
value: optional
justification:
tag: justification
value: 'A bare null is ambiguous: the producer may not distinguish null kinds
at all, or the pipeline may have dropped the indicator. Stating the reason
makes the absence deliberate and auditable rather than a silent gap.'
explanation:
tag: explanation
value: When there is no null-kind column, this slot says why — for example
because the source never separates "no data" from "real zero" in the first
place.
description: Reason `null_semantics_column` is null.
title: Reason Null-Semantics Column Is Missing
examples:
- value: not_provided_by_source
description: the producer does not distinguish kinds of missing / zero
from_schema: https://w3id.org/linkml/microschemas/envar
owner: DataLayout
domain_of:
- DataLayout
range: MissingReasonEnum
rules:
- preconditions:
slot_conditions:
table_orientation:
name: table_orientation
equals_string: long
postconditions:
slot_conditions:
variable_column:
name: variable_column
required: true
variable_key:
name: variable_key
required: true
description: 'Long-format layouts must name the variable discriminator: the shared
value column is only interpretable once `variable_column` says which column discriminates
variables and `variable_key` says which row value selects this record''s rows
(SPEC.md rule 7; tier conditionally_core context "long orientation").'
Identifier and Mapping Information
Annotations
| property | value |
|---|---|
| domain_of_use | environmental_exposure |
Schema Source
- from schema: https://w3id.org/linkml/microschemas/envar
Mappings
| Mapping Type | Mapped Value |
|---|---|
| self | envar:DataLayout |
| native | envar:DataLayout |