Uncertainty and Quality

name: Uncertainty
annotations:
  domain_of_use:
    tag: domain_of_use
    value: environmental_exposure
description: 'Uncertainty and quality character of a value series: per-value uncertainty
  type / units, model-aggregate uncertainty summary, quality flag vocabulary, missing-data
  handling, and data completeness. The uncertainty / QA-flag column bindings live
  in DataLayout (see envar_layout). One per record; slots may be null with reasons.'
title: Uncertainty and Quality
from_schema: https://w3id.org/linkml/microschemas/envar
see_also:
- https://en.wikipedia.org/wiki/Uncertainty_quantification
rank: 1000
attributes:
  per_value_uncertainty_type:
    name: per_value_uncertainty_type
    annotations:
      tier:
        tag: tier
        value: recommended
      justification:
        tag: justification
        value: 'A "±" number means nothing until you know what kind of number it is:
          a standard error, a 95 % prediction interval, and an ensemble spread are
          not interchangeable and cannot be pooled or propagated the same way. Without
          the type, downstream code either mishandles the uncertainty or drops it,
          so exposure measurement error goes unaccounted for and health-effect estimates
          are biased, usually toward the null.'
      explanation:
        tag: explanation
        value: Every estimated value comes with a "how sure are we" number, but there
          are several different kinds. This says which kind you are looking at — for
          example a standard error (a ± number saying how far off the estimate could
          plausibly be) versus a prediction interval (a range the true value should
          fall inside, say 95 times out of 100).
      covered_by:
        tag: covered_by
        annotations:
          omop_gaia:
            tag: omop_gaia
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: No gaia output carries a per-value uncertainty type.
          degauss:
            tag: degauss
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: DeGAUSS emits no per-value uncertainty.
          amadeus:
            tag: amadeus
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: Amadeus emits no per-value uncertainty type.
          cher:
            tag: cher
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: C-HER's column_tag Meta value can hold uncertainty columns
                  in principle, but no per-value uncertainty type is a modelled field.
          codata:
            tag: codata
            annotations:
              extent:
                tag: extent
                value: out_of_layer
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: Per-value uncertainty is instance-layer; out of scope for CODATA.
    description: Kind of per-value uncertainty captured in the column named by `DataLayout.value_uncertainty_column`
      (see envar_layout).
    title: Per-Value Uncertainty Type
    examples:
    - value: standard_error
      description: Daymet daily Tmax reports a per-value standard error.
    - value: prediction_interval
      description: ACAG satellite PM2.5 reports a per-value prediction interval.
    from_schema: https://w3id.org/linkml/microschemas/envar
    see_also:
    - https://en.wikipedia.org/wiki/Standard_error
    - https://en.wikipedia.org/wiki/Prediction_interval
    owner: Uncertainty
    domain_of:
    - Uncertainty
    range: UncertaintyTypeEnum
  per_value_uncertainty_type_missing_reason:
    name: per_value_uncertainty_type_missing_reason
    annotations:
      tier:
        tag: tier
        value: optional
      justification:
        tag: justification
        value: Records the difference between "we know the source has no per-value
          uncertainty" and "someone forgot to fill this in". Without the reason, a
          blank uncertainty type is ambiguous and a validator cannot tell an honest
          gap from an oversight, so quality-completeness reporting is unreliable.
      explanation:
        tag: explanation
        value: If the uncertainty-type field is left empty, this simply says why —
          for instance the original dataset never published one. It turns a confusing
          blank into an explicit, honest "not available, and here is the reason".
    description: Reason `per_value_uncertainty_type` is null.
    title: Reason Uncertainty Type Is Missing
    examples:
    - value: not_provided_by_source
      description: The source product publishes no per-value uncertainty.
    from_schema: https://w3id.org/linkml/microschemas/envar
    owner: Uncertainty
    domain_of:
    - Uncertainty
    range: MissingReasonEnum
  per_value_uncertainty_units_ucum:
    name: per_value_uncertainty_units_ucum
    annotations:
      tier:
        tag: tier
        value: recommended
      justification:
        tag: justification
        value: 'An uncertainty number is meaningless without its units: a per-value
          error of "2" is 2 °C or 2 K depending on this field, and a mismatch between
          value units and uncertainty units silently corrupts any error propagation
          into the health analysis.'
      explanation:
        tag: explanation
        value: This says what the uncertainty is measured in — degrees Celsius, micrograms
          per cubic metre, and so on — written in a standard code (UCUM) that computers
          read the same way every time. Usually it matches the units of the value
          itself.
      covered_by:
        tag: covered_by
        annotations:
          omop_gaia:
            tag: omop_gaia
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: With no per-value uncertainty column, gaia carries no uncertainty
                  units.
          degauss:
            tag: degauss
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: No per-value uncertainty column exists, so no uncertainty units.
          amadeus:
            tag: amadeus
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: No per-value uncertainty column exists, so no uncertainty units.
          cher:
            tag: cher
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: No per-value uncertainty field, so no uncertainty units in
                  C-HER.
          codata:
            tag: codata
            annotations:
              extent:
                tag: extent
                value: out_of_layer
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: Per-value uncertainty units are instance-layer; out of scope
                  for CODATA.
    description: Units of the per-value uncertainty in UCUM syntax. Usually the same
      as the value units.
    title: Uncertainty Units (UCUM)
    examples:
    - value: Cel
      description: Standard error of a daily Tmax value, in degrees Celsius.
    from_schema: https://w3id.org/linkml/microschemas/envar
    see_also:
    - https://ucum.org/
    owner: Uncertainty
    domain_of:
    - Uncertainty
    range: string
  per_value_uncertainty_units_ucum_missing_reason:
    name: per_value_uncertainty_units_ucum_missing_reason
    annotations:
      tier:
        tag: tier
        value: optional
      justification:
        tag: justification
        value: Distinguishes "there is no uncertainty column, so units genuinely do
          not apply" from an accidental omission. Without the reason a blank units
          field is ambiguous and completeness checks cannot tell a legitimate not-applicable
          from a missing entry.
      explanation:
        tag: explanation
        value: If the uncertainty-units field is blank, this explains why — often
          because there is no uncertainty column at all, so there is nothing to put
          units on. It stops a reader from wondering whether the units were simply
          forgotten.
    description: Reason `per_value_uncertainty_units_ucum` is null.
    title: Reason Uncertainty Units Are Missing
    examples:
    - value: not_applicable
      description: No per-value uncertainty column exists, so units do not apply.
    from_schema: https://w3id.org/linkml/microschemas/envar
    owner: Uncertainty
    domain_of:
    - Uncertainty
    range: MissingReasonEnum
  model_aggregate_uncertainty:
    name: model_aggregate_uncertainty
    annotations:
      tier:
        tag: tier
        value: recommended
      justification:
        tag: justification
        value: Per-value uncertainty is often absent, so the whole-model cross-validation
          summary (R², RMSE) is frequently the only quantitative handle on how accurate
          the product is. Without it an analyst cannot judge whether the exposure
          estimates are precise enough for the health question, and cannot compare
          the reliability of two products.
      explanation:
        tag: explanation
        value: A report card for how well the model predicts reality overall, checked
          by holding some data back and seeing how close its guesses came — for example
          an R² near 1 means the model tracks the true values closely. It also records
          where those scores were published.
      covered_by:
        tag: covered_by
        annotations:
          omop_gaia:
            tag: omop_gaia
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: No gaia output carries cross-validation R²/RMSE for the source
                  model (Tmax scenario emits no model-accuracy summary).
          degauss:
            tag: degauss
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: DeGAUSS emits no whole-model cross-validation metrics.
          amadeus:
            tag: amadeus
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: Amadeus emits no whole-model cross-validation metrics.
          cher:
            tag: cher
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: C-HER carries no model cross-validation summary field.
          codata:
            tag: codata
            annotations:
              extent:
                tag: extent
                value: out_of_layer
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: Model-aggregate uncertainty is instance-layer; out of scope
                  for CODATA.
    description: Summary statistics for the model as a whole — cross-validation metrics
      and where they are reported.
    title: Model Aggregate Uncertainty
    examples:
    - description: Cross-validated R² for ACAG satellite PM2.5 and its reporting DOI.
      object:
        cv_r2: 0.9
        reported_in: 10.1021/acs.est.1c05309
    from_schema: https://w3id.org/linkml/microschemas/envar
    see_also:
    - https://en.wikipedia.org/wiki/Cross-validation_(statistics)
    owner: Uncertainty
    domain_of:
    - Uncertainty
    range: ModelAggregateUncertainty
    inlined: true
  model_aggregate_uncertainty_missing_reason:
    name: model_aggregate_uncertainty_missing_reason
    annotations:
      tier:
        tag: tier
        value: optional
      justification:
        tag: justification
        value: Separates "the producer published no cross-validation metrics" from
          a forgotten entry. Without the reason, a missing model-accuracy summary
          looks like a data-entry lapse, and reviewers cannot tell whether the information
          was ever available.
      explanation:
        tag: explanation
        value: When the model-accuracy summary is missing, this says why — usually
          because the people who made the dataset never reported those scores. It
          makes the absence deliberate and explainable rather than a mystery blank.
    description: Reason `model_aggregate_uncertainty` is null.
    title: Reason Aggregate Uncertainty Is Missing
    examples:
    - value: not_provided_by_source
      description: The producer reports no whole-model cross-validation metrics.
    from_schema: https://w3id.org/linkml/microschemas/envar
    owner: Uncertainty
    domain_of:
    - Uncertainty
    range: MissingReasonEnum
  quality_flag_vocabulary:
    name: quality_flag_vocabulary
    annotations:
      tier:
        tag: tier
        value: optional
      justification:
        tag: justification
        value: QA flags (e.g. "estimated", "below detection limit", "instrument malfunction")
          are only interpretable against the code list that defines them; without
          naming the vocabulary, a flagged value cannot be correctly filtered or trusted,
          so suspect measurements may enter the analysis unnoticed.
      explanation:
        tag: explanation
        value: Some datasets tag individual values with short quality codes — think
          of the little footnote letters next to numbers in a table. This points to
          the key that explains what each code means, so you know which readings to
          trust or set aside.
    description: Reference to the QA flag vocabulary (e.g. an EPA AQS qualifier-code
      list) used by the column named in `DataLayout.quality_flag_column` (see envar_layout).
    title: Quality Flag Vocabulary
    examples:
    - value: EPA AQS qualifier codes
      description: Flag vocabulary for monitor-derived series; gridded products often
        have none.
    from_schema: https://w3id.org/linkml/microschemas/envar
    owner: Uncertainty
    domain_of:
    - Uncertainty
    range: string
  quality_flag_vocabulary_missing_reason:
    name: quality_flag_vocabulary_missing_reason
    annotations:
      tier:
        tag: tier
        value: optional
      justification:
        tag: justification
        value: Distinguishes "this product has no QA flags at all" (common for gridded
          data) from an omission. Without the reason, an empty vocabulary field is
          ambiguous and a completeness check cannot tell an inapplicable entry from
          a missing one.
      explanation:
        tag: explanation
        value: If there is no quality-code key listed, this says why — often because
          the dataset simply does not use quality codes. It turns a blank into a clear
          statement rather than leaving the reader guessing.
    description: Reason `quality_flag_vocabulary` is null.
    title: Reason Quality Flag Vocabulary Is Missing
    examples:
    - value: not_provided_by_source
      description: Daymet publishes no QA-flag column, so no vocabulary exists.
    from_schema: https://w3id.org/linkml/microschemas/envar
    owner: Uncertainty
    domain_of:
    - Uncertainty
    range: MissingReasonEnum
  missing_data_handling_method:
    name: missing_data_handling_method
    annotations:
      tier:
        tag: tier
        value: recommended
      justification:
        tag: justification
        value: 'How gaps were filled is often invisible downstream: an interpolated
          value looks identical to a measured one, so without this field an analyst
          overstates coverage and treats imputed exposures as if they were observed,
          biasing associations in unknown directions. It is the difference between
          apparent and real completeness.'
      explanation:
        tag: explanation
        value: 'Real data has holes — a cloud blocks a satellite, snow covers a sensor.
          This says what the dataset did about those holes: leave them empty, guess
          from nearby days and places, copy the last known value, and so on. Filled-in
          numbers can look just like real measurements, so it matters to know which
          is which.'
      covered_by:
        tag: covered_by
        annotations:
          omop_gaia:
            tag: omop_gaia
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: Gaia's meta_etl nodata registers the missing-value sentinel,
                  not the source's missing-data handling method.
          degauss:
            tag: degauss
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: DeGAUSS records no missing-data-handling method.
          amadeus:
            tag: amadeus
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: Amadeus's THREDDS metadata declares a _FillValue sentinel but
                  not the source's gap-filling method.
          cher:
            tag: cher
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: C-HER's null taxonomy classifies null cells but carries no
                  source missing-data-handling method field.
          codata:
            tag: codata
            annotations:
              extent:
                tag: extent
                value: out_of_layer
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: Missing-data handling method is instance-layer; out of scope
                  for CODATA.
    description: How the source handles missing values (e.g. how Daymet handles snow-covered
      pixels).
    title: Missing Data Handling Method
    examples:
    - value: spatiotemporal_interpolation
      description: Daymet fills missing cells by spatiotemporal interpolation.
    from_schema: https://w3id.org/linkml/microschemas/envar
    see_also:
    - https://daymet.ornl.gov/
    owner: Uncertainty
    domain_of:
    - Uncertainty
    range: MissingDataHandlingEnum
  missing_data_handling_method_missing_reason:
    name: missing_data_handling_method_missing_reason
    annotations:
      tier:
        tag: tier
        value: optional
      justification:
        tag: justification
        value: Separates "the producer never documented how gaps were handled" from
          an entry someone forgot. Without the reason, a blank handling method is
          ambiguous and an analyst cannot judge whether the gap-filling behaviour
          is unknown or simply unrecorded here.
      explanation:
        tag: explanation
        value: When the gap-handling method is left empty, this explains why — often
          because the original dataset never said how it dealt with missing values.
          It makes the unknown explicit instead of a silent blank.
    description: Reason `missing_data_handling_method` is null.
    title: Reason Handling Method Is Missing
    examples:
    - value: not_provided_by_source
      description: The producer does not document its missing-data handling.
    from_schema: https://w3id.org/linkml/microschemas/envar
    owner: Uncertainty
    domain_of:
    - Uncertainty
    range: MissingReasonEnum
  missing_value_sentinel:
    name: missing_value_sentinel
    annotations:
      tier:
        tag: tier
        value: optional
      justification:
        tag: justification
        value: A sentinel read as data is a catastrophic outlier (−9999 °C) or a silent
          bias (32767 packed); anyone touching the source files rather than the cleaned
          companion file needs to know which value to mask.
      explanation:
        tag: explanation
        value: Datasets mark holes with an agreed impossible number, like −9999. If
          software doesn't know the marker, it will average it in as if it were a
          real reading. This field records the marker.
    description: 'The stored value that means "missing" in the source product''s files,
      verbatim, e.g. `32767` (gridMET `_FillValue`) or `-9999` (Daymet nodata). Complements
      `missing_data_handling_method`: the policy slot says *how* gaps were handled,
      this says *which number encodes a gap*.'
    title: Missing Value Sentinel
    comments:
    - 'Added after the reverse gap survey (docs/reverse-gap-survey.md, 2026-07): two
      pipelines surfaced the same field independently — the Amadeus THREDDS metadata
      declares `_FillValue`/`missing_value` 32767, and GAIA''s ETL descriptor registers
      `nodata: [float4, -9999]`. Neither number had a home: `missing_data_handling_method`
      records policy, not the sentinel, and a consumer re-reading the source grid
      needs the number itself. Kept as a string so non-numeric sentinels (`NaN`, empty
      string) stay representable.'
    examples:
    - value: '32767'
      description: gridMET tmmx `_FillValue` / `missing_value` (packed int16)
    - value: '-9999'
      description: Daymet nodata sentinel as registered in the GAIA catalog
    from_schema: https://w3id.org/linkml/microschemas/envar
    see_also:
    - https://cfconventions.org/cf-conventions/cf-conventions.html#missing-data
    owner: Uncertainty
    domain_of:
    - Uncertainty
    range: string
  data_completeness_pct:
    name: data_completeness_pct
    annotations:
      tier:
        tag: tier
        value: recommended
      justification:
        tag: justification
        value: A quantitative handle on how much of the requested exposure window
          is actually populated. A series that is 60 % complete supports very different
          inferences from one that is 100 % complete; without this number, sparse
          coverage is hidden and averages or exposure windows are computed over gaps
          as if they were full, biasing the health analysis.
      explanation:
        tag: explanation
        value: Out of all the days and places you asked about, this is the percentage
          that actually have a value — 100 means nothing is missing, 60 means four
          in ten slots are blank. It is a quick honesty check on how full the dataset
          really is.
      covered_by:
        tag: covered_by
        annotations:
          omop_gaia:
            tag: omop_gaia
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: No gaia output carries a data-completeness percentage for the
                  extracted window.
          degauss:
            tag: degauss
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: DeGAUSS emits no completeness percentage.
          amadeus:
            tag: amadeus
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: Amadeus emits no completeness percentage.
          cher:
            tag: cher
            annotations:
              extent:
                tag: extent
                value: absent
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: C-HER carries no per-window data-completeness percentage field.
          codata:
            tag: codata
            annotations:
              extent:
                tag: extent
                value: out_of_layer
              status:
                tag: status
                value: asserted
              where:
                tag: where
                value: no column
              note:
                tag: note
                value: Data completeness percentage is instance-layer; out of scope
                  for CODATA.
    description: Percent of (location, date) cells in the extracted window that have
      a non-missing value. 0-100.
    title: Data Completeness Percentage
    examples:
    - value: '100'
      description: Every (location, date) cell in the extracted window has a value.
    from_schema: https://w3id.org/linkml/microschemas/envar
    owner: Uncertainty
    domain_of:
    - Uncertainty
    range: float
  data_completeness_pct_missing_reason:
    name: data_completeness_pct_missing_reason
    annotations:
      tier:
        tag: tier
        value: optional
      justification:
        tag: justification
        value: Distinguishes "completeness is genuinely uncomputable" from "it could
          be derived but the pipeline does not yet do so" — a distinction that tells
          a data steward whether the gap is a limitation of the source or a fixable
          pipeline shortfall.
      explanation:
        tag: explanation
        value: If the completeness percentage is missing, this says why — for example
          the number could be worked out from the data but the software has not been
          set up to calculate it yet. It flags the blank as a known to-do rather than
          an impossibility.
    description: Reason `data_completeness_pct` is null.
    title: Reason Data Completeness Is Missing
    examples:
    - value: available_but_not_extracted
      description: Completeness could be computed from the output but the pipeline
        does not yet do so.
    from_schema: https://w3id.org/linkml/microschemas/envar
    owner: Uncertainty
    domain_of:
    - Uncertainty
    range: MissingReasonEnum