Changing expected column names¶

Neptoon separates source-column mapping, canonical column names, and runtime relabelling. These mechanisms solve different problems:

Requirement	Use
A logger file uses names such as `N2Cts`, `P4_mb`, or `RH1`	Map the source columns in the sensor YAML.
An older configuration uses a supported historical or pluralised name	Let the column-alias resolver map it to the canonical name.
A Python application must make neptoon use a different internal/output label	Relabel the corresponding `ColumnInfo.Name` member.
A new column must become part of neptoon itself	Add it to the `ColumnInfo` registry and update the related code, tests, and documentation.

For most users, YAML mapping is the correct solution. Runtime relabelling is an advanced, process-wide customization and should be used deliberately.

Canonical names and raw source names are different¶

Raw files do not need to use neptoon's canonical names. Map the logger columns in time_series_data.key_column_info:

config: sensor

time_series_data:
  key_column_info:
    epithermal_neutron_columns:
      - N2Cts
    pressure_columns:
      - P4_mb
    temperature_columns:
      - T1_C
    relative_humidity_columns:
      - RH1
    date_time_columns:
      - Date Time(UTC)
    neutron_count_units: counts_per_hour
    pressure_units: hPa
    temperature_units: C
    relative_humidity_units: percent

The parser reads the source names and creates the canonical processing columns, for example:

N2Cts          -> epithermal_neutrons_cph
P4_mb          -> air_pressure
T1_C           -> air_temperature
RH1            -> air_relative_humidity
Date Time(UTC) -> DatetimeIndex / date_time on export

Use this approach when the only difference is the input-file schema. It keeps the processing and output schema consistent with the rest of neptoon.

See Sensor configuration and Columns in neptoon for the full input mapping model.

The `ColumnInfo` registry¶

ColumnInfo is the central registry for column names used by neptoon internals. Its nested Name enum identifies the meaning of a column, while the string returned by str(...) is the current DataFrame label:

from neptoon.columns import ColumnInfo

column_id = ColumnInfo.Name.SOIL_MOISTURE_VOL
column_label = str(column_id)

assert column_label == "smv"

Use the enum member in reusable Python code instead of repeating a literal:

soil_moisture = data_hub.crns_data_frame[
    str(ColumnInfo.Name.SOIL_MOISTURE_VOL)
]

This remains valid when the label is changed at runtime.

Look up a name from an enum-member string¶

ColumnInfo.get_col_name() accepts the enum member name, case-insensitively:

from neptoon.columns import ColumnInfo

label = ColumnInfo.get_col_name("soil_moisture_vol")
assert label == "smv"

It does not accept an arbitrary DataFrame label. Passing an unknown enum member raises AttributeError.

Inspect the active registry¶

The current labels can be inspected programmatically:

from neptoon.columns import ColumnInfo

active_columns = {
    member.name: str(member)
    for member in ColumnInfo.Name
}

This is useful when an application supports optional relabelling or needs to build an export schema from the labels active in the current process.

Runtime relabelling¶

Use ColumnInfo.relabel() when a Python workflow must make neptoon use a different label for a known semantic column:

from neptoon.columns import ColumnInfo

ColumnInfo.relabel(
    ColumnInfo.Name.SOIL_MOISTURE_VOL,
    "theta_v",
)

assert str(ColumnInfo.Name.SOIL_MOISTURE_VOL) == "theta_v"

Relabel before the processing step that creates or consumes the column. For a complete custom Python workflow, apply relabelling before constructing the CRNSDataHub or DataHubFromConfig object.

Relabel several columns¶

from neptoon.columns import ColumnInfo

custom_labels = {
    ColumnInfo.Name.EPI_NEUTRON_COUNT_CPH: "neutrons_cph",
    ColumnInfo.Name.CORRECTED_EPI_NEUTRON_COUNT: "neutrons_corrected",
    ColumnInfo.Name.SOIL_MOISTURE_VOL: "theta_v",
}

for column, label in custom_labels.items():
    ColumnInfo.relabel(column, label)

Always restore the defaults¶

Relabelling changes class-level state for the entire Python process. Use try/finally so the defaults are restored even if processing fails:

from neptoon.columns import ColumnInfo
from neptoon.workflow import ProcessWithConfig

try:
    ColumnInfo.relabel(
        ColumnInfo.Name.SOIL_MOISTURE_VOL,
        "theta_v",
    )

    processor = ProcessWithConfig(
        path_to_sensor_config="sensor.yaml",
        path_to_process_config="process.yaml",
    )
    processor.run_full_process()
    result = processor.data_hub.crns_data_frame.copy()
finally:
    ColumnInfo.reset_labels()

reset_labels() restores all entries to their defaults; it does not restore only the most recently changed label.

Do not reset labels during a running pipeline

Neptoon uses the same mechanism internally to make _FINAL enum members point to smoothed or Kalman-filtered columns. Reset labels only after the complete workflow has finished.

Existing DataFrames are not renamed automatically¶

ColumnInfo.relabel() changes future name lookups. It does not rename columns already present in a DataFrame.

When adopting a custom label for an existing table, rename the table and the registry together:

from neptoon.columns import ColumnInfo

old_label = str(ColumnInfo.Name.SOIL_MOISTURE_VOL)
new_label = "theta_v"

frame = frame.rename(columns={old_label: new_label})
ColumnInfo.relabel(ColumnInfo.Name.SOIL_MOISTURE_VOL, new_label)

The same rule applies to a separately managed flag table. Data and flags must remain aligned where a column participates in quality assessment.

Relabelling and configuration-driven output¶

Configuration values that name processed columns, such as data_storage.output_columns, are matched against the actual processed table. When runtime relabelling is active, configure the active label:

ColumnInfo.relabel(
    ColumnInfo.Name.SOIL_MOISTURE_VOL,
    "theta_v",
)

data_storage:
  output_mode: both
  output_columns:
    - date_time
    - theta_v
    - crns_measurement_depth_cm

Static-pivot exclusions and user-defined plot columns are also string-based. Use the labels that exist in the DataFrame at the point where those features run.

See Saving data for selected and compact output modes.

Aliases are compatibility names, not relabelling¶

resolve_column_alias() accepts a small set of historical or pluralised labels when the corresponding canonical column is available.

Current aliases are:

Accepted alias	Canonical label
`thermal_neutrons_raw`	`thermal_neutron_count_raw`
`thermal_neutrons_cph`	`thermal_neutron_count_cph`
`thermal_neutron_raw`	`thermal_neutron_count_raw`
`thermal_neutron_cph`	`thermal_neutron_count_cph`
`epithermal_neutron_count_raw`	`epithermal_neutrons_raw`
`epithermal_neutron_count_cph`	`epithermal_neutrons_cph`

Alias resolution has three important properties:

If the requested string already exists in the available columns, it is returned unchanged.
An alias is resolved only when its canonical target exists.
Unsupported names return None; neptoon does not guess arbitrary labels.

Aliases preserve compatibility with older examples and configurations. They do not globally rename processing columns and should not be used as the basis for new APIs. Avoid combining legacy aliases with runtime relabelling, because the alias table describes the default canonical schema.

Process-wide state and concurrency¶

ColumnInfo._current_representation is shared class state:

the change affects every hub and workflow in the same Python interpreter;
it is not thread-local;
it is not stored in YAML or automatically persisted between processes;
relabel() does not validate uniqueness or prevent empty labels;
two enum members may intentionally share a label;
concurrent workflows requiring different schemas should run in separate processes.

For a long-running service, prefer canonical names at the neptoon boundary and rename a copy of the final result for each client. This avoids cross-request state leakage.

Intentional shared labels and `_FINAL` selectors¶

Several enum members share the same default label. This is intentional:

Base member	Final selector	Default label
`EPI_NEUTRON_COUNT_CPH`	`EPI_NEUTRON_COUNT_FINAL`	`epithermal_neutrons_cph`
`CORRECTED_EPI_NEUTRON_COUNT`	`CORRECTED_EPI_NEUTRON_COUNT_FINAL`	`corrected_epithermal_neutrons`
`CORRECTED_THERM_NEUTRON_COUNT`	`CORRECTED_THERM_NEUTRON_COUNT_FINAL`	`corrected_thermal_neutrons`
`SOIL_MOISTURE_VOL`	`SOIL_MOISTURE_VOL_FINAL`	`smv`

The base member identifies the ordinary product. The _FINAL member is a selector used by downstream processing. Smoothing and Kalman filtering can relabel a final selector to the generated filtered column while leaving the base product available for inspection.

Relabel the base and final member separately only when the workflow design requires different columns.

Soil-moisture spelling and provenance¶

Canonical output uses smv for volumetric and smg for gravimetric soil moisture. Depths are expressed only in centimetres: smv10cm, smg20cm, or a qualified sensor column such as smv10cm_T1. Input-facing aliases are case-insensitive and may include an underscore before the depth or redundant decimal zeros, so smv_10cm, SMV10cm, and smv10.0cm all resolve to smv10cm. Metre, millimetre, and unitless depth suffixes remain invalid.

Bare smv is the CRNS-derived main product. Fixed-depth in-situ measurements use a depth suffix, while smv_crns_weighted_mean denotes an external-sensor aggregate weighted to the CRNS footprint. This convention distinguishes physical provenance without embedding a changing effective CRNS depth in the primary product name.

Complete canonical column registry¶

The following table documents every ColumnInfo.Name member in the current 0.57 API. The labels shown are the defaults restored by ColumnInfo.reset_labels().

Time and neutron observations¶

Enum member	Default label	Purpose
`DATE_TIME`	`date_time`	Timestamp name used when a time index is materialised for export.
`TIME_STEP_SECONDS`	`time_step_seconds`	Observed or derived interval between records in seconds.
`EPI_NEUTRON_COUNT_RAW`	`epithermal_neutrons_raw`	Raw/working epithermal neutron observation before standard cph output.
`EPI_NEUTRON_COUNT_CPH`	`epithermal_neutrons_cph`	Epithermal neutron rate normalised to counts per hour.
`EPI_NEUTRON_COUNT_FINAL`	`epithermal_neutrons_cph`	Downstream selector for the final raw-neutron signal.
`THERM_NEUTRON_COUNT_RAW`	`thermal_neutron_count_raw`	Raw/working thermal neutron observation.
`THERM_NEUTRON_COUNT_CPH`	`thermal_neutron_count_cph`	Thermal neutron rate normalised to counts per hour.

Meteorology and external observations¶

Enum member	Default label	Purpose
`AIR_PRESSURE`	`air_pressure`	Atmospheric pressure in neptoon's internal pressure unit.
`AIR_RELATIVE_HUMIDITY`	`air_relative_humidity`	Relative humidity in percent.
`AIR_TEMPERATURE`	`air_temperature`	Air temperature in degrees Celsius.
`INCOMING_NEUTRON_INTENSITY`	`incoming_neutron_intensity`	Incoming/reference neutron-monitor intensity.
`SATURATION_VAPOUR_PRESSURE`	`saturation_vapour_pressure`	Calculated saturation vapour pressure.
`ACTUAL_VAPOUR_PRESSURE`	`actual_vapour_pressure`	Calculated actual vapour pressure.
`ABSOLUTE_HUMIDITY`	`absolute_humidity`	Calculated absolute air humidity.
`PRECIPITATION`	`precipitation`	Precipitation for the record interval.
`PRECIPITATION_DAILY`	`precipitation_daily`	Daily precipitation product.
`SOLAR_RADIATION`	`solar_radiation`	Solar-radiation observation.

Correction factors and corrected neutrons¶

Enum member	Default label	Purpose
`HUMIDITY_CORRECTION`	`humidity_correction`	Epithermal-neutron humidity correction factor.
`INTENSITY_CORRECTION`	`incoming_neutron_intensity_correction`	Incoming-intensity correction factor.
`PRESSURE_CORRECTION`	`atmospheric_pressure_correction`	Atmospheric-pressure correction factor.
`ABOVEGROUND_BIOMASS_CORRECTION`	`aboveground_biomass_correction`	Above-ground biomass correction factor.
`CORRECTED_EPI_NEUTRON_COUNT`	`corrected_epithermal_neutrons`	Corrected epithermal neutron product before optional final filtering.
`CORRECTED_EPI_NEUTRON_COUNT_FINAL`	`corrected_epithermal_neutrons`	Downstream selector for the final corrected-neutron signal.
`RAW_EPI_NEUTRON_COUNT_UNCERTAINTY`	`epithermal_neutrons_uncertainty_cph`	Raw epithermal neutron counting uncertainty in cph.
`CORRECTED_EPI_NEUTRON_COUNT_UNCERTAINTY`	`corrected_epithermal_neutrons_uncertainty`	Uncertainty of the corrected neutron signal.
`CORRECTED_EPI_NEUTRON_COUNT_UPPER`	`corrected_epithermal_neutrons_upper`	Upper corrected-neutron uncertainty bound.
`CORRECTED_EPI_NEUTRON_COUNT_LOWER`	`corrected_epithermal_neutrons_lower`	Lower corrected-neutron uncertainty bound.
`THERMAL_HUMIDITY_CORRECTION`	`thermal_neutron_humidity_correction`	Thermal-neutron humidity correction factor.
`CORRECTED_THERM_NEUTRON_COUNT`	`corrected_thermal_neutrons`	Corrected thermal-neutron product.
`CORRECTED_THERM_NEUTRON_COUNT_FINAL`	`corrected_thermal_neutrons`	Downstream selector for the final thermal-neutron product.

Soil-moisture products and footprint information¶

Enum member	Default label	Purpose
`SOIL_MOISTURE_GRAV`	`smg`	Gravimetric soil-moisture product.
`SOIL_MOISTURE_VOL`	`smv`	Volumetric soil-moisture product.
`SOIL_MOISTURE_VOL_FINAL`	`smv`	Downstream selector for final volumetric soil moisture.
`SOIL_MOISTURE_UNCERTAINTY_VOL_UPPER`	`smv_uncertainty_upper`	Upper volumetric soil-moisture uncertainty bound.
`SOIL_MOISTURE_UNCERTAINTY_VOL_LOWER`	`smv_uncertainty_lower`	Lower volumetric soil-moisture uncertainty bound.
`SOIL_MOISTURE_MEASURMENT_DEPTH`	`crns_measurement_depth_cm`	Estimated CRNS measurement depth in centimetres. The enum's historical misspelling is retained for compatibility.
`SOIL_MOISTURE_MEASUREMENT_RADIUS`	`crns_measurement_radius_m`	Estimated CRNS measurement radius in metres.

Site and neutron-monitor metadata¶

Enum member	Default label	Purpose
`LATITUDE`	`latitude`	Sensor latitude.
`LONGITUDE`	`longitude`	Sensor longitude.
`ELEVATION`	`elevation`	Sensor elevation.
`REFERENCE_INCOMING_NEUTRON_VALUE`	`reference_incoming_neutron_value`	Reference intensity used by the incoming-neutron correction.
`SITE_CUTOFF_RIGIDITY`	`site_cutoff_rigidity`	Site cutoff rigidity.
`REFERENCE_MONITOR_CUTOFF_RIGIDITY`	`reference_monitor_cutoff_rigidity`	Cutoff rigidity of the reference monitor.
`REFERENCE_MONITOR_LATITUDE`	`reference_monitor_latitude`	Latitude of the reference monitor.
`REFERENCE_MONITOR_ELEVATION`	`reference_monitor_elevation`	Elevation of the reference monitor.
`REFERENCE_MONITOR_AIR_PRESSURE`	`reference_monitor_air_pressure`	Reference-monitor air pressure.
`REFERENCE_MONITOR_ATMOSPHERIC_DEPTH`	`reference_monitor_atmospheric_depth`	Reference-monitor atmospheric depth.
`NMDB_REFERENCE_STATION`	`nmdb_reference_station`	NMDB station code used for the reference series.
`RC_CORRECTION_FACTOR`	`rc_correction_factor`	Rigidity correction factor for incoming-neutron correction.

Site properties and calibration parameters¶

Enum member	Default label	Purpose
`DRY_SOIL_BULK_DENSITY`	`dry_soil_bulk_density`	Site dry-soil bulk density.
`LATTICE_WATER`	`lattice_water`	Lattice-water contribution.
`SOIL_ORGANIC_CARBON`	`soil_organic_carbon`	Soil organic carbon supplied for water-equivalent handling.
`MEAN_PRESSURE`	`mean_pressure`	Site mean atmospheric pressure.
`SITE_BIOMASS`	`site_biomass`	Site above-ground biomass value.
`N0`	`n0`	Calibration parameter used by supported neutron-to-soil-moisture methods.
`BETA_COEFFICIENT`	`beta_coefficient`	Pressure-correction beta coefficient.

Calibration-sample columns¶

Enum member	Default label	Purpose
`CALIB_DEPTH_OF_SAMPLE`	`depth_of_sample`	Calibration sample depth.
`CALIB_DISTANCE_TO_SENSOR`	`distance_to_sensor`	Horizontal distance between a sample and the sensor.
`CALIB_BULK_DENSITY`	`bulk_density`	Calibration-sample bulk density.
`CALIB_PROFILE_ID`	`profile_id`	Calibration profile identifier.
`CALIB_SOIL_MOISTURE_GRAVIMETRIC`	`smg`	Gravimetric moisture measured for a calibration sample.
`CALIB_SOIL_ORGANIC_CARBON`	`soil_organic_carbon`	Soil organic carbon measured for a calibration sample.
`CALIB_LATTICE_WATER`	`lattice_water`	Lattice water measured for a calibration sample.

Adding a permanent column to neptoon¶

Runtime relabelling is not the correct way to introduce a new scientific product. A permanent column requires a code contribution.

At minimum:

Add a member to ColumnInfo.Name in neptoon/columns/column_information.py.
Add its default string to ColumnInfo._default_representation.
Add it to SITE_INFO_TO_COLUMN_INFO when it represents a sensor/site property broadcast by CRNSDataHub.prepare_static_values().
Use str(ColumnInfo.Name.YOUR_COLUMN) throughout processing code rather than a hard-coded label.
Decide whether the column belongs in QA flag tables. Update neptoon.utils.flaggable_columns when it is metadata or a diagnostic that must not be flagged.
Add a public alias only when compatibility with an already published name is required. New documentation should use the canonical label.
Update schemas, exporters, figures, reports, selected-output behavior, and static-pivot exclusions where relevant.
Add tests for creation, flags, aggregation, saving, and any public import.
Update the user guide, API reference, and changelog for user-facing changes.

The project convention is documented in agent/conventions/column_info.md in the source repository.

Testing code that relabels columns¶

Tests must restore the registry after each test. The repository provides the reset_columns pytest marker for this purpose:

import pytest

from neptoon.columns import ColumnInfo


@pytest.mark.reset_columns
def test_custom_soil_moisture_label():
    ColumnInfo.relabel(
        ColumnInfo.Name.SOIL_MOISTURE_VOL,
        "theta_v",
    )
    assert str(ColumnInfo.Name.SOIL_MOISTURE_VOL) == "theta_v"

For external applications, use a fixture or try/finally with ColumnInfo.reset_labels().