Smoothing neutron and soil-moisture data¶

Smoothing suppresses high-frequency counting noise after the scientific corrections have been applied. In the normal workflow, corrected neutron counts are smoothed before soil-moisture conversion. Soil moisture can also be smoothed separately when a downstream product requires it.

Do not smooth raw neutron counts before pressure, humidity, incoming-intensity, and biomass corrections. Doing so mixes environmental influences into the smoothing window.

Configuration-driven smoothing¶

data_smoothing:
  smooth_corrected_neutrons: true
  smooth_soil_moisture: false
  settings:
    algorithm: rolling_mean
    window: 12h
    alignment: centered
    min_proportion_good_data: 0.7
    insufficient_data_policy: skip

Available settings:

Setting	Default	Description
`smooth_corrected_neutrons`	`true`	Smooth the corrected epithermal-neutron signal.
`smooth_soil_moisture`	`false`	Smooth the calculated soil-moisture product.
`settings.algorithm`	`rolling_mean`	`rolling_mean`, `savitzky_golay`, or the legacy spelling `savitsky_golay`.
`settings.window`	`12h`	Pandas-compatible duration such as `30min`, `12h`, or `1d`.
`settings.alignment`	`centered`	Rolling-mean placement: `centered` uses observations on both sides of the timestamp; `trailing` uses only current and earlier observations. The input alias `centred` is accepted and normalised to `centered`.
`settings.min_proportion_good_data`	`0.7`	Minimum valid-data proportion required in a rolling window.
`settings.poly_order`	`2`	Polynomial order for Savitzky-Golay smoothing.
`settings.insufficient_data_policy`	`skip`	Savitzky-Golay behaviour when too few valid observations are available: `skip`, `raise`, or `fallback_rolling_mean`.

Rolling mean¶

The rolling mean uses a time-based window. Neptoon estimates the number of expected observations from the time-series cadence and requires at least min_proportion_good_data of them before producing an output value.

The default centered alignment is intended for retrospective CRNS processing: each value uses observations on both sides of the output timestamp, avoiding the phase lag of a trailing average. A centered window therefore uses future observations and has edge effects at both ends of the series. Select trailing explicitly for causal or operational processing where values after time t must not contribute. The British spelling centred, case variants such as CENTERED, and an explicit YAML null are accepted as forgiving inputs and normalised to the canonical default centered.

For example, with hourly data, a 12h window and a threshold of 0.7, at least roughly 70% of the expected observations must be valid. Windows with insufficient data remain NaN.

data_smoothing:
  smooth_corrected_neutrons: true
  settings:
    algorithm: rolling_mean
    window: 24h
    alignment: centered
    min_proportion_good_data: 0.75

Savitzky-Golay¶

Savitzky-Golay smoothing fits a local polynomial. It requires poly_order and a time window that can be converted into a suitable number of observations.

data_smoothing:
  smooth_corrected_neutrons: true
  settings:
    algorithm: savitzky_golay
    window: 12h
    poly_order: 2
    insufficient_data_policy: fallback_rolling_mean

Use fallback_rolling_mean when short deployments or large gaps should still produce a conservative smoothed series. Use raise when insufficient data must stop the run, and skip when the smoothing step should leave the signal unchanged.

Programmatic API¶

from neptoon.columns import ColumnInfo

hub.smooth_data(
    column_to_smooth=str(
        ColumnInfo.Name.CORRECTED_EPI_NEUTRON_COUNT
    ),
    smooth_method="rolling_mean",
    window="12h",
    alignment="centered",
    min_proportion_good_data=0.7,
)

For Savitzky-Golay:

hub.smooth_data(
    column_to_smooth=str(
        ColumnInfo.Name.CORRECTED_EPI_NEUTRON_COUNT
    ),
    smooth_method="savitzky_golay",
    window="12h",
    poly_order=2,
    insufficient_data_policy="fallback_rolling_mean",
)

The lower-level implementation is available as neptoon.data_prep.SmoothData. See the stable data preparation API reference.

Output-column behaviour¶

For the canonical corrected-neutron and soil-moisture columns, Neptoon can update the corresponding *_FINAL selector to point at the newly smoothed column. This allows downstream conversion and export steps to consume the smoothed signal without overwriting the correction-only source column.

Trailing rolling means use the unsuffixed alignment form, for example rollingmean_12h. The default centered rolling mean includes _centered in the generated column name so the two scientific choices cannot be confused.

Set auto_update_final_col=False in direct Python calls when smoothing a custom column or when the caller will manage output-column selection explicitly.

Input requirements and common errors¶

The data must use a monotonic increasing DatetimeIndex.
At least two timestamped rows are required.
The smoothing window must be a non-empty duration string.
Rolling-mean alignment must resolve to centered or trailing; centred and case variants are accepted as input aliases.
min_proportion_good_data must be between 0 and 1.
Savitzky-Golay requires a non-negative integer poly_order.
The requested smoothing window must not be shorter than the data timestep.
A custom column cannot update Neptoon's canonical *_FINAL selectors unless automatic updating is disabled.

Inspect the correction-only and smoothed columns together before selecting a window for production processing. Very long windows reduce noise but can also suppress real rainfall and drying responses.