Worked soil-profile examples

This page collects complete patterns for the stable neptoon.soil_profile service. The examples progress from a normal local profile reconstruction to explicit uncertainty analysis and a layered-soil/Windows multiprocessing setup.

Before using them, read Reconstructing a sparse in-situ soil-moisture profile for input assumptions and Understanding soil-profile uncertainty and error estimates for interpretation of the reported error budget.

Example 1: three in-situ depths with weather forcing

Assume a pandas DataFrame indexed by timestamps and containing volumetric water content at 5, 20, and 50 cm, together with daily air temperature, relative humidity, and precipitation:

from neptoon.soil_profile import (
    ProfileInputColumns,
    SparseProfileRequest,
    SparseSoilProfileService,
)

request = SparseProfileRequest(
    latitude_deg=53.10,
    sensor_columns={
        5.0: "sm_5cm",
        20.0: "sm_20cm",
        50.0: "sm_50cm",
    },
    input_columns=ProfileInputColumns(
        temperature_mean="air_temperature_c",
        relative_humidity="relative_humidity_percent",
        precipitation="precipitation_mm",
    ),
    texture_class="sandy_loam",
    bulk_density_g_cm3=1.42,
    measurement_std_by_depth={
        5.0: 0.012,
        20.0: 0.015,
        50.0: 0.020,
    },
)

result = SparseSoilProfileService().reconstruct(data, request)

profile = result.soil_moisture
uncertainty = result.uncertainty
lower_90, upper_90 = result.get_interval(0.90)
diagnostics = result.diagnostics

The input soil-moisture columns must contain volumetric fractions such as 0.24, not percentages such as 24. Sensor depths are in centimetres. The three measurement standard deviations in this example deliberately increase with depth to represent different calibration/representativeness quality; use values based on the actual sensors and installation.

A compact inspection of the result can preserve the observational-support class:

long_profile = result.to_long_frame().merge(
    result.layer_metadata[["layer", "centre_cm", "support"]],
    on="layer",
    how="left",
)
print(long_profile.head())

For an operational run, also inspect at least the water-balance, innovation, and sensor-gap diagnostics described in the uncertainty guide.

Example 2: time-varying measurement error and uncertainty decomposition

Some sensor pipelines provide a daily quality-derived uncertainty estimate. Map those columns separately from the soil-moisture observations:

from neptoon.soil_profile import (
    ProfileInputColumns,
    SparseProfileRequest,
    SparseSoilProfileService,
)

request = SparseProfileRequest(
    latitude_deg=50.82,
    sensor_columns={
        5.0: "theta_5",
        15.0: "theta_15",
        30.0: "theta_30",
    },
    measurement_std_columns={
        5.0: "theta_5_std",
        15.0: "theta_15_std",
        30.0: "theta_30_std",
    },
    input_columns=ProfileInputColumns(
        temperature_mean="temperature_c",
        temperature_min="temperature_min_c",
        temperature_max="temperature_max_c",
        relative_humidity="rh_percent",
        precipitation="rain_mm",
        vegetation_cover="vegetation_fraction",
    ),
    texture_class="loam",
    texture_class_probabilities={
        "sandy_loam": 0.20,
        "loam": 0.60,
        "silt_loam": 0.20,
    },
    bulk_density_g_cm3=1.38,
    model_overrides={
        "decompose_uncertainty": True,
        "ensemble_size": 64,
        "hydraulic_parameter_groups": 8,
        "quantify_model_choice_uncertainty": True,
    },
)

result = SparseSoilProfileService().reconstruct(data, request)

Daily measurement-error values take precedence over a constant depth-specific or global fallback. Non-finite or non-positive daily values fall back to the configured constant. Positive daily values are bounded internally to the supported 0.005-0.12 m³/m³ range.

The individual uncertainty products can then be summarized without confusing standard deviations and variances:

import pandas as pd

budget = pd.DataFrame(
    {
        "total": result.uncertainty.mean(axis=1),
        "statistical": result.uncertainty_statistical.mean(axis=1),
        "hydraulic": result.uncertainty_hydraulic_parameters.mean(axis=1),
        "structure": result.uncertainty_model_structure.mean(axis=1),
        "model_choice": result.uncertainty_model_choice.mean(axis=1),
    }
)
print(budget.tail())

quantify_model_choice_uncertainty=True runs a complete alternative transport reconstruction and therefore approximately doubles runtime. In this mode, uncertainty_model_structure already includes the model-choice term in quadrature for backward-compatible result semantics; model_choice above is a separate attribution diagnostic and must not be added to structure again when reconstructing the total.

For production uncertainty validation, remove a complete probe depth from sensor_columns for multi-week blocks and score the reconstructed value against that withheld series. Randomly hiding isolated points is not a substitute because it does not test event memory, missing-sensor uncertainty growth, or smoothing.

Example 3: layered hydraulic priors and Windows-safe process parallelism

A site with an independently known restrictive subsoil can supply a separate hydraulic prior for that depth interval. Process parallelism is opt-in and must follow Python's guarded-entry-point convention on Windows.

from neptoon.soil_profile import (
    HydraulicProfilePrior,
    ProfileInputColumns,
    SparseProfileRequest,
    SparseSoilProfileService,
)


def reconstruct_site(data):
    request = SparseProfileRequest(
        latitude_deg=53.10,
        sensor_columns={
            5.0: "sm_5cm",
            20.0: "sm_20cm",
            60.0: "sm_60cm",
        },
        input_columns=ProfileInputColumns(
            temperature_mean="temperature_c",
            relative_humidity="rh_percent",
            precipitation="precipitation_mm",
        ),
        texture_class="sandy_loam",
        bulk_density_g_cm3=1.40,
        hydraulic_profile_priors=(
            HydraulicProfilePrior(
                top_depth_cm=35.0,
                bottom_depth_cm=90.0,
                texture_class="clay_loam",
                lower_interface_conductivity_multiplier=0.10,
                name="restrictive subsoil",
                provenance="independent soil survey",
            ),
        ),
        model_overrides={
            "ensemble_parallel_backend": "process",
            "ensemble_max_workers": 4,
            "ensemble_process_start_method": "spawn",
        },
    )
    return SparseSoilProfileService().reconstruct(data, request)


if __name__ == "__main__":
    result = reconstruct_site(data)
    print(result.layer_metadata[
        [
            "layer",
            "support",
            "hydraulic_horizon_name",
            "hydraulic_horizon_provenance",
        ]
    ])

The process workers perform deterministic member forecasts only. Random forcing, parameter-group setup, process noise, analysis, diagnostics, and result ordering remain controlled by the parent process so the configured random seed has the same scientific meaning across worker counts. The process backend should be benchmarked on the target machine; small ensembles can be faster in serial mode.

The horizon in this example is an externally supplied prior. The model does not infer a clay layer merely because the deep sensor is wet, and the provenance field is retained in layer_metadata so downstream validation can distinguish measured prior information from assimilated moisture observations.

Example 4: standalone YAML and exported uncertainty tables

For command-independent profile reconstruction, use the dedicated compact YAML format. It is intentionally separate from Neptoon's main CRNS sensor/process configuration schemas.

input:
  csv: sparse_profile_input.csv
  timestamp_column: timestamp
  timezone: UTC

site:
  latitude_deg: 53.10
  texture_class: sandy_loam
  bulk_density_g_cm3: 1.42

sensors:
  - depth_cm: 5
    column: sm_5cm
    measurement_std: 0.012
  - depth_cm: 20
    column: sm_20cm
    measurement_std: 0.015
  - depth_cm: 50
    column: sm_50cm
    measurement_std: 0.020

weather:
  temperature_mean: air_temperature_c
  relative_humidity: relative_humidity_percent
  precipitation: precipitation_mm

model:
  maximum_depth_cm: 80
  smoother_lag_days: 30
  overrides:
    decompose_uncertainty: true
    hydraulic_parameter_groups: 8

output:
  directory: profile_output

Run it from Python:

from neptoon.soil_profile import SparseSoilProfileService

result = SparseSoilProfileService().reconstruct_from_yaml("profile.yaml")

The output directory contains the profile, total uncertainty, separated statistical/hydraulic/structural uncertainty tables, diagnostics, and layer metadata. The compact YAML interface currently accepts a constant measurement_std per sensor; use the Python SparseProfileRequest API when daily measurement_std_columns are required. The repository also contains the copyable examples/configuration_recipes/sparse_soil_profile.yaml, sparse_soil_profile_uncertainty.yaml, and sparse_soil_profile_layered_windows.yaml recipes.

Choosing between the examples

Situation	Start with
Normal pandas workflow with known probe depths	Example 1
Quality-dependent observation errors or uncertainty research	Example 2
Independently known hydraulic horizons or Windows process workers	Example 3
File-based reproducible batch setup	Example 4

For every case, keep the scientific validation question separate from whether the code executed successfully. A successful reconstruction means the numerical and configuration contracts were satisfied; it does not establish that deep extrapolation or the nominal uncertainty interval is calibrated for the site.