Index

CRNS Data Hub

The CRNSDataHub is used to manage the time series data throughout the processing steps. Some key features:

• It stores a DataFrame for a site
• As we progress through the steps, data can be added to the DataFrame and the shadow DataFrame's updated.

Raw data is checked against the RawDataSchema which is a first line of defense against incorrectly formatted tables. Should a fail happen here data must be either reformatted using one of the provided routines or manually formatted to match the standard.

Inputs to the CRNSDataHub.

Parameters:

Name	Type	Description	Default
crns_data_frame	DataFrame	CRNS data in a dataframe format. It will be validated to ensure it has been formatted correctly.	required
configuration_manager	ConfigurationManager	A ConfigurationManager instance storing configuration YAML information, by default None	required
quality_assessor	SaQC	SaQC object which is used for quality assessment. Used for the creation of flags to define poor data.	None
validation	bool	Toggle for whether to have continued validation of data tables during processing (see data_management>data_validation_tables.py for examples of tables being validated). These checks ensure data is correctly formatted for internal processing.	True
calibration_samples_data	DataFrame	The sample data taken during the calibration campaign.	None
data_storage	DataStorageConfig	Data storage configuration from the sensor config. When provided, save_data() uses save_location as the default output directory instead of cwd().	None

crns_data_frame property writable

crns_data_frame

flags_data_frame property writable

flags_data_frame

attach_nmdb_data

attach_nmdb_data(station='JUNG', new_column_name=str(INCOMING_NEUTRON_INTENSITY), resolution='60', nmdb_table='revori', reference_value=None)

Utilises the NMDBDataAttacher class to attach NMDB incoming intensity data to the crns_data_frame. Collects data using www.NMDB.eu

See NMDBDataAttacher documentation for more information.

Parameters:

Name	Type	Description	Default
station	str	The station to collect data from, by default "JUNG"	'JUNG'
new_column_name	str	The name of the column were data will be written to, by default "incoming_neutron_intensity"	str(INCOMING_NEUTRON_INTENSITY)
resolution	str	The resolution in minutes, by default "60"	'60'
nmdb_table	str	The table to pull data from, by default "revori"	'revori'
reference_value	int	The reference value of the neutron monitor, if left as None it will use the value from the first data point in the time series.	None

Report

Neutron monitoring data was attached from NMDB.eu. The station used was {station} at a resolution of {resolution} minutes. The data table used was {nmdb_table}.

add_quality_flags

add_quality_flags(custom_flags=None, add_check=None)

Add QualityChecks to undertake on the dataframe

Parameters:

Name	Type	Description	Default
custom_flags	QualityAssessmentFlagBuilder	user can build a QualityAssessmentFlagBuilder with checks and attach this as a whole, by default None	None
add_check	Check	user can add individual Checks, or a list of Checks. These will be then added to the QualityAssessmentFlagBuilder, by default None	None

apply_quality_flags

apply_quality_flags()

Flags data based on quality assessment. A user can supply a QualityAssessmentFlagBuilder object that has been custom built, they can flag using the config file (if supplied), or they can choose a standard flagging routine.

Everything is off by default so a user must choose.

Parameters:

Name	Type	Description	Default
custom_flags	QualityAssessmentFlagBuilder	A custom built set of Flags , by default None	required
flags_from_config	bool	State if to conduct QA using config supplied configuration, by default False	required
flags_default	str	A string representing a default version of flagging, by default None	required

select_correction

select_correction(correction_type='empty', correction_theory=None)

Method to select corrections to be applied to data.

Individual corrections can be applied using a CorrectionType and CorrectionTheory. If a user assigns a CorrectionType without a CorrectionTheory, then the default correction for that CorrectionType is applied.

Parameters:

Name	Type	Description	Default
correction_type	CorrectionType	A CorrectionType, by default "empty"	'empty'
correction_theory	CorrectionTheory	A CorrectionTheory, by default None	None

correct_neutrons

correct_neutrons()

Create correction factors as well as the corrected epithermal neutrons column.

calibrate_station

calibrate_station(config=None)

Calibrate the sensor

Parameters:

Name	Type	Description	Default
config	CalibrationConfiguration	Config file which contains all the required info for calibration, by default None	None

Raises:

Type	Description
ValueError	When no calibration data provided

Report

Calibration was undertaken. The N0 number was calculated as {n0}, using the {config.neutron_conversion_method} method. From the samples, the average dry soil bulk density is {avg_dry_soil_bulk_density}, the average soil organic carbon is {avg_soil_organic_carbon}, and the average lattice water content is {avg_lattice_water}.

produce_soil_moisture_estimates

produce_soil_moisture_estimates(n0=None, conversion_theory='desilets_etal_2010', dry_soil_bulk_density=None, lattice_water=None, soil_organic_carbon=None, koehli_parameters='Mar21_mcnp_drf')

Produces SM estimates with the NeutronsToSM class. If values for n0, dry_soil_bulk_density, lattice_water, or soil_organic_carbon are not supplied, the values are taken from the internal sensor_info class.

Parameters:

Name	Type	Description	Default
n0	float	n0 calibration term, by default None	None
dry_soil_bulk_density	float	given in g/cm3, by default None	None
lattice_water	float	given as decimal percent e.g., 0.01, by default None	None
soil_organic_carbon	float	Given as decimal percent, e.g., 0.001, by default None	None

Report

Soil moisture was estimated using an n0 of {default_params[n0]}, a bulk density of {default_params[dry_soil_bulk_density]}, a lattice water content of {default_params[lattice_water]}, and a soil organic carbon content of

Column Information System