Key Functions

synopsis:: BASIN-3D Synthesis API
module author:: Val Hendrix <vhendrix@lbl.gov>, Danielle Svehla Christianson <dschristianson@lbl.gov>. Catherine Wong <catwong@lbl.gov>

Functions 

register() - Register the specified plugins or implicitly register loaded plugins

synthesis.DataSynthesizer Functions 

DataSynthesizer.attribute_mappings()- Search for attribute_mappings which describe how the datasource vocabularies are mapped to BASIN-3D vocabularies, including observed properties, statistics, result_quality, etc.
DataSynthesizer.measurement_timeseries_tvp_observations()- Search for Measurement Timeseries TVP Observations from specified Monitoring features and observed property variables
DataSynthesizer.monitoring_features()- Search for all monitoring features, features by parent monitoring features, features by monitoring feature identifiers, or look for a single monitoring feature by id.
DataSynthesizer.observed_properties()- Search for observed properties

class basin3d.synthesis.DataSynthesizer(plugins, catalog)[source]

Bases: object

Synthesis API

attribute_mappings(datasource_id=None, attr_type=None, attr_vocab=None, from_basin3d=False)[source]

>>> from basin3d.plugins import usgs
>>> from basin3d import synthesis
>>> synthesizer = synthesis.register()
>>> response = synthesizer.attribute_mappings()  # list all attribute mappings registered
>>> for attr_mapping in response:
...     print(f'{attr_mapping.attr_type} | {attr_mapping.basin3d_vocab} -- {attr_mapping.datasource_vocab}')
OBSERVED_PROPERTY:SAMPLING_MEDIUM | PH:WATER -- 00400
OBSERVED_PROPERTY:SAMPLING_MEDIUM | RDC:WATER -- 00060
OBSERVED_PROPERTY:SAMPLING_MEDIUM | WLE:WATER -- 63161
OBSERVED_PROPERTY:SAMPLING_MEDIUM | WT:WATER -- 00010
OBSERVED_PROPERTY:SAMPLING_MEDIUM | DO:WATER -- 00300
...

>>> response = synthesizer.attribute_mappings(datasource_id='USGS', attr_type='STATISTIC')
>>> for attr_mapping in response:
...     print(f'{attr_mapping.attr_type} | {attr_mapping.basin3d_vocab} -- {attr_mapping.datasource_vocab}')
STATISTIC | MEAN -- 00003
STATISTIC | MIN -- 00002
STATISTIC | MAX -- 00001
STATISTIC | TOTAL -- 00006

>>> response = synthesizer.attribute_mappings(datasource_id='USGS', attr_type='RESULT_QUALITY', attr_vocab=['VALIDATED', 'ESTIMATED'], from_basin3d=True)
>>> for attr_mapping in response:
...     print(f'{attr_mapping.attr_type} | {attr_mapping.basin3d_vocab} -- {attr_mapping.datasource_vocab}, {attr_mapping.datasource_desc}')
RESULT_QUALITY | ESTIMATED -- e, Value has been edited or estimated by USGS personnel and is write protected
RESULT_QUALITY | ESTIMATED -- E, Value was computed from estimated unit values.
RESULT_QUALITY | VALIDATED -- A, Approved for publication -- Processing and review completed.

Return all the basin3d.core.models.AttributMapping registered or those that match the specified fields.

Parameters:

datasource_id – str, The datasource identifier
attr_type – str, The attribute type (e.g., OBSERVED_PROPERTY, STATISTIC, etc)
attr_vocab – str, The attribute vocabulary, either the BASIN-3D vocabulary or the datasource vocabulary
from_basin3d – bool, True = the specified attr_vocab is a BASIN-3D vocabulary, False: the specified attr_vocab is from the datasource

Returns:

iterator of basin3d.core.models.AttributeMapping objects

property datasources: List[DataSource]: The Datasources loaded in this synthesizer :return:

measurement_timeseries_tvp_observations(query=None, **kwargs)[source]

Search for Measurement Timeseries TVP Observations for the specified query arguments.

Aggregation Duration for DAY (default) and NONE are both supported.

Search with aggregation duration DAY:

>>> from basin3d.plugins import usgs
>>> from basin3d import synthesis
>>> synthesizer = synthesis.register()
>>> timeseries = synthesizer.measurement_timeseries_tvp_observations(monitoring_feature=['USGS-09110990'],observed_property=['RDC','WT'],start_date='2019-10-01',end_date='2019-10-30',aggregation_duration='DAY')
>>> for timeseries in timeseries:
...    print(f"{timeseries.feature_of_interest.id} - {timeseries.observed_property.get_basin3d_vocab()}")
USGS-09110990 - RDC

Search with aggregation duration NONE:

>>> from basin3d.plugins import usgs
>>> from basin3d import synthesis
>>> synthesizer = synthesis.register()
>>> timeseries = synthesizer.measurement_timeseries_tvp_observations(monitoring_feature=["USGS-09110990", "USGS-09111250"],observed_property=['RDC','WT'],start_date='2020-04-01',end_date='2020-04-30',aggregation_duration='NONE')
>>> for timeseries in timeseries:
...    print(f"{timeseries.feature_of_interest.id} - {timeseries.observed_property.get_basin3d_vocab()}")
USGS-09110990 - RDC
USGS-09111250 - RDC

Parameters:

query (Optional[QueryMeasurementTimeseriesTVP]) – (optional) basin3d.core.schema.query.QueryMeasurementTimeseriesTVP object
kwargs – (required) Measurement Timeseries TVP Query parameters. See Query info below.

Return type:

DataSourceModelIterator

Returns:

a DataSourceModelIterator that yields basin3d.core.models.MeasurementTimeseriesTVPObservation objects

Note

Measurement Timeseries TVP Query parameters basin3d.core.schema.query.QueryMeasurementTimeseriesTVP

Required arguments:

* monitoring_feature List of monitoring features id(s)
* observed_property List of observed property(ies), i.e., BASIN-3D observed property vocabulary. See basin3d.synthesis.DataSynthesizer.observed_properties()
* start_date Start date YYYY-MM-DD

Optional arguments:

* end_date End date YYYY-MM-DD
* aggregation_duration A single aggregation duration basin3d.core.schema.enum.AggregationDurationEnum (YEAR|MONTH|DAY|HOUR|MINUTE|SECOND|NONE)
* statistic List of statistic(s) basin3d.core.schema.enum.StatisticEnum (MEAN|MIN|MAX|INSTANTANEOUS)
* result_quality List of result quality(ies) basin3d.core.schema.enum.ResultQualityEnum (VALIDATED|UNVALIDATED|SUSPECTED|REJECTED|ESTIMATED)
* sampling_medium List of sampling medium(s) basin3d.core.schema.enum.SamplingMediumEnum (SOLID_PHASE|WATER|GAS|OTHER)
* datasource A single data source id prefix

Note

Measurement Timeseries TVP Observation attributes basin3d.core.models.MeasurementTimeseriesTVPObservation

Attributes values are dependent on data source features.

basin3d.core.models.MappedAttribute are returned for several attributes so that the data source values are also available.

* id Observation identifier
* type Type of observation: MEASUREMENT_TVP_TIMESERIES
* observed_property The observation’s observed property basin3d.core.models.MappedAttribute
* datasource The data source basin3d.core.models.DataSource
* sampling_medium Observed property sampling medium basin3d.core.models.MappedAttribute (SOLID_PHASE, WATER, GAS, OTHER)
* phenomenon_time Datetime of the observation, for a timeseries the start and end times can be provided
* utc_offset Coordinate Universal Time offset in hours (offset in hours), e.g., +9
* feature_of_interest Monitoring Feature object basin3d.core.models.MonitoringFeature, feature on which the observation is being made
* feature_of_interest_type Feature type of the feature of interest, basin3d.core.schema.enum.FeatureTypeEnum
* result Observed values of the observed property being assessed, and (opt) their result quality, basin3d.core.models.ResultListTVP
* time_reference_position Position of timestamp in aggregated_duration (START, MIDDLE, END)
* aggregation_duration Time period represented by observation basin3d.core.models.MappedAttribute (YEAR, MONTH, DAY, HOUR, MINUTE, SECOND)
* unit_of_measurement Units in which the observation is reported
* statistic Statistical property of the observation result basin3d.core.models.MappedAttribute (MEAN, MIN, MAX, TOTAL)
* result_quality List quality assessment found in the results basin3d.core.models.MappedAttribute (VALIDATED, UNVALIDATED, SUSPECTED, REJECTED, ESTIMATED)

monitoring_features(query=None, **kwargs)[source]

Search for all Monitoring Features, Monitoring Features by parent monitoring features, or Monitoring Feature by id(s).

To see feature types for a given plugin: <plugin_module>.<plugin_class>.feature_types

Search for a single monitoring feature by id:

>>> from basin3d.plugins import usgs, epa
>>> from basin3d import synthesis
>>> synthesizer = synthesis.register()
>>> response = synthesizer.monitoring_features(id='USGS-0101')
>>> mf = response.data
>>> print(f"{mf.id} - {mf.description}")
USGS-0101 - SUBREGION: St. John

Search for all USGS monitoring features:

>>> for mf in synthesizer.monitoring_features(datasource='USGS', feature_type='region'): 
...     print(f"{mf.id} - {mf.description}")
USGS-01 - REGION: New England
USGS-02 - REGION: Mid Atlantic
USGS-03 - REGION: South Atlantic-Gulf
...

Search for USGS points by parent (subbasin) monitoring features:

>>> for mf in synthesizer.monitoring_features(feature_type='point',parent_feature=['USGS-17040101']): 
...    print(f"{mf.id} {mf.coordinates and [(p.x, p.y) for p in mf.coordinates.absolute.horizontal_position]}")
USGS-13010000 [(-110.6647222, 44.1336111)]
USGS-13010065 [(-110.6675, 44.09888889)]
USGS-13010450 [(-110.5874305, 43.9038296)]
...

Search for USGS points by monitoring features identifiers:

>>> for mf in synthesizer.monitoring_features(feature_type='point', monitoring_feature=['USGS-13010000', 'USGS-13010450']): 
...    print(f"{mf.id} {mf.coordinates and [(p.x, p.y) for p in mf.coordinates.absolute.horizontal_position]}")
USGS-13010000 [(-110.6647222, 44.1336111)]
USGS-13010450 [(-110.5874305, 43.9038296)]

Unsupported feature types warning:

The code below is an example of what you will see if a registered plugin does not support the requested feature type.

>>> response_itr = synthesizer.monitoring_features(feature_type='horizontal_path')
>>> for mf in response_itr:
...   print(mf)
...

Output warning messages from the returned iterator

This is an example of checking the synthesis response messages in the basin3d.core.synthesis.DataSourceModelIterator.

>>> response_itr.synthesis_response.messages
[SynthesisMessage(msg='Feature type HORIZONTAL_PATH not supported by USGS.', level='WARN', where=['USGS', 'MonitoringFeature']), SynthesisMessage(msg='Feature type HORIZONTAL_PATH not supported by EPA Water Quality eXchange.', level='WARN', where=['EPA', 'MonitoringFeature'])]

Parameters:

query (Optional[QueryMonitoringFeature]) – (optional) A Monitoring Feature Query basin3d.core.schema.query.QueryMonitoringFeature object
kwargs – (optional) Monitoring Feature Query parameters. See Query info below.

Return type:

Union[DataSourceModelIterator, SynthesisResponse]

Returns:

a single SynthesisResponse for a query by id or a DataSourceModelIterator for multple basin3d.core.models.MonitoringFeature objects.

Note

Monitoring Feature Query parameters basin3d.core.schema.query.QueryMonitoringFeature

All parameters are optional.

* datasource A single data source id prefix.
* feature_type The basin3d.core.schema.enum.FeatureTypeEnum of the desired Monitoring Feature(s). Data Sources may not support all Feature Types.

Only one of the following can be specified in a query:

* id A single Monitoring Feature ID for the Monitoring Feature desired. Returns a single Synthesis Response object.
* monitoring_feature List of Monitoring Feature IDs for the Monitoring Features desired. Returns iterator of MonitoringFeature objects.
* parent_feature List of Monitoring Feature IDs for the parent features of the desired Monitoring Features. Returns iterator of MonitoringFeature objects.

Note

Monitoring Feature attributes basin3d.core.models.MonitoringFeature

Attributes values are dependent on data source features.

* id Unique feature identifier, prefixed by data source id
* name Feature name
* description Description of the Monitoring Feature
* feature_type basin3d.core.schema.enum.FeatureTypeEnum REGION, SUBREGION, BASIN, SUBBASIN, WATERSHED, SUBWATERSHED, SITE, PLOT, HORIZONTAL PATH, VERTICAL PATH, POINT
* observed_properties List of observed properties basin3d.core.models.ObservedProperty collected at the feature.
* related_sampling_feature_complex List of basin3d.core.models.RelatedSamplingFeature. PARENT features are currently supported.
* shape Shape of the feature: POINT, CURVE, SURFACE, SOLID
* coordinates Location of feature in absolute and/or representative datum: basin3d.core.models.Coordinate
* description_reference Additional information about the feature
* related_party List of people or organizations responsible for the feature
* utc_offset Coordinate Universal Time offset in hours (offset in hours), e.g., +9
* datasource The feature’s data source basin3d.core.models.DataSource

observed_properties()[source]

>>> from basin3d.plugins import usgs
>>> from basin3d import synthesis
>>> synthesizer = synthesis.register()
>>> response = synthesizer.observed_properties()
>>> for opv in response:
...     print(f'{opv.basin3d_vocab} -- {opv.full_name} -- {opv.units}')
ACT -- Acetate (CH3COO) -- mM
Br -- Bromide (Br) -- mM
Cl -- Chloride (Cl) -- mM
DIN -- Dissolved Inorganic Nitrogen (Nitrate + Nitrite) -- mg/L
DTN -- Dissolved Total Nitrogen (DTN) -- mg/L
F -- Fluoride (F) -- mM
...

BASIN-3D observed properties. An observed property defines what is being measured. Data source observed property vocabularies are mapped and thus synthesized to the BASIN-3D observed property vocabulary.

Returns:: an iterator of basin3d.core.models.ObservedProperty objects

exception basin3d.synthesis.SynthesisException[source]

Bases: Exception

Special Exception for Synthesis module

basin3d.synthesis.register(plugins=None)[source]

Register the specified plugins or implicitly register loaded plugins

>>> from basin3d import synthesis
>>> synthesizer = synthesis.register(['basin3d.plugins.usgs.USGSDataSourcePlugin'])
>>> synthesizer.datasources
[DataSource(id='USGS', name='USGS', id_prefix='USGS', location='https://waterservices.usgs.gov/nwis/', credentials={})]

Parameters:: plugins (Optional[List[str]]) – [Optional] plugins to registered
Returns:: DataSynthesizer(plugin_dict, catalog)

synopsis:: The BASIN-3D monitoring module
module author:: Val Hendrix <vhendrix@lbl.gov>

This module holds the logging functionality of BASIN-3D. It supports logging the messages and their contexts

Functions 

get_logger() - Get the Basin3DLogger.
configure() - Configure logging in BASIN-3D

basin3d.monitor.configure(log_config_path=None, **kwargs)[source]

Load YAML python logging configuraiton file

Parameters:: log_config_path (Optional[str]) – Path to the YAML file for configuring Python logging
Return type:: Dict
Returns:: Logging configuration as dictionary

Keyword Args Overwrite default logging config.

filters (dict)
formatters (dict)
handlers (dict)
loggers (dict)

basin3d.monitor.get_logger(name=None)[source]

Get the basin3d logger for the specified name. Using this logger provides additional context for BASIN-3D synthesis.

See logging.getLogger()

Param:: Name of the logger
Return type:: Logger
Returns:: The BASIN-3D Logger object

Key Functions

Functions

Classes

synthesis.DataSynthesizer Functions

Functions

Functions 

Classes 

synthesis.DataSynthesizer Functions 

Functions 