Quality Control Operations

Common classes

QCOperation objects build on the Profile and Trace, updating the Profile and/or performing operations with side effects like creating a plot or logging information to stderr. The QCOperationContext class provides methods for common actions to minimize the number of custom QCOperation classes needed to implement a production QC workflow.

class medsrtqc.qc.operation.QCOperation

A QC operation is instantiated with the parameters that govern the functioning of the test (if any) and are run() with the medsrtqc.core.Profile as an argument. QC operations should implement the run_impl() method and use the built-in methods to do any data updates or communication with the user.

error(message): Convenience wrapper for QCOperationContext.error()

log(message): Convenience wrapper for QCOperationContext.log()

pyplot(): Convenience wrapper for QCOperationContext.pyplot()

run(profile, previous_profile=None, context=None): Run the test. This is the method used by callers to actually run the test. The default method temporarily sets self.profile and self.previous_profile and calls run_impl().

run_impl(): Test implementation. This method must be implemented by test subclasses.

update_trace(k, trace): Convenience wrapper for QCOperationContext.update_trace()

class medsrtqc.qc.operation.QCOperationContext

QC operations may be run in many different contexts. The obvious context is to apply the result of the operation to the underlying Profile (the default), but this class is used to give flexibility should users wish to do something else (e.g., print what actions would be taken without actually applying them) or require specialized actions to perform data updates that are impossible or inconvenient to implement in the Profile subclass.

error(profile, message): Shortcut for raise QCOperationError()

log(profile, message): Print a log message for a given Profile. The default method prints the message to sys.stderr.

pyplot(profile): Get a version of matplotlib.pyplot used for use in a with statement. The default method returns a context manager that wraps a dummy version of the module that does nothing.

update_trace(profile, k, trace): Updates a given Trace for a Profile. The default method runs profile[k] = trace.

exception medsrtqc.qc.operation.QCOperationError(*args, profile=None, **kwargs): An Exception subclass with attributes profile, trace, and trace_key. These errors give an opportunity for inspection on debugging and potentially more informative printing because they contain some context.

class medsrtqc.qc.operation.QCOperationProfileContext(op, profile=None, previous_profile=None, context=None): Internal class used by QCOperation.run() to set the profile, previous profile, and context during execution of QCOperation.run_impl().

class medsrtqc.qc.flag.Flag

Flags for check output. These values are valid values of the qc and adjusted_qc attributes of a Trace object. Utility functions are provided as static methods to get the name or value of a flag or to update flag values ensuring that values that are already marked at a “worse” QC level are not inadvertently changed.

static label(flag): Return the label of a QC flag

static update_safely(qc, to, where=None): Safely update qc to the value to. Values that are already marked at a “worse” QC level are not modified.

static value(label): Return the value of a QC flag

Named Argo QC tests

The named_tests module contains the named tests in the Argo QC handbook. The main difference between a these tests and a regular QCOperation is that they (1) have an ID, binary ID,and NVS URI and (2) the run() method returns True or False to indicate a “passed” or “failed” check. Most tests also modify the input Profile (e.g., to set flags).

class medsrtqc.qc.named_tests.FrozenProfileTest

This test is used to detect a float that reproduces the same profile (with very small deviations) over and over again.

run_impl(): Run the test and return True if it passed or False otherwise

class medsrtqc.qc.named_tests.GlobalRangeTest

This test applies a gross filter on observed values for DOXY, TEMP_DOXY, CHLA and BBP.

run_impl(): Run the test and return True if it passed or False otherwise

class medsrtqc.qc.named_tests.PressureIncreasingTest

The pressure increasing test checks for monotonically increasing pressure. The test modifies the QC flags for PRES, TEMP, and PSAL and fails if any of these flags were set to Flag.BAD.

>>> from medsrtqc.qc.named_tests import PressureIncreasingTest
>>> from medsrtqc.qc.flag import Flag
>>> from medsrtqc.core import Trace, Profile
>>> import numpy as np
>>> qc5 = np.repeat([Flag.NO_QC], 5)
>>> pres =  Trace([0, 50, 0, 50, 100], qc=qc5)
>>> pres.pres = pres.value
>>> prof = Profile({
...     'PRES': pres,
...     'TEMP': Trace([10, 5, 7, 7, 7], qc=qc5, pres=pres.value),
...     'PSAL': Trace([8, 9, 10, 11, 12], qc=qc5, pres=pres.value)
... })
>>> PressureIncreasingTest().run(prof)
>>> prof['PRES']

run_impl(): Run the test and return True if it passed or False otherwise

class medsrtqc.qc.named_tests.QCTest

Whereas a QCOperation is generic and can do anything, a QCTest is a specific operation defined within the Argo data management framework. ``QCTest``s have names, binary IDs, and specific descriptions.

run_impl() → bool: Run the test and return True if it passed or False otherwise

class medsrtqc.qc.named_tests.SpikeTest

The difference between sequential measurements, where one measurement is significantly different from adjacent ones, is a spike in both size and gradient. For CHLA and BBP spikes contain information, mainly in case of positive spikes. This is the reason why we set up a test to discriminate negative spikes for these variables. For DOXY and TEMP_DOXY, negative and positive spikes are flagged.

run_impl(): Run the test and return True if it passed or False otherwise

class medsrtqc.qc.named_tests.StuckValueTest

This test looks for all biogeochemical sensor outputs (i.e. ‘i’ and ‘b’ parameter measurements transmitted by the float) in a vertical profile being identical.

run_impl(): Run the test and return True if it passed or False otherwise