Metric Configuration

Datachecks will read metrics configuration under the key metrics in the configuration file. User can define multiple metrics in the configuration file under metrics key.

For example:

metrics:
  - name: freshness_example
    type: freshness
    resource: mysql_db.table_name.last_updated
    validation:
      threshold: "> 86400" ##Freshness metric value is in seconds. Validation error if metric value is greater than 86400 seconds.

Configuration Details

Parameter	Mandatory	Description
name		The name of the metric. The name should be unique.
type		The type of the metric. Possible values are freshness, row_count etc. Type of metric mentioned in every metric documentation
resource		The resource for which metric should be generates. A resource can be a Table, Index, Collection or Field. In case of Table, Index or Collection the pattern of the resource name would be <datasource>.<table_name> or <datasource>.<index_name> In case of a Field the pattern of the resource name would be <datasource>.<table_name>.<field_name> or <datasource>.<index_name>.<field_name>
filters		The filters to be applied on the resource. Filters can have where as a nested field. In where field we can pass SQL Query(In ase of SQl DB) or Search Query(In ase of search engine). For example: filters:  where: city = 'bangalore' AND age >= 30
validation		The validation will be applied on the metric value. A validation error will be invoked if the metric value violate threshold value. Possible values for threshold are >, >=, = , <, <=. We can combine multiple operators For example: validation:  threshold: ">= 10 & <= 100"

Metric Types

Supported metric types are

Metric Group	Metric Type
Reliability	Freshness
Reliability	Row Count
Reliability	Document Count
Numeric Distribution	Average
Numeric Distribution	Minimum
Numeric Distribution	Maximum
Numeric Distribution	Sum
Numeric Distribution	Variance
Numeric Distribution	Standard Deviation
Uniqueness	Distinct Count
Uniqueness	Duplicate Count
Completeness	Null Count
Completeness	Null Percentage
Completeness	Empty Count
Completeness	Empty Percentage
Special	Combined
Special	Custom SQL