Validation

Validations are components responsible for monitoring and validating data in your sources. While a validation is connected to a single source, each source can have multiple validators to track individual metric calculations and their statistics.

Datachecks monitors metrics for database tables or columns and alert you if they go out of bounds.

Example:

• The total number of transactions processed in the payments table per day.
• The percentage of duplicate entries in the customer_email column.

On the Validations page, you can view and manage all the validations you have access to on the platform or manage validators per source. For more information, see Manage Validations

You can apply filters to exclude data from validation and use collections to partition the validation into one or more parallel groups for analysis. You can segment and monitor validation results over specific timeframes. You can configure thresholds to define what values of the calculated metric should be considered an incident. All metric values that exceed your threshold are flagged as alerts and sent in notifications.

Managing Validations

You can review and manage validations form the Validations page, or the individual Validator details page.

When you arrive on the Validations page, you’ll see all the validations added to your workspace, along with their status. The key columns include:

• Name – The name of the validation.
• Type – The category to which the validation belongs.
• Last Results – A visual indicator of recent validation outcomes.
• Last Evaluated – The timestamp of the last time this validation was executed.

On each validation, you can execute, pause, edit, delete the validation, or view details as needed.

Above Validation Details, you can create a new validation by clicking on New Validation. For more information, see Configure Validation.

You can group multiple validations into a Collection by clicking on Add to Collection. For more information, see Collections.

On the right hand side you You can filter alerts by their status, datasource, and validation to focus on critical issues and take necessary actions efficiently.

View Validation

To view an individual validation, select the specific alert from the list you want to explore.It will open a more detailed validation incident view.

Incident Metrics Graph

The metrics graph provides details about the values and bounds of the detected incidents, their severity, and when each was detected. Each data point represents the number of incidents detected within a specific timeframe.

You can use this graph to identify spikes or unusual patterns, correlate incidents with recent changes. Hover on a datapoint to display its detected time range, value, bounds, and severity.

Incidents List

Below the Incident Metrics Graph, the Incidents List displays:

Additionally, on the individual validations you can:

• Filter validations by a specific timeframe.
• Pause or execute the validation to manage the process.
• Edit or delete the validation as needed.
• View additional info for detailed insights into the validation.

Configure Validation

Validators are configured on sources. You can add more than one validator on a source to monitor calculated metrics and statistics for different use cases.

1. To create a new validation, start by navigating to Validation via the left-hand sidebar and clicking on New Validation.

2. Choose the Validation Type you want to configure for this source. For more information, see the Validation Type section.

3. Provide Validator Details

   • Select the relevant Data Source and Dataset for your validation
   • Give a name to your validation, or optionally, you can use the suggested name.
   
   

4. Configure Filters section to apply conditions to your data.

   For example, select the column tr_volume, set the condition to Less Than (<), and enter the value 5000. This will filter the data to include only records where the trading volume is below 5000.

5. Define the Threshold to use for detecting anomalies and triggering alert notifications. For more information, see About Thresholds.

6. Configure the Schedule Frequency by choosing how often the validation should run

7. Click Create to create the validator.