# Asset Discovery Agent
Source: https://ploutosaitechnologiespvtltd.mintlify.app/agents/asset-discovery
Explore your data assets using natural language. The Asset Discovery agent queries your registered schemas and generates SQL on demand.
The Asset Discovery agent lets you explore your data assets through conversation. Instead of writing queries manually, you describe what you want to know — the agent retrieves relevant schema context and either answers directly or generates a SQL query for you.
## What it does
* Answers questions about table schemas, column names, and data types
* Discovers relationships between tables using foreign key metadata
* Generates SQL queries against your registered datasets
* Provides data profile information for columns and tables
## What you can ask
Here are examples of questions the agent handles well:
* *"What tables are in the orders schema?"*
* *"What columns does the `customer` table have?"*
* *"Show me all tables that reference `product_id`."*
* *"Write a query that joins orders and customers by customer ID."*
* *"Which columns in the `sales` table might contain PII?"*
## Prerequisites
Before using the Asset Discovery agent:
1. Your data source must be connected and added to your workspace. See [Data Sources](/data-sources/overview).
2. Datasets must be registered as assets. Go to **Assets** to confirm they appear.
## Create an agent
1. Go to the **Agents** tab in the left sidebar.
2. Click **Create Agent**.
3. Select **Asset Discovery** as the agent type.
4. Select the data source, and add a name and description for the agent.
5. Click **Create** to save.
## Run an agent
1. Go to the **Agents** tab and click the agent you created.
2. Select the tables you want the agent to run on — you can select all tables or choose specific ones.
3. Click **Run**. A chat window opens where you can ask questions and review results.
4. You can create multiple runs for the same agent, each with a different table selection.
## Output
The agent returns:
* A natural language explanation
* A SQL query (when applicable), formatted for your registered database dialect
* References to the specific tables and columns it used
## Limitations
* The agent queries schema metadata and embeddings — it does not execute queries against live data unless you explicitly request SQL execution.
* Results are based on the schema information registered at the time of the last sync.
# Data Validation Agent
Source: https://ploutosaitechnologiespvtltd.mintlify.app/agents/data-validation
Describe what you want to validate in plain language and the agent configures the validation for you.
The Data Validation agent helps you set up validations without manually navigating configuration options. Describe what you want to check — the agent identifies the target dataset and validation type, then generates the appropriate validation configuration for your approval.
## What it does
* Identifies the target dataset from your natural language description
* Determines the appropriate validation type
* Generates validation configurations with a description and confidence score
* Waits for your approval before making tests available to run
## Create an agent
1. Go to the **Agents** tab in the left sidebar.
2. Click **Create Agent**.
3. Select **Validation Generation** as the agent type.
4. Select the data source, and add a name and description for the agent.
5. Optionally, add additional instructions to guide the agent's behavior (for example, focus areas or validation preferences).
6. Click **Create** to save.
## Run an agent
1. Go to the **Agents** tab and click the agent you created.
2. Select the table you want the agent to run on.
3. Click **Run**. You can create multiple runs for the same agent, each targeting a different table.
4. Once the run completes, the agent generates a list of validations — each with a description and a confidence score.
5. Review each validation and **approve** the ones you want. Only approved validations become tests that are ready to run.
## Supported validation types
The agent can configure the following validation types:
| Type | Description |
| -------------------- | ----------------------------------------------- |
| **NOT NULL** | Checks that a column contains no null values |
| **UNIQUE** | Checks that all values in a column are distinct |
| **CHECK** | Checks that column values satisfy a condition |
| **FOREIGN KEY** | Checks referential integrity between tables |
| **Custom SQL** | Validates data using a custom SQL assertion |
| **Delta Validation** | Checks the change between two time periods |
For a full list of supported validation types and their configuration options, see [Validation Types](/reference/validation-types).
# Migration Assessment Agent
Source: https://ploutosaitechnologiespvtltd.mintlify.app/agents/migration-assessment
Analyze stored procedures, views, and SQL complexity in your source database and get a structured migration readiness report.
The Migration Assessment agent analyzes your source database to evaluate migration complexity before you begin. It examines stored procedures, views, and translation difficulty, then generates a structured assessment dashboard with an executive summary and detailed findings.
## What it does
* Analyzes stored procedures for migration complexity
* Evaluates views and their dependencies
* Assesses translation difficulty for SQL logic in the source dialect
* Generates an assessment dashboard with an executive summary and detailed findings
* Lets you refine and update the assessment through chat
## Create an agent
1. Go to the **Agents** tab in the left sidebar.
2. Click **Create Agent**.
3. Select **Migration Assessment** as the agent type.
4. Select the source data source, and add a name and description for the agent.
5. Select the target database type (for example, Snowflake, BigQuery, Databricks).
6. Click **Create** to save.
## Run an agent
1. Go to the **Agents** tab and click the agent you created.
2. Click **Run** to start the assessment. The agent analyzes your source database and produces an assessment dashboard.
3. You can create multiple runs and view previous runs at any time.
## Assessment dashboard
When the run completes, the agent produces an assessment dashboard you can use to evaluate migration readiness. The dashboard includes:
| Section | Description |
| ------------------------ | -------------------------------------------------------------------------------- |
| **Executive Summary** | High-level migration readiness score and key findings |
| **Procedure Analysis** | Per-procedure complexity assessment and migration notes |
| **View Analysis** | View dependency mapping and complexity flags |
| **Translation Analysis** | Assessment of SQL constructs that require translation, with complexity estimates |
You can continue the conversation in the chat to ask follow-up questions, request a deeper analysis of specific objects, or update the scope. The agent will modify the assessment dashboard based on your input.
# Agents Overview
Source: https://ploutosaitechnologiespvtltd.mintlify.app/agents/overview
Datachecks uses four AI agents to automate discovery, assessment, validation, and translation across your migration.
Datachecks is built around four AI agents, each responsible for a distinct phase of the data migration lifecycle. The agents are powered by [LangGraph](https://langchain-ai.github.io/langgraph/) workflows and run as background jobs you can monitor in real time.
## The four agents
Ask natural language questions about your data schemas, tables, columns, and relationships. The agent queries your registered assets and generates SQL on demand.
Analyze stored procedures, views, and translation complexity in your source database. Get a structured readiness report before committing to a migration.
Describe what you want to validate in plain language. The agent extracts the target table and validation type and configures the validation for you.
Translate SQL queries from your source database dialect to the target platform. The agent classifies each query, translates it, and verifies correctness by running a comparison.
## How agents fit into the migration workflow
The agents are designed to work sequentially, but you can run them independently:
```
Asset Discovery
↓
Understand what you have
↓
Migration Assessment
↓
Know what will be complex
↓
Translation
↓
Convert SQL logic to target dialect
↓
Data Validation
↓
Confirm data integrity after migration
```
## Where to find agents
Navigate to the **Agents** tab in the left sidebar to access the agent interface. Each agent run is tracked as a job — go to **Jobs** to monitor progress and view results.
# Translation Agent
Source: https://ploutosaitechnologiespvtltd.mintlify.app/agents/translation
Automatically translate SQL queries, views, and stored procedures from your source database dialect to your target platform.
The Translation agent converts SQL from a source database dialect to a target dialect. You organise translations into projects, each scoped to a source and target data source. The agent translates each query, seeds data into the target, and runs a comparison to verify the result is correct.
## What it does
* Translates SQL queries, views, and stored procedures from source to target dialect
* Verifies translations by running the translated SQL against the target database and comparing results
* Automatically detects and fixes translation errors in a feedback loop
* Generates PySpark scripts as an alternative output for stored procedures
## Create a translation project
A translation project groups related translations under a single source and target data source pair.
1. Navigate to **Translation** in the left sidebar.
2. Click **New Project**.
3. Enter a **Project Name**.
4. Select the **Source Datasource** — the database you are migrating from.
5. Select the **Target Datasource** — the database you are migrating to.
6. Click **Create**.
## Add a translation
Once a project is created, add individual SQL objects to translate.
1. Open the translation project.
2. Click **New Translation**.
3. Select the **Translation Type** — SQL Query, View, or Stored Procedure.
4. Paste or enter the source SQL.
5. For stored procedures, choose the **Output Format**:
* **Target SQL Script** — Generates the equivalent stored procedure in the target dialect.
* **PySpark Script** — Generates a PySpark equivalent for platforms such as Databricks.
6. Click **Translate**. The agent runs the translation and verification workflow.
## Supported translation types
| Type | Description |
| -------------------- | ----------------------------------------------------------------------------------------------------------- |
| **SQL Query** | Translates a SQL SELECT or DML statement to the target dialect. |
| **View** | Translates a view definition to the target dialect. |
| **Stored Procedure** | Translates a stored procedure. Choose between a target SQL script or a PySpark script as the output format. |
# Email Notifications
Source: https://ploutosaitechnologiespvtltd.mintlify.app/alerts/channels/email
Configure SMTP email as a notification channel for validation alerts.
Send validation alerts to your team via email by configuring an SMTP notification channel.
## Prerequisites
* Valid SMTP server credentials (server address, port, username, password)
* A sender email address
* One or more recipient email addresses
## Configure the channel
1. Go to **Settings → Your Workspace**.
2. Click **New Notification Channel** and select **Email**.
3. Fill in the SMTP configuration form:
| Field | Description |
| -------------------- | ----------------------------------------------------- |
| **Channel Name** | A label to identify this channel in Datachecks. |
| **Email** | The sender email address used to dispatch alerts. |
| **Password** | Authentication password for the SMTP account. |
| **SMTP Server** | Your SMTP server address (e.g. `smtp.gmail.com`). |
| **Port** | SMTP port — typically `587` for TLS or `465` for SSL. |
| **Recipients Email** | One or more recipient addresses to receive alerts. |
4. Click **Connect** to save and enable the channel.
The channel is now active. You can assign it to alerts from the Alerts configuration page.
# Slack Notifications
Source: https://ploutosaitechnologiespvtltd.mintlify.app/alerts/channels/slack
Configure Slack as a notification channel to send validation alerts to your team.
Send validation alerts directly to Slack channels by connecting a Slack bot to Datachecks.
## Prerequisites
* A Slack Bot Token from the [Slack API platform](https://api.slack.com/apps)
* The name or ID of the Slack channel where alerts should be posted
* Permissions to install apps in your Slack workspace
## Configure the channel
1. Go to **Settings → Your Workspace**.
2. Click **New Notification Channel** and select **Slack**.
3. Fill in the configuration form:
| Field | Description |
| ------------------ | ----------------------------------------------------------------------------------------------------------------- |
| **Channel Name** | A label to identify this channel in Datachecks. |
| **Bot Token** | The Slack Bot Token used to authenticate with the Slack API. |
| **Slack Channels** | The Slack channel name (e.g. `#alerts`) or channel ID where alerts will be posted. You can add multiple channels. |
4. Click **Connect** to save and enable the channel.
The channel is now active. You can assign it to alerts from the Alerts configuration page.
# Microsoft Teams Notifications
Source: https://ploutosaitechnologiespvtltd.mintlify.app/alerts/channels/teams
Configure Microsoft Teams as a notification channel to send validation alerts via webhook.
Send validation alerts to a Microsoft Teams channel using an incoming webhook.
## Prerequisites
* A Microsoft Teams workspace with permission to add incoming webhooks
* An incoming webhook URL for the target Teams channel — see [Microsoft's documentation](https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/add-incoming-webhook) for setup instructions
## Configure the channel
1. Go to **Settings → Your Workspace**.
2. Click **New Notification Channel** and select **Teams**.
3. Fill in the configuration form:
| Field | Description |
| ---------------- | ----------------------------------------------- |
| **Channel Name** | A label to identify this channel in Datachecks. |
| **Webhook URL** | The incoming webhook URL from Microsoft Teams. |
4. Click **Connect** to save and enable the channel.
The channel is now active. You can assign it to alerts from the Alerts configuration page.
# Alert Details
Source: https://ploutosaitechnologiespvtltd.mintlify.app/alerts/details
Inspect the full context of an alert — metrics, anomaly log, and status management.
Click any alert in the list to open its detail view. This page shows the full context of the alert: the validation that triggered it, a chart of the metric over time, and a log of every anomaly detected.
## Summary
The top of the page shows key metadata about the validation that triggered the alert:
| Field | Description |
| ------------------- | ---------------------------------------------------------- |
| **Validation Type** | The category of the check (e.g. Completeness, Uniqueness). |
| **Schedule** | How frequently the validation runs. |
| **Threshold** | The limit that was crossed to trigger the alert. |
| **Sensitivity** | The deviation tolerance before an alert activates. |
| **First Seen** | Timestamp when the alert first triggered. |
| **Last Seen** | Timestamp of the most recent detection. |
## Metrics chart
The chart plots the validation's measured values over the selected time window. It shows:
* **Values** — The metric recorded at each run
* **Upper / lower bounds** — Expected range based on the threshold configuration
* **Severity** — How far each value deviated from the expected range
* **Threshold line** — A dotted reference line indicating the alert boundary
Hover over any data point to see the details for that specific run.
## Anomaly log
The log table records every anomaly detected for this alert:
| Column | Description |
| ------------- | --------------------------------------------------- |
| **Value** | The metric value recorded at the time of detection. |
| **Deviation** | How far the value deviated from the expected range. |
| **Seen At** | Timestamp when the anomaly was detected. |
## Managing alert status
The right-side panel lets you update the alert status and track activity:
* **Mark as Incident** — Flag the alert as an active incident under investigation.
* **Resolve** — Close the alert with one of three resolution types: False Positive, Expected, or No Action Needed.
* **Activity log** — A chronological record of every status change and user comment on the alert.
See [Alert Statuses](/alerts/statuses) for a description of each status, and [Alert Lifecycle](/alerts/lifecycle) for how alerts progress from creation to resolution.
# Alert Lifecycle
Source: https://ploutosaitechnologiespvtltd.mintlify.app/alerts/lifecycle
How alerts move from creation through investigation to resolution in Datachecks.
Every alert in Datachecks follows a defined lifecycle — from the moment a validation crosses a threshold to the point where the issue is resolved or dismissed.
## Lifecycle stages
### 1. Created
When a validation detects a threshold violation, an alert is created automatically with a status of **No Status**. It is flagged as a potential issue awaiting investigation.
### 2. Investigating
A team member opens the alert and begins examining it. The status is updated to **Investigating** to signal that someone is actively looking at the issue and prevent duplicate effort.
### 3. Mark as Incident
If the alert is confirmed as a real problem, it can be escalated by marking it as an **Incident**. This signals that the issue is under active investigation and may require remediation. An alert can be marked as an incident directly from the **No Status** state — the Investigating step is optional.
### 4. Resolve
Once an alert has been assessed, it is closed with one of three resolution statuses:
| Resolution | When to use |
| -------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| **False Positive** | The detection was not a valid anomaly and does not indicate a real problem. |
| **Expected** | The anomaly was valid but caused by a known event — a pipeline change, maintenance window, or deliberate data modification. |
| **No Action Needed** | The anomaly was valid but not significant enough to require any further action. |
## State diagram
```
Created (No Status)
│
├──────────────────────────────────┐
▼ │
Investigating │
│ │
▼ ▼
Mark as Incident ──────────────▶ Resolve
├─ False Positive
├─ Expected
└─ No Action Needed
```
All transitions are recorded in the [alert activity log](/alerts/details#managing-alert-status), giving you a full audit trail of how each alert was handled.
# Overview
Source: https://ploutosaitechnologiespvtltd.mintlify.app/alerts/overview
Monitor and manage alerts triggered by validation threshold violations.
Alerts are created automatically when a validation detects a threshold violation. Navigate to **Alerts** in the left sidebar to view and manage all alerts across your workspace.
## Alert groups
Alerts are organized into groups — each validation maps to a specific alert group. A group displays up to 50 alerts by default. When a validation generates 50 or more alerts, the same group name repeats with new alert instances so no alerts are lost.
## List columns
| Column | Description |
| -------------- | ------------------------------------------------------------- |
| **ID** | Unique identifier for the alert. |
| **Group** | Groups related alerts and shows the current status. |
| **Alerts** | Total number of alerts within the group (up to 50 per group). |
| **First Seen** | Timestamp when the alert was first triggered. |
| **Last Seen** | Timestamp of the most recent alert detection. |
## Filters
| Filter | Description |
| ---------------- | ---------------------------------------------------------------------------------------------------------- |
| **Issue Status** | Filter by the type of issue the alert represents. |
| **Datasources** | Filter alerts by the connected data source. |
| **Alert Status** | Filter by current status — No Status, Fixed, Investigating, Expected, No Action Needed, or False Positive. |
| **Date Range** | Show alerts within a predefined or custom time range. |
A historical chart above the list shows total alert counts for the selected date range.
## Next steps
* [View Alert Details](/alerts/details) — Inspect the metrics graph and anomaly log for an individual alert
* [Alert Statuses](/alerts/statuses) — Understand what each status means
* [Alert Lifecycle](/alerts/lifecycle) — Learn how alerts move from creation to resolution
# Alert Statuses
Source: https://ploutosaitechnologiespvtltd.mintlify.app/alerts/statuses
Reference for all six alert statuses in Datachecks and what each one means.
Every alert has a status that reflects its current state in the investigation and resolution workflow. Keeping statuses up to date helps your team prioritize urgent issues and avoid duplicate work.
## Statuses
| Status | Description |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| **No Status** | The initial state when an alert is first created. No action has been taken yet. |
| **Investigating** | A team member is actively examining whether the alert represents a genuine issue. |
| **Fixed** | The underlying problem has been resolved. No further action is needed. |
| **Expected** | The anomaly is statistically valid but was caused by an anticipated event — such as a pipeline change, scheduled maintenance, or a known data event. |
| **No Action Needed** | The anomaly is statistically valid but not significant enough to warrant investigation or remediation. |
| **False Positive** | The alert does not reflect an actual statistical anomaly or genuine problem and can be dismissed. |
## How to update a status
Open the [alert detail view](/alerts/details) and use the status panel on the right side of the page to mark the alert as an incident or apply a resolution status.
All status changes are recorded in the activity log so your team has a full audit trail.
# Create Comparison
Source: https://ploutosaitechnologiespvtltd.mintlify.app/api-reference/comparisons/create
POST /comparisons
Create a new comparison between a source and target dataset.
Bearer token. Format: `Bearer `
Your workspace UUID.
If `true`, creates the comparison without triggering execution.
Display name for the comparison.
UUID of the source data source.
UUID of the target data source.
UUID or name of the source dataset. Required if `source_query` is not set.
UUID or name of the target dataset. Required if `target_query` is not set.
Comparison configuration.
List of column names to compare from the source dataset.
List of column names to compare from the target dataset.
Primary key columns in the source dataset.
Primary key columns in the target dataset.
SQL WHERE clause to filter source rows.
SQL WHERE clause to filter target rows.
Custom SQL query for the source. Overrides `source_dataset_id`.
Custom SQL query for the target. Overrides `target_dataset_id`.
Whether string comparisons are case-sensitive.
Run a fast row-count-only comparison instead of a full diff.
Include schema differences in the comparison.
## Response
The created comparison.
UUID of the comparison.
Comparison name.
Source dataset UUID.
Target dataset UUID.
The comparison configuration as stored.
Workspace UUID.
UUID of the user who created the comparison.
Status message.
```curl cURL theme={null}
curl -X POST "https://YOUR_BASE_URL/comparisons" \
-H "Authorization: Bearer " \
-H "Workspace-Id: " \
-H "Content-Type: application/json" \
-d '{
"name": "orders table comparison",
"source_datasource_id": "aaaaaaaa-0000-0000-0000-000000000001",
"target_datasource_id": "bbbbbbbb-0000-0000-0000-000000000002",
"source_dataset_id": "cccccccc-0000-0000-0000-000000000003",
"target_dataset_id": "dddddddd-0000-0000-0000-000000000004",
"configuration": {
"source_columns": ["order_id", "customer_id", "total"],
"target_columns": ["order_id", "customer_id", "total"],
"primary_keys_source": ["order_id"],
"primary_keys_target": ["order_id"]
}
}'
```
```python Python theme={null}
import requests
url = "https://YOUR_BASE_URL/comparisons"
headers = {
"Authorization": "Bearer ",
"Workspace-Id": "",
}
payload = {
"name": "orders table comparison",
"source_datasource_id": "aaaaaaaa-0000-0000-0000-000000000001",
"target_datasource_id": "bbbbbbbb-0000-0000-0000-000000000002",
"source_dataset_id": "cccccccc-0000-0000-0000-000000000003",
"target_dataset_id": "dddddddd-0000-0000-0000-000000000004",
"configuration": {
"source_columns": ["order_id", "customer_id", "total"],
"target_columns": ["order_id", "customer_id", "total"],
"primary_keys_source": ["order_id"],
"primary_keys_target": ["order_id"],
},
}
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```
```javascript JavaScript theme={null}
const payload = {
name: "orders table comparison",
source_datasource_id: "aaaaaaaa-0000-0000-0000-000000000001",
target_datasource_id: "bbbbbbbb-0000-0000-0000-000000000002",
source_dataset_id: "cccccccc-0000-0000-0000-000000000003",
target_dataset_id: "dddddddd-0000-0000-0000-000000000004",
configuration: {
source_columns: ["order_id", "customer_id", "total"],
target_columns: ["order_id", "customer_id", "total"],
primary_keys_source: ["order_id"],
primary_keys_target: ["order_id"],
},
};
const response = await fetch("https://YOUR_BASE_URL/comparisons", {
method: "POST",
headers: {
"Authorization": "Bearer ",
"Workspace-Id": "",
"Content-Type": "application/json",
},
body: JSON.stringify(payload),
});
const data = await response.json();
console.log(data);
```
```json 201 theme={null}
{
"message": "Comparison created successfully",
"data": {
"id": "eeeeeeee-0000-0000-0000-000000000005",
"name": "orders table comparison",
"source_dataset_id": "cccccccc-0000-0000-0000-000000000003",
"target_dataset_id": "dddddddd-0000-0000-0000-000000000004",
"configuration": {
"source_columns": ["order_id", "customer_id", "total"],
"target_columns": ["order_id", "customer_id", "total"],
"primary_keys_source": ["order_id"],
"primary_keys_target": ["order_id"],
"case_sensitive": false,
"quick_comparison": false
},
"workspace_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"user_id": "ffffffff-0000-0000-0000-000000000006"
}
}
```
# Execute Comparison
Source: https://ploutosaitechnologiespvtltd.mintlify.app/api-reference/comparisons/execute
POST /comparisons/{comparison_id}/execute
Submit a comparison for execution. Returns a job ID you can use to monitor progress.
Bearer token. Format: `Bearer `
Your workspace UUID.
UUID of the comparison to execute.
If provided, the running job with this task ID is terminated before a new execution starts.
If `true`, the job is placed at the front of the execution queue.
Translation iteration number. Used when executing as part of a translation workflow.
Translation run ID. Used when executing as part of a translation workflow.
Event key for tracking the execution in an agent workflow.
## Response
Status message.
The job ID. Use this to monitor progress via the Jobs API.
The background task ID assigned to this execution.
```curl cURL theme={null}
curl -X POST "https://YOUR_BASE_URL/comparisons/eeeeeeee-0000-0000-0000-000000000005/execute" \
-H "Authorization: Bearer " \
-H "Workspace-Id: "
```
```python Python theme={null}
import requests
comparison_id = "eeeeeeee-0000-0000-0000-000000000005"
url = f"https://YOUR_BASE_URL/comparisons/{comparison_id}/execute"
headers = {
"Authorization": "Bearer ",
"Workspace-Id": "",
}
response = requests.post(url, headers=headers)
print(response.json())
```
```javascript JavaScript theme={null}
const comparisonId = "eeeeeeee-0000-0000-0000-000000000005";
const response = await fetch(
`https://YOUR_BASE_URL/comparisons/${comparisonId}/execute`,
{
method: "POST",
headers: {
"Authorization": "Bearer ",
"Workspace-Id": "",
},
}
);
const data = await response.json();
console.log(data);
```
```json 200 theme={null}
{
"message": "Comparison execution started",
"run_id": 1042,
"celery_task_id": "7f3a1b2c-4d5e-6f7a-8b9c-0d1e2f3a4b5c"
}
```
# Execute Multiple Comparisons
Source: https://ploutosaitechnologiespvtltd.mintlify.app/api-reference/comparisons/execute-multiple
POST /comparisons/execute_multiple
Execute a batch of comparisons as part of an agent run.
Bearer token. Format: `Bearer `
Your workspace UUID.
UUID of the agent run this batch belongs to.
List of comparison configurations to execute.
Config ID used to map results back to the original configuration.
Full comparison configuration. See [Create Comparison](/api-reference/comparisons/create) for the `configuration` schema.
## Response
Status message.
Map of config ID to job run ID. Use the run IDs to monitor progress via the Jobs API.
```curl cURL theme={null}
curl -X POST "https://YOUR_BASE_URL/comparisons/execute_multiple" \
-H "Authorization: Bearer " \
-H "Workspace-Id: " \
-H "Content-Type: application/json" \
-d '{
"agent_run_id": "aaaaaaaa-1111-2222-3333-444444444444",
"comparison_agent_configs": [
{
"id": 1,
"configuration": {
"name": "orders comparison",
"source_datasource_id": "aaaaaaaa-0000-0000-0000-000000000001",
"target_datasource_id": "bbbbbbbb-0000-0000-0000-000000000002",
"source_dataset_id": "cccccccc-0000-0000-0000-000000000003",
"target_dataset_id": "dddddddd-0000-0000-0000-000000000004",
"configuration": {
"source_columns": ["order_id", "total"],
"target_columns": ["order_id", "total"],
"primary_keys_source": ["order_id"],
"primary_keys_target": ["order_id"]
}
}
}
]
}'
```
```python Python theme={null}
import requests
url = "https://YOUR_BASE_URL/comparisons/execute_multiple"
headers = {
"Authorization": "Bearer ",
"Workspace-Id": "",
}
payload = {
"agent_run_id": "aaaaaaaa-1111-2222-3333-444444444444",
"comparison_agent_configs": [
{
"id": 1,
"configuration": {
"name": "orders comparison",
"source_datasource_id": "aaaaaaaa-0000-0000-0000-000000000001",
"target_datasource_id": "bbbbbbbb-0000-0000-0000-000000000002",
"source_dataset_id": "cccccccc-0000-0000-0000-000000000003",
"target_dataset_id": "dddddddd-0000-0000-0000-000000000004",
"configuration": {
"source_columns": ["order_id", "total"],
"target_columns": ["order_id", "total"],
"primary_keys_source": ["order_id"],
"primary_keys_target": ["order_id"],
},
},
}
],
}
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```
```javascript JavaScript theme={null}
const payload = {
agent_run_id: "aaaaaaaa-1111-2222-3333-444444444444",
comparison_agent_configs: [
{
id: 1,
configuration: {
name: "orders comparison",
source_datasource_id: "aaaaaaaa-0000-0000-0000-000000000001",
target_datasource_id: "bbbbbbbb-0000-0000-0000-000000000002",
source_dataset_id: "cccccccc-0000-0000-0000-000000000003",
target_dataset_id: "dddddddd-0000-0000-0000-000000000004",
configuration: {
source_columns: ["order_id", "total"],
target_columns: ["order_id", "total"],
primary_keys_source: ["order_id"],
primary_keys_target: ["order_id"],
},
},
},
],
};
const response = await fetch(
"https://YOUR_BASE_URL/comparisons/execute_multiple",
{
method: "POST",
headers: {
"Authorization": "Bearer ",
"Workspace-Id": "",
"Content-Type": "application/json",
},
body: JSON.stringify(payload),
}
);
const data = await response.json();
console.log(data);
```
```json 200 theme={null}
{
"message": "Comparisons submitted for execution",
"mapped_run_ids": {
"1": 1043
}
}
```
# List Data Sources
Source: https://ploutosaitechnologiespvtltd.mintlify.app/api-reference/data-sources/list
GET /sources
List all data sources in the workspace with optional filtering, sorting, and pagination.
Bearer token. Format: `Bearer `
Your workspace UUID.
Search string to filter data sources by name.
Page number for pagination.
Number of results per page.
Field to sort by. Options: `last_scanned`, `name`, `type`.
Sort direction. Options: `asc`, `desc`.
Filter by interactive status. Omit to return all.
Returns additional fields used by the settings view.
## Response
Paginated response object.
List of data source objects.
UUID of the data source.
Name of the data source.
Database type (e.g. `snowflake`, `bigquery`, `postgresql`).
Connection status.
Whether the data source is interactive.
Number of registered datasets.
Total number of validations.
Number of failing validations.
Total number of columns.
Number of sensitive columns detected.
ISO 8601 timestamp of the last scan.
Database engine version.
Workspace UUID this data source belongs to.
Current page number.
Page size.
Total number of results.
Total number of pages.
Status message.
Additional metadata.
```curl cURL theme={null}
curl -X GET "https://YOUR_BASE_URL/sources?page=1&size=20&sort_by=name&sort_order=asc" \
-H "Authorization: Bearer " \
-H "Workspace-Id: "
```
```python Python theme={null}
import requests
url = "https://YOUR_BASE_URL/sources"
headers = {
"Authorization": "Bearer ",
"Workspace-Id": "",
}
params = {
"page": 1,
"size": 20,
"sort_by": "name",
"sort_order": "asc",
}
response = requests.get(url, headers=headers, params=params)
print(response.json())
```
```javascript JavaScript theme={null}
const response = await fetch(
"https://YOUR_BASE_URL/sources?page=1&size=20&sort_by=name&sort_order=asc",
{
method: "GET",
headers: {
"Authorization": "Bearer ",
"Workspace-Id": "",
},
}
);
const data = await response.json();
console.log(data);
```
```json 200 theme={null}
{
"message": "",
"meta": {},
"data": {
"items": [
{
"id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"name": "production-snowflake",
"type": "snowflake",
"status": "active",
"is_interactive": true,
"dataset_count": 42,
"validation_count": 128,
"failed_validation_count": 3,
"column_count": 560,
"sensitive_column_count": 12,
"last_scanned_at": "2025-01-15T10:30:00Z",
"database_version": "8.23.0",
"workspace_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
}
],
"page": 1,
"size": 20,
"total": 1,
"pages": 1
}
}
```
# API Reference
Source: https://ploutosaitechnologiespvtltd.mintlify.app/api-reference/introduction
REST API for programmatic access to Datachecks.
The Datachecks API gives you programmatic access to all operations — comparisons, validations, data sources, jobs, and more.
## Authentication
All API requests require a Bearer token and a workspace ID header:
```http theme={null}
Authorization: Bearer
Workspace-Id:
```
You can create API keys under **Settings → API Keys**.
## Base URL
The base URL depends on your deployment. Replace `YOUR_BASE_URL` in all examples with your instance URL.
```
https://YOUR_BASE_URL
```
## Versioning
The current API version is `v1`. All endpoints are prefixed with `/api/v1/`.
## Scopes
API keys are scoped to control what operations they can perform. Each endpoint requires a specific scope:
| Scope | Description |
| ------------------ | ------------------------------ |
| `read:datasource` | List and read data sources |
| `read:job` | List and read comparison jobs |
| `write:job` | Terminate jobs |
| `write:comparison` | Create and execute comparisons |
## Response format
All responses use a consistent envelope format:
```json theme={null}
{
"message": "...",
"meta": {},
"data": { ... }
}
```
Paginated responses include pagination metadata inside `data`:
```json theme={null}
{
"data": {
"items": [...],
"page": 1,
"size": 50,
"total": 120,
"pages": 3
}
}
```
# Get Job Details
Source: https://ploutosaitechnologiespvtltd.mintlify.app/api-reference/jobs/details
POST /jobs/comparison/details
Retrieve details for a list of comparison jobs by their job IDs.
Bearer token. Format: `Bearer `
Your workspace UUID.
List of integer job IDs to fetch details for.
## Response
List of job detail objects.
Job ID.
UUID of the comparison.
Name of the comparison.
Job status. One of `PENDING`, `STARTED`, `SUCCESS`, `FAILURE`, `REVOKED`.
Reason for failure, if applicable.
ISO 8601 start timestamp.
ISO 8601 completion timestamp.
Comparison result summary. Present only when status is `SUCCESS`.
Overall diff outcome. One of `MATCH`, `MISMATCH`.
List of reasons explaining differences found.
Execution duration in seconds.
Additional metadata.
```curl cURL theme={null}
curl -X POST "https://YOUR_BASE_URL/jobs/comparison/details" \
-H "Authorization: Bearer " \
-H "Workspace-Id: " \
-H "Content-Type: application/json" \
-d '[1042, 1043]'
```
```python Python theme={null}
import requests
url = "https://YOUR_BASE_URL/jobs/comparison/details"
headers = {
"Authorization": "Bearer ",
"Workspace-Id": "",
}
response = requests.post(url, json=[1042, 1043], headers=headers)
print(response.json())
```
```javascript JavaScript theme={null}
const response = await fetch(
"https://YOUR_BASE_URL/jobs/comparison/details",
{
method: "POST",
headers: {
"Authorization": "Bearer ",
"Workspace-Id": "",
"Content-Type": "application/json",
},
body: JSON.stringify([1042, 1043]),
}
);
const data = await response.json();
console.log(data);
```
```json 200 theme={null}
{
"data": [
{
"run_id": 1042,
"comparison_id": "eeeeeeee-0000-0000-0000-000000000005",
"comparison_name": "orders table comparison",
"status": "SUCCESS",
"failure_reason": null,
"started_at": "2025-01-15T10:00:02Z",
"finished_at": "2025-01-15T10:01:45Z",
"result": {
"diff_status": "MISMATCH",
"diff_reasons": ["Row count mismatch", "Column value differences in total"],
"time_taken": 103.2
}
}
],
"meta": {}
}
```
# Get Diff Details
Source: https://ploutosaitechnologiespvtltd.mintlify.app/api-reference/jobs/diff-details
GET /jobs/comparison/{job_id}/diff-details
Get the full row-level diff for a completed comparison job.
Bearer token. Format: `Bearer `
Your workspace UUID.
The job ID of the completed comparison run.
## Response
Full diff details for the comparison job.
List of primary key column names used to match rows.
Map of source column names to target column names.
Per-column summary of mismatches. Keys are column names, values contain mismatch counts and details.
List of rows with differences. Each entry contains the primary key values and the differing column values from source and target.
Additional metadata.
```curl cURL theme={null}
curl -X GET "https://YOUR_BASE_URL/jobs/comparison/1042/diff-details" \
-H "Authorization: Bearer " \
-H "Workspace-Id: "
```
```python Python theme={null}
import requests
job_id = 1042
url = f"https://YOUR_BASE_URL/jobs/comparison/{job_id}/diff-details"
headers = {
"Authorization": "Bearer ",
"Workspace-Id": "",
}
response = requests.get(url, headers=headers)
print(response.json())
```
```javascript JavaScript theme={null}
const jobId = 1042;
const response = await fetch(
`https://YOUR_BASE_URL/jobs/comparison/${jobId}/diff-details`,
{
method: "GET",
headers: {
"Authorization": "Bearer ",
"Workspace-Id": "",
},
}
);
const data = await response.json();
console.log(data);
```
```json 200 theme={null}
{
"data": {
"primary_key_columns": ["order_id"],
"column_mapping": {
"order_id": "order_id",
"customer_id": "customer_id",
"total": "total"
},
"mismatched_columns_mapping": {
"total": {
"mismatch_count": 3,
"details": "Numeric value differences"
}
},
"diff_rows": [
{
"order_id": 10045,
"source_total": 199.99,
"target_total": 200.00
},
{
"order_id": 10078,
"source_total": 49.95,
"target_total": 49.00
}
]
},
"meta": {}
}
```
# List Comparison Jobs
Source: https://ploutosaitechnologiespvtltd.mintlify.app/api-reference/jobs/list
GET /jobs/comparison
List comparison jobs in the workspace with optional status, dataset, and time range filters.
Bearer token. Format: `Bearer `
Your workspace UUID.
Page number for pagination.
Number of results per page.
Search string to filter jobs by comparison name.
Comma-separated list of statuses to include. Example: `SUCCESS,FAILURE`.
Comma-separated list of source dataset names to filter by.
Comma-separated list of target dataset names to filter by.
ISO 8601 start of the time range. Defaults to 30 days ago.
ISO 8601 end of the time range. Defaults to now.
## Response
Paginated list of jobs.
List of comparison job objects.
Job ID.
UUID of the comparison.
Name of the comparison.
UUID of the result record, once available.
Job status. One of `PENDING`, `STARTED`, `SUCCESS`, `FAILURE`, `REVOKED`.
Source data source name.
Target data source name.
Source dataset name.
Target dataset name.
Source database type.
Target database type.
Celery task ID.
ISO 8601 creation timestamp.
ISO 8601 start timestamp.
ISO 8601 completion timestamp.
Execution duration in seconds.
Number of retries.
Status or error message.
Current page.
Page size.
Total number of jobs.
Total pages.
Includes available filter values for `status_filters`, `source_dataset_filters`, and `target_dataset_filters`.
```curl cURL theme={null}
curl -X GET "https://YOUR_BASE_URL/jobs/comparison?page=1&size=20&status_filters=FAILURE" \
-H "Authorization: Bearer " \
-H "Workspace-Id: "
```
```python Python theme={null}
import requests
url = "https://YOUR_BASE_URL/jobs/comparison"
headers = {
"Authorization": "Bearer ",
"Workspace-Id": "",
}
params = {
"page": 1,
"size": 20,
"status_filters": "FAILURE",
}
response = requests.get(url, headers=headers, params=params)
print(response.json())
```
```javascript JavaScript theme={null}
const response = await fetch(
"https://YOUR_BASE_URL/jobs/comparison?page=1&size=20&status_filters=FAILURE",
{
method: "GET",
headers: {
"Authorization": "Bearer ",
"Workspace-Id": "",
},
}
);
const data = await response.json();
console.log(data);
```
```json 200 theme={null}
{
"data": {
"items": [
{
"job_id": 1042,
"comparison_id": "eeeeeeee-0000-0000-0000-000000000005",
"comparison_name": "orders table comparison",
"comparison_result_id": "ffffffff-0000-0000-0000-000000000006",
"status": "SUCCESS",
"source_datasource_name": "prod-postgres",
"target_datasource_name": "prod-snowflake",
"source_dataset_name": "orders",
"target_dataset_name": "orders",
"source_datasource_type": "postgresql",
"target_datasource_type": "snowflake",
"task_id": "7f3a1b2c-4d5e-6f7a-8b9c-0d1e2f3a4b5c",
"created_at": "2025-01-15T10:00:00Z",
"started_at": "2025-01-15T10:00:02Z",
"date_done": "2025-01-15T10:01:45Z",
"time_taken": 103.2,
"retry_count": 0,
"message": ""
}
],
"page": 1,
"size": 20,
"total": 1,
"pages": 1
},
"meta": {
"status_filters": ["SUCCESS", "FAILURE", "PENDING"],
"source_dataset_filters": ["orders"],
"target_dataset_filters": ["orders"]
}
}
```
# Terminate Job
Source: https://ploutosaitechnologiespvtltd.mintlify.app/api-reference/jobs/terminate
POST /jobs/{celery_task_id}/terminate
Terminate a running comparison job by its task ID.
Bearer token. Format: `Bearer `
Your workspace UUID.
The Celery task ID of the running job to terminate. Returned by [Execute Comparison](/api-reference/comparisons/execute) as `celery_task_id`.
## Response
Confirmation message indicating the job was terminated.
```curl cURL theme={null}
curl -X POST "https://YOUR_BASE_URL/jobs/7f3a1b2c-4d5e-6f7a-8b9c-0d1e2f3a4b5c/terminate" \
-H "Authorization: Bearer " \
-H "Workspace-Id: "
```
```python Python theme={null}
import requests
task_id = "7f3a1b2c-4d5e-6f7a-8b9c-0d1e2f3a4b5c"
url = f"https://YOUR_BASE_URL/jobs/{task_id}/terminate"
headers = {
"Authorization": "Bearer ",
"Workspace-Id": "",
}
response = requests.post(url, headers=headers)
print(response.json())
```
```javascript JavaScript theme={null}
const taskId = "7f3a1b2c-4d5e-6f7a-8b9c-0d1e2f3a4b5c";
const response = await fetch(
`https://YOUR_BASE_URL/jobs/${taskId}/terminate`,
{
method: "POST",
headers: {
"Authorization": "Bearer ",
"Workspace-Id": "",
},
}
);
const data = await response.json();
console.log(data);
```
```json 200 theme={null}
{
"message": "Job terminated successfully"
}
```
# Dataset Discovery
Source: https://ploutosaitechnologiespvtltd.mintlify.app/assets/dataset-discovery
Control which tables are included or excluded when Datachecks scans a connected data source.
When you connect a data source, Datachecks automatically scans for databases and tables to monitor. Dataset Discovery lets you control exactly which tables are included using inclusion and exclusion patterns.
## Inclusion and exclusion lists
You can configure two lists for each data source:
* **Table Inclusion List** — Only tables matching these patterns are profiled and monitored.
* **Table Exclusion List** — Tables matching these patterns are ignored.
**Inclusion takes precedence.** If a table matches both lists, it is included.
## Pattern matching
Patterns support wildcards for flexible matching:
| Pattern | Matches | Example |
| ---------- | --------------------------------------- | ----------------------------------------------------- |
| `%` | Any sequence of characters | `%user` matches `admin_user`, `active_user` |
| `_` | Exactly one character | `customer__` matches `customer01`, `customerAB` |
| `*` | Zero or more characters | `user*data` matches `user_metadata`, `user_salesdata` |
| `\` | Escape a wildcard to match it literally | `north\_%` matches `north_star`, `north_end` |
| Exact name | Only that specific table | `retailorders` matches only `retailorders` |
## View scanning
When setting up a data source, you can enable or disable **view scanning** to control whether database views are included during discovery and profiling.
# Overview
Source: https://ploutosaitechnologiespvtltd.mintlify.app/assets/overview
A centralized hub for exploring and monitoring all datasets connected to your workspace.
Assets is the central place to explore and monitor all data sources connected to your workspace. It organizes your connected databases into a three-level hierarchy and surfaces data quality metrics, validation results, and schema details in one view.
## Three-level hierarchy
### Datasources
The top level lists every database connected to your workspace. For each datasource you can see:
* Name and connection type
* Status
* Number of datasets and columns
* Validation metrics including failed validation count
You can search across datasources, trigger a rescan to pick up schema changes, and view aggregate quality statistics.
### Datasets
Clicking into a datasource shows its datasets (tables and views). For each dataset you can see:
* Status
* Column count
* Number of validations run
* Domain classification
### Dataset detail
Clicking into a dataset opens the detail view with three tabs: **Overview**, **Validation & Alerts**, and **Configuration**.
**Overview tab**
* **Health score** — a quality score broken down across six dimensions: reliability, uniqueness, completeness, distributions, custom checks, and validity
* **Row trends** — row count over time
* **Data preview** — a sample of rows from the dataset so you can inspect the data directly without leaving Datachecks
**Configuration tab**
Manage dataset-level settings:
* **Description** — add or edit a human-readable description for the dataset
* **Domain** — assign the dataset to a domain to group related datasets across datasources
## Validations and alerts
The **Validation & Alerts** tab on a dataset shows all validations applied to it along with any alerts they have triggered.
**Validations**
All validation rules for the dataset are listed here. You can:
* Filter by status, category, method type, threshold type, and collection
* Enable or disable individual validations
* Adjust thresholds and configuration without leaving the dataset view
**Alerts**
When a validation detects a threshold violation, an alert is raised. The alerts for this dataset are shown alongside the validations that triggered them, so you can see at a glance which checks are failing and what the current alert state is.
## Column masking
Sensitive or PII columns can be masked within the UI. This is useful in shared workspace environments where not all users should see raw column values. Masking is configured through the [Profiling](/assets/profiling) workflow, which identifies sensitive columns automatically.
# Profiling
Source: https://ploutosaitechnologiespvtltd.mintlify.app/assets/profiling
Datachecks profiles your datasets using both statistical and semantic analysis to give you a complete picture of your data before and after migration.
Profiling runs automatically when a dataset is scanned. It combines statistical metrics with AI-powered semantic analysis to characterize your data, identify sensitive columns, and recommend masking strategies.
## Statistical profiling
Statistical profiling collects quantitative metrics about your dataset:
| Metric | Description |
| ------------------- | ----------------------------------------------------------------------------------- |
| **Row count** | Total number of rows in the dataset, tracked over time to detect unexpected changes |
| **Column count** | Number of columns and their data types |
| **Null counts** | Per-column null value counts used to power completeness validations |
| **Distinct values** | Cardinality metrics used to power uniqueness validations |
| **Distributions** | Value distribution metrics surfaced in the health score |
These metrics feed directly into the dataset's health score and row trend charts visible in the [Assets](/assets/overview) overview tab.
## Semantic profiling
Semantic profiling uses AI to analyze a sample of values from each column and determine what kind of data it contains — beyond just the raw data type.
For each column, the profiler produces:
| Field | Description |
| ---------------------------- | ----------------------------------------------------------------------------------- |
| **Data type** | The storage type of the column (string, numeric, date, etc.) |
| **Semantic type (general)** | Broad category — for example, `identifier`, `financial`, `contact`, `location` |
| **Semantic type (specific)** | Precise type — for example, `email`, `phone`, `aadhar`, `pan`, `api_key`, `address` |
| **Sensitivity** | How sensitive the column is — used to flag columns for masking |
| **Confidence** | How confident the model is in its classification |
Semantic profiling examines up to five sample values per column and is powered by an LLM prompt that evaluates naming conventions, value patterns, and domain context together.
## Sensitive column identification
Columns with a high sensitivity rating are automatically flagged as potentially containing PII or confidential data. These columns appear visually marked in the dataset view and are candidates for masking.
Sensitivity is assessed based on the semantic type detected — for example, columns identified as `email`, `phone`, `aadhar`, `pan`, `bank_account`, or `api_key` are treated as sensitive.
## Masking strategies
Once a sensitive column is identified, the profiler recommends a masking strategy. The following strategies are supported:
| Strategy | Description | Example |
| ---------------------------- | ------------------------------------------------------ | ------------------ |
| `keep_last_n` | Mask all but the last N characters | `****9012` |
| `keep_prefix` | Keep the first N characters, mask the rest | `AB*******` |
| `mask_username_in_email` | Mask the email username, preserve the domain | `r***@example.com` |
| `numeric_random_same_length` | Replace digits with random digits, preserve separators | `+91-8234167890` |
| `date_mask_parts` | Mask year, month, or day portions of a date | `****-08-15` |
| `hash_sha256` | Replace with a SHA-256 hash (optionally salted) | `3a5b9f2c...` |
| `tokenize_uuid` | Replace with a random UUID | `f47ac10b-...` |
| `none` | No masking applied | — |
Masking is applied in the UI so that sensitive values are not displayed to users in shared workspaces. The underlying data in your database is not modified.
## When profiling runs
Profiling runs automatically when:
* A dataset is first discovered after connecting a datasource
* You manually trigger a **Rescan** from the Assets view
You can view the profiling results for any column from the dataset detail view.
# Create a Comparison
Source: https://ploutosaitechnologiespvtltd.mintlify.app/comparisons/create
Step-by-step guide to configuring and running a comparison between two datasets.
Navigate to **Comparisons** in the left sidebar and click **New Comparison** to open the creation wizard.
## Step 1 — Select comparison type
| Type | When to use |
| ----------- | --------------------------------------------------------------------------------------------- |
| **Deep** | When you need a complete diff — all differences identified and reported. |
| **Shallow** | When you only need to know if differences exist, not where. Stops at the first mismatch. |
| **Schema** | When you only need to compare structure (columns, types, constraints) without examining data. |
## Step 2 — Name the comparison
Enter a descriptive name to identify this comparison in the list.
## Step 3 — Choose data sources
Select the **source** and **target** data sources from your connected integrations.
## Step 4 — Select comparison method
| Method | Description |
| --------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| **Table** | Compare data directly from a database table. Select a dataset and optionally apply a SQL filter clause (e.g. `product.price > 100`). |
| **Query** | Compare datasets using custom SQL queries. Useful when the comparison scope doesn't map to a single table. |
## Step 5 — Define primary keys
Select the columns that uniquely identify each row in the source and target. Primary keys are used to align records accurately across both datasets.
## Step 6 — Select columns
Choose which columns to include in the comparison. Columns exclusive to one dataset cannot be selected. Enable **Case Sensitive** to treat uppercase and lowercase values as distinct.
## Step 7 — Map columns (optional)
If source and target columns have different names, define the mapping using **Add Column Mapping Pair**. Select the source column and its corresponding target column for each pair.
## Step 8 — Attach rules (optional)
Rules control how specific differences are handled — for example, ignoring whitespace differences or rounding numeric values before comparison. Click **+ Add Rule** to apply a pre-configured rule.
## Step 9 — Configure semantic similarity (Shallow only)
For Shallow comparisons, you can enable semantic similarity to detect near-matches in text columns:
| Setting | Description |
| ----------------------- | ---------------------------------------------------------------------------- |
| **Model** | The text analysis model to use. |
| **Pre-processing** | Functions applied before comparison — lowercasing, punctuation removal, etc. |
| **Similarity function** | The algorithm used to score similarity (e.g. Levenshtein Distance). |
| **Match threshold** | A score between 0 and 1. Values above this are treated as matches. |
## Step 10 — Advanced configuration (optional)
| Parameter | Description |
| ------------------------- | ------------------------------------------------------------------------------------------- |
| **Bisection Threshold** | Minimum record count above which the dataset is split into smaller segments for processing. |
| **Bisection Factor** | The division factor used when breaking down large datasets. |
| **Max Threadpool Size** | Maximum number of parallel threads used during comparison. |
| **Egress Limit** | Maximum number of differing rows before the comparison automatically stops. |
| **Per Column Diff Limit** | Maximum number of differences detected per column. |
| **Timeout Limit** | Maximum allowed duration in minutes for the comparison job. |
## Step 11 — Submit
Click **Submit** to create and start the comparison. You can monitor progress in [Jobs](/reference/jobs).
# Comparison Details
Source: https://ploutosaitechnologiespvtltd.mintlify.app/comparisons/details
Inspect the full results of a comparison run across schema, primary keys, and column values.
Click any row in the Comparisons list to open the detail view. Results are organized across four tabs.
## Overview tab
The Overview tab shows a high-level summary of the comparison run.
**Run metadata**
| Field | Description |
| ----------------------- | ----------------------------------------------------- |
| **Run ID** | Unique identifier for this run. |
| **Status** | Outcome of the comparison. |
| **Diff PKs** | Number of differing primary keys. |
| **Diff Rows** | Number of differing rows. |
| **Duration** | How long the comparison took. |
| **Origin** | Whether the run was triggered from the Web UI or CLI. |
| **Created At** | Timestamp of the run. |
| **Comparison Type** | Deep, Shallow, or Schema. |
| **Semantic Similarity** | Whether semantic similarity detection was enabled. |
**Summary metrics**
| Metric | Description |
| --------------------- | --------------------------------------------- |
| **Rows Comparison** | Source and target row counts side by side. |
| **Schema Comparison** | Number of columns analyzed. |
| **Primary Keys** | Count of exclusive keys in source and target. |
| **Values Comparison** | Number of columns with differing values. |
## Schema tab
The Schema tab shows the structural alignment between the two datasets.
| Column | Description |
| ------------------------ | ---------------------------------------------------------- |
| **Source / Target Name** | Dataset and column names. |
| **Type** | Data type of each column. |
| **Schema Status** | Whether the column schema is identical or has differences. |
| **Values Status** | Percentage of rows with value differences in that column. |
| **Exclusive Values** | Count of values that exist in only one dataset. |
**Filters:** Filter by schema status or values status, or search by column name.
## Primary Keys tab
The Primary Keys tab breaks down key-level differences:
* **Duplicate PKs in Source** — Primary key values that appear more than once in the source.
* **Duplicate PKs in Target** — Primary key values that appear more than once in the target.
* **Rows exclusive to Source** — Records that exist only in the source dataset.
* **Rows exclusive to Target** — Records that exist only in the target dataset.
Download results as Excel using the download button.
## Values tab
The Values tab highlights column-level value differences between the two datasets.
* Each row shows a column with the percentage and count of differing values.
* Toggle **Semantic Similarity** to surface near-matches alongside exact differences.
* The diff table uses color coding: **red** for significant differences, **yellow** for partial similarity, and no color for identical values.
* Hover over a cell to see the similarity score.
* Use **Columns toggle** to show only changed columns and their primary keys.
Download results as Excel using the download button.
## Actions
Click the **⋯** menu in the top right to:
| Action | Description |
| ---------- | ------------------------------------------------------------------------------------- |
| **Info** | View the comparison configuration and column mappings. |
| **Stats** | View performance metrics — query timings, min/max/average, P90 for source and target. |
| **Report** | Download a detailed comparison report. |
| **Rerun** | Execute the same comparison again. |
| **Clone** | Duplicate the comparison configuration to create a similar run. |
| **Delete** | Permanently remove the comparison. |
# View Comparisons
Source: https://ploutosaitechnologiespvtltd.mintlify.app/comparisons/overview
View, filter, and manage all dataset comparisons in your workspace.
The Comparisons page lists every comparison run in your workspace. Navigate to **Comparisons** in the left sidebar to access it.
## Comparison types
| Type | Description |
| ----------- | ------------------------------------------------------------------------------------------------ |
| **Deep** | Scans both datasets fully and identifies all differences. Generates a detailed diff report. |
| **Shallow** | Stops after the first value difference found. Faster than Deep for quick checks. |
| **Schema** | Compares structural elements only — column names, data types, and constraints. Ignores row data. |
## Table columns
| Column | Description |
| --------------------------- | -------------------------------------------------------------------------- |
| **Run ID** | Unique identifier for the comparison execution. |
| **Comparison Name** | Name assigned to the comparison, plus its type (Deep, Shallow, or Schema). |
| **Source & Target Dataset** | The two datasets being compared. |
| **Results** | Whether the datasets are Identical or Different. |
| **Diff PKs** | Number of primary keys that differ between source and target. |
| **Diff Rows** | Number of rows that differ between source and target. |
| **Origin** | Where the comparison was triggered — Web UI or CLI. |
| **Created At** | Timestamp of when the comparison was created. |
## Filters
| Filter | Options |
| --------------------------- | ---------------------------------------------------------------- |
| **Results** | Identical, Different |
| **Origin** | Web, CLI |
| **Comparison Type** | Deep, Shallow, Schema |
| **Source / Target Dataset** | Select a specific dataset pair |
| **Diff PKs / Rows %** | Filter by percentage threshold |
| **Search** | Search by name or Run ID |
| **Date range** | Today, Yesterday, Last Week, Last Month, Last Quarter, or custom |
## Actions
* **View details** — Click any row to open the full comparison result with schema, primary key, and value diff tabs.
* **Generate report** — Select one or more runs via checkbox, then click **Report** to export a breakdown of schema diffs, primary key exclusives, duplicates, value differences, and row counts.
* **Delete** — Select a run and click **Delete Run** to permanently remove it.
## Create a comparison
Click **New Comparison** to set up a new run. See [Create a Comparison](/comparisons/create) for a step-by-step guide.
# Alerts Dashboard
Source: https://ploutosaitechnologiespvtltd.mintlify.app/dashboards/alerts
Monitor incidents, alert volume, and resolution rates across your workspace.
The Alerts Dashboard tracks validation incidents — when they occur, how many are active, and how quickly they're being resolved. Navigate to **Dashboards → Alerts** in the left sidebar.
Use the **date range picker** to set the time window (default: last 20 days). A header line shows the total alert count for the selected range.
## Summary metrics
Four KPI cards sit at the top of the dashboard:
| Metric | Description |
| ---------------------- | ------------------------------------------------------------------ |
| **Incident Issued** | Total number of incidents created in the selected period. |
| **Incident Resolved** | Total number of incidents marked as resolved. |
| **Alerts** | Current alert count with percentage change vs the previous period. |
| **Status Update Rate** | Percentage of alerts that have had their status updated. |
## Charts
**Incident Issued (timeseries)** — Bar chart showing the number of incidents created per day over the selected period. Hover to see the exact count for each date.
**Alerts trend** — Area chart with a trend line showing alert volume over time, including the current count and percentage change.
**Alerts from Custom SQL** — Bar chart showing alerts triggered specifically by Custom SQL validations.
**Status Update Rate** — Percentage metric showing how actively alert statuses are being managed.
## Filters
| Filter | Description |
| ---------------- | -------------------------------------------------------------- |
| **Alert Status** | Filter by status — Active, Resolved, or Suppressed |
| **Category** | Filter by validation category (Completeness, Uniqueness, etc.) |
| **Datasources** | Filter to one or more connected data sources |
| **Date Range** | Set the time window for all metrics |
## Alert groups table
Below the charts, alerts are grouped by alert group. Each row shows:
| Column | Description |
| ---------------- | ------------------------------------------------------------------------------- |
| **ID** | Unique alert group identifier. |
| **Group** | Alert group name with status chip and datasource → dataset → column breadcrumb. |
| **Alerts Count** | Number of individual alerts in the group. |
| **First Seen** | When the first alert in the group was triggered. |
| **Last Seen** | When the most recent alert in the group was triggered. |
| **Status** | Current status — Active, Resolved, or Suppressed. |
Click any row to view the full alert group detail.
# Comparisons Dashboard
Source: https://ploutosaitechnologiespvtltd.mintlify.app/dashboards/comparisons
Monitor comparison job performance, results distribution, and dataset diff metrics over time.
The Comparisons Dashboard gives you an at-a-glance view of how your comparison runs are performing. Navigate to **Dashboards → Comparisons** in the left sidebar.
Use the **date range picker** in the top-right to set the time window (default: last 30 days). All metrics and charts update to reflect the selected range.
## Summary cards
Two metric cards sit at the top of the dashboard, each showing the current value and a trend indicator compared to the previous period.
**Job Status**
| Metric | Description |
| ------------------- | ------------------------------------------------------ |
| **Successful Jobs** | Count of comparison runs that completed without error. |
| **Failed Jobs** | Count of comparison runs that errored out. |
| **Avg Run Time** | Average duration of a comparison run, in seconds. |
| **Total Jobs** | Total comparison executions in the selected period. |
**Performance Statistics**
| Metric | Description |
| ------------------------- | ------------------------------------------------------------- |
| **Avg Diff PK (%)** | Average percentage of differing primary keys across all runs. |
| **Avg Diff Rows (%)** | Average percentage of differing rows across all runs. |
| **Total Rows Downloaded** | Total number of rows read during comparisons. |
## Charts
**Comparison Types** — Pie chart breaking down runs by type: Deep, Shallow, and Schema. The total count is shown in the center.
**Comparison Results** — Horizontal bar chart showing how runs resolved:
| Result | Description |
| ------------------ | ---------------------------------------------- |
| **Identical** | Source and target matched completely. |
| **Schema Diff** | Structural differences detected. |
| **Row Count Diff** | Row counts differed between source and target. |
| **Value Diff** | Column value differences found. |
## Filters
Use the filter panel to narrow down which runs are included in the metrics and charts:
| Filter | Description |
| --------------------- | ---------------------------------------------- |
| **Results** | Identical, Different |
| **Origin** | Manual (Web UI), Agent |
| **Comparison Type** | Deep, Shallow, Schema |
| **Source Datasource** | Filter to a specific source connection |
| **Target Datasource** | Filter to a specific target connection |
| **Source Dataset** | One or more source datasets |
| **Target Dataset** | One or more target datasets |
| **Diff PKs range** | Min/max slider for primary key diff percentage |
| **Diff Rows range** | Min/max slider for row diff percentage |
# Validations Dashboard
Source: https://ploutosaitechnologiespvtltd.mintlify.app/dashboards/validations
Track schema coverage, validation health, and execution trends across your datasets.
The Validations Dashboard shows how well your datasets are covered by validations and how healthy those validations are over time. Navigate to **Dashboards → Validations** in the left sidebar.
Use the **date range picker** to adjust the time window (default: last 30 days). Trend charts use a 90-day baseline for context.
## Summary metrics
Two percentage cards appear at the top of the dashboard:
| Metric | Description |
| ------------------- | ------------------------------------------------------------------------------------------------ |
| **Schema Coverage** | Percentage of datasets that have at least one validation executed in the last 90 days. |
| **Health** | Percentage of validations that passed compared to the total number executed in the last 90 days. |
## Charts
**Schema Coverage** — Line chart plotting coverage percentage over time. Rising coverage means more of your datasets are actively being validated.
**Health** — Line chart plotting the pass rate of validations over time. A drop in health indicates validations are failing or alerting more frequently.
**Validation Execution** — A combined chart showing:
* A trend line for total validations executed over the selected period
* A stacked bar chart breaking down executions by validation type
You can toggle individual validation types on and off using the filter panel:
| Validation Type | Color |
| -------------------- | ------ |
| Completeness | Purple |
| Custom SQL | Red |
| Numeric Distribution | Cyan |
| Reliability | Blue |
| Uniqueness | Orange |
| Validity | Green |
## Filters
| Filter | Description |
| ------------------- | ----------------------------------------------------- |
| **Datasource** | Narrow all charts to a single connected data source |
| **Validation Type** | Toggle individual types on/off in the Execution chart |
| **Date Range** | Set the time window for all metrics |
# Azure SQL
Source: https://ploutosaitechnologiespvtltd.mintlify.app/data-sources/azure-sql
Connect Azure SQL as a data source in Datachecks.
To add Azure SQL as a data source, go to **Settings → Workspace → Integrations → Datasources** and click **New Datasource**. Select **Azure SQL** and fill in the connection parameters below, then click **Test & Save Datasource**.
## Connection parameters
| Field | Description |
| ------------------- | ------------------------------------------------------------------------------- |
| **Connection Name** | A label for this connection within Datachecks |
| **Host** | The server name hosting your database |
| **Port** | Default is `1433`. Specify an alternative if your server uses a different port |
| **Username** | The dedicated user for Datachecks |
| **Password** | The password for the Datachecks user |
| **Database** | The name of the Azure SQL database you want to connect to |
| **Schema** | The schema you want to access |
| **ODBC Driver** | The driver required to establish the connection (for example, `ODBC Driver 18`) |
## Setup
Run the following SQL on your Azure SQL instance to create a dedicated user for Datachecks.
```sql theme={null}
USE DatabaseName;
CREATE SCHEMA datachecks_tmp;
CREATE LOGIN DatachecksUser WITH PASSWORD = 'YourSecurePassword';
CREATE USER DatachecksUser FOR LOGIN DatachecksUser;
GRANT SELECT ON SCHEMA::YourSchema TO DatachecksUser;
GRANT SELECT, INSERT, UPDATE, DELETE ON SCHEMA::datachecks_tmp TO DatachecksUser;
```
The `datachecks_tmp` schema is used for intermediate processing and keeps Datachecks operations isolated from your production data.
# BigQuery
Source: https://ploutosaitechnologiespvtltd.mintlify.app/data-sources/bigquery
Connect BigQuery as a data source in Datachecks.
To add BigQuery as a data source, go to **Settings → Workspace → Integrations → Datasources** and click **New Datasource**. Select **BigQuery** and fill in the connection parameters below, then click **Test & Save Datasource**.
## Connection parameters
| Field | Description |
| ------------------- | ----------------------------------------------------------------------- |
| **Connection Name** | A label for this connection within Datachecks |
| **Project** | Your BigQuery project ID, found in the URL of your Google Cloud Console |
| **Dataset** | The name of the dataset you want to connect |
| **Key** | The service account key with access to your BigQuery datasets |
## Notes
* The service account must have sufficient permissions to read the target dataset.
* The project ID can be found in your Google Cloud Console URL.
# Databricks
Source: https://ploutosaitechnologiespvtltd.mintlify.app/data-sources/databricks
Connect Databricks as a data source in Datachecks.
To add Databricks as a data source, go to **Settings → Workspace → Integrations → Datasources** and click **New Datasource**. Select **Databricks** and fill in the connection parameters below, then click **Test & Save Datasource**.
## Connection parameters
| Field | Description |
| ------------------- | ----------------------------------------------------------------------------- |
| **Connection Name** | A label for this connection within Datachecks |
| **Host** | The server name hosting your Databricks workspace |
| **Port** | Default is `443`. Specify an alternative if your server uses a different port |
| **HTTP Path** | The HTTP path for your Databricks SQL warehouse |
| **Access Token** | The authentication token for secure access |
| **Catalog** | The catalog within Databricks where your database resides |
| **Schema** | The schema you want to access |
# MySQL
Source: https://ploutosaitechnologiespvtltd.mintlify.app/data-sources/mysql
Connect MySQL as a data source in Datachecks.
To add MySQL as a data source, go to **Settings → Workspace → Integrations → Datasources** and click **New Datasource**. Select **MySQL** and fill in the connection parameters below, then click **Test & Save Datasource**.
## Connection parameters
| Field | Description |
| ------------------- | ------------------------------------------------------------------------------ |
| **Connection Name** | A label for this connection within Datachecks |
| **Host** | The server name hosting your database |
| **Port** | Default is `3306`. Specify an alternative if your server uses a different port |
| **Username** | The dedicated user for Datachecks |
| **Password** | The password for the Datachecks user |
| **Database** | The name of the MySQL database you want to connect to |
| **Security** | Authentication methods or encryption settings (for example, SSL/TLS) |
## Setup
Run the following SQL on your MySQL instance to create a dedicated user for Datachecks.
```sql theme={null}
CREATE DATABASE IF NOT EXISTS datachecks_tmp;
CREATE USER 'datachecks_user'@'%' IDENTIFIED BY 'YourSecurePassword';
GRANT SELECT ON `YourSchema`.* TO 'datachecks_user'@'%';
GRANT ALL ON `datachecks_tmp`.* TO 'datachecks_user'@'%';
FLUSH PRIVILEGES;
```
The `datachecks_tmp` database is used for temporary processing and keeps Datachecks operations isolated from your production data.
# Oracle
Source: https://ploutosaitechnologiespvtltd.mintlify.app/data-sources/oracle
Connect Oracle as a data source in Datachecks.
To add Oracle as a data source, go to **Settings → Workspace → Integrations → Datasources** and click **New Datasource**. Select **Oracle** and fill in the connection parameters below, then click **Test & Save Datasource**.
## Connection parameters
| Field | Description |
| ------------------- | ------------------------------------------------------------------------------ |
| **Connection Name** | A label for this connection within Datachecks |
| **Host** | The server name hosting your database |
| **Port** | Default is `1521`. Specify an alternative if your server uses a different port |
| **Username** | The dedicated user for Datachecks |
| **Password** | The password for the Datachecks user |
| **Service Name** | The database or service name to connect to |
## Setup
Run the following SQL to create a dedicated user with the required permissions.
```sql theme={null}
ALTER SESSION SET CONTAINER = YOURCONTAINER;
CREATE USER DATACHECKS IDENTIFIED BY YourSecurePassword;
GRANT CREATE SESSION TO DATACHECKS;
GRANT CREATE TABLE TO DATACHECKS;
GRANT SELECT ON "YOURSCHEMA"."YOURTABLE" TO DATACHECKS;
GRANT EXECUTE ON SYS.DBMS_CRYPTO TO DATACHECKS;
GRANT UNLIMITED TABLESPACE TO DATACHECKS;
COMMIT;
```
Datachecks uses a temporary schema within the DATACHECKS user for intermediate processing, keeping your production data unchanged.
# Overview
Source: https://ploutosaitechnologiespvtltd.mintlify.app/data-sources/overview
Connect your source and target databases to Datachecks.
Datachecks connects to your databases as data sources. To add a new data source, go to **Settings → Workspace → Integrations → Datasources** and click **New Datasource**.
## Supported databases
# PostgreSQL
Source: https://ploutosaitechnologiespvtltd.mintlify.app/data-sources/postgresql
Connect PostgreSQL as a data source in Datachecks.
To add PostgreSQL as a data source, go to **Settings → Workspace → Integrations → Datasources** and click **New Datasource**. Select **PostgreSQL** and fill in the connection parameters below, then click **Test & Save Datasource**.
## Connection parameters
| Field | Description |
| ------------------- | ------------------------------------------------------------------------------ |
| **Connection Name** | A label for this connection within Datachecks |
| **Host** | The server name hosting your database |
| **Port** | Default is `5432`. Specify an alternative if your server uses a different port |
| **Username** | The dedicated user for Datachecks |
| **Password** | The password for the Datachecks user |
| **Database** | The name of the PostgreSQL database you want to connect to |
| **Schema** | Optional — specify to limit access to a single schema |
## Setup
Run the following SQL on your PostgreSQL instance to create a dedicated user for Datachecks.
```sql theme={null}
CREATE SCHEMA datachecks_tmp;
CREATE ROLE datachecks WITH LOGIN ENCRYPTED PASSWORD 'YourSecurePassword';
GRANT ALL ON SCHEMA datachecks_tmp TO datachecks;
GRANT USAGE ON SCHEMA TO datachecks;
GRANT SELECT ON ALL TABLES IN SCHEMA TO datachecks;
```
The `datachecks_tmp` schema is used for temporary processing and keeps Datachecks operations isolated from your production data.
# Snowflake
Source: https://ploutosaitechnologiespvtltd.mintlify.app/data-sources/snowflake
Connect Snowflake as a data source in Datachecks.
To add Snowflake as a data source, go to **Settings → Workspace → Integrations → Datasources** and click **New Datasource**. Select **Snowflake** and fill in the connection parameters below, then click **Test & Save Datasource**.
## Connection parameters
| Field | Description |
| ---------------------- | -------------------------------------------------------- |
| **Connection Name** | A label for this connection within Datachecks |
| **Account Identifier** | The unique identifier for your Snowflake account |
| **Username** | The dedicated user for Datachecks |
| **Password** | The password for the Datachecks user |
| **Warehouse** | The Snowflake virtual warehouse used for query execution |
| **Database** | The Snowflake database you want to connect to |
| **Role** | The role assigned to the Datachecks user |
| **Schema** | Optional — specify to limit access to a single schema |
## Setup
Run the following SQL in your Snowflake account to create a dedicated role and user for Datachecks.
### 1. Create role and user
```sql theme={null}
CREATE ROLE DATACHECKSROLE;
CREATE USER DATACHECKS
DEFAULT_ROLE = "DATACHECKSROLE"
MUST_CHANGE_PASSWORD = FALSE;
GRANT ROLE DATACHECKSROLE TO USER DATACHECKS;
```
### 2. Set password
```sql theme={null}
ALTER USER DATACHECKS SET PASSWORD = 'YourStrongPassword';
```
### 3. Grant execution monitoring
```sql theme={null}
GRANT MONITOR EXECUTION ON ACCOUNT TO ROLE DATACHECKSROLE;
GRANT IMPORTED PRIVILEGES ON DATABASE SNOWFLAKE TO ROLE DATACHECKSROLE;
```
### 4. Create temporary schema
Datachecks uses this schema for intermediate processing.
```sql theme={null}
CREATE SCHEMA .DATACHECKS_TMP;
GRANT ALL ON SCHEMA .DATACHECKS_TMP TO ROLE DATACHECKSROLE;
```
### 5. Grant data access
Repeat for each database you want to connect:
```sql theme={null}
GRANT USAGE ON WAREHOUSE TO ROLE DATACHECKSROLE;
GRANT USAGE ON DATABASE TO ROLE DATACHECKSROLE;
GRANT USAGE ON ALL SCHEMAS IN DATABASE TO ROLE DATACHECKSROLE;
GRANT SELECT ON ALL TABLES IN DATABASE TO ROLE DATACHECKSROLE;
GRANT SELECT ON ALL VIEWS IN DATABASE TO ROLE DATACHECKSROLE;
```
# Sybase
Source: https://ploutosaitechnologiespvtltd.mintlify.app/data-sources/sybase
Connect Sybase as a data source in Datachecks.
To add Sybase as a data source, go to **Settings → Workspace → Integrations → Datasources** and click **New Datasource**. Select **Sybase** and fill in the connection parameters below, then click **Test & Save Datasource**.
## Connection parameters
| Field | Description |
| ------------------- | ------------------------------------------------------------------------------ |
| **Connection Name** | A label for this connection within Datachecks |
| **Host** | The server name hosting your database |
| **Port** | Default is `1433`. Specify an alternative if your server uses a different port |
| **Username** | The dedicated user for Datachecks |
| **Password** | The password for the Datachecks user |
| **Database** | The name of the Sybase database you want to connect to |
| **Schema** | The schema you want to access |
| **ODBC Driver** | The driver required to establish the connection (for example, `Sybase IQ`) |
# Core Concepts
Source: https://ploutosaitechnologiespvtltd.mintlify.app/getting-started/core-concepts
Key terms and concepts used across the Datachecks platform.
## Workspace
A workspace is the top-level organizational unit in Datachecks. All data sources, assets, validations, comparisons, and jobs belong to a workspace. You can create multiple workspaces to separate projects, teams, or environments (for example, staging vs production).
Only admin users can manage workspaces and add data sources to them.
## Data Source
A data source is a database connection registered in your workspace. Datachecks supports Snowflake, Databricks, BigQuery, PostgreSQL, MySQL, Oracle, Azure SQL, and Sybase.
You connect a data source once under Settings and then reference it across validations, comparisons, and agent runs.
## Asset
An asset represents a dataset (table or view) within a data source. When you add a data source to a workspace, Datachecks discovers and registers its datasets as assets. Each asset tracks schema, column metadata, and data profile information.
Assets are the primary unit that agents operate on.
## Validation
A validation is a rule applied to an asset to check data quality or integrity. Examples include null checks, uniqueness checks, range checks, and custom SQL assertions.
Validations run on a schedule or on demand. When a validation detects a threshold violation, it triggers an alert.
## Comparison
A comparison checks whether data in a source dataset matches a target dataset. Comparisons are used during migration to verify that data has been moved accurately. You can compare row counts, column values, and schema structure.
A comparison produces a job that runs asynchronously and returns detailed results including differential percentages and row-level diffs.
## Structural Comparison
A structural comparison checks the schema of a source dataset against a target dataset — column names, data types, nullability, and primary keys — without examining the actual data values.
## Collection
A collection is a user-defined group of validations. You use collections to organize related validations together for easier monitoring and bulk operations.
## Job
A job is an asynchronous background task. Running a comparison, an agent analysis, or a batch validation all create jobs. You can monitor job status, view progress, and terminate running jobs from the Jobs view.
## Agent
An agent is an AI-powered workflow that automates a specific migration task. Datachecks has four agents:
* **Asset Discovery** — Natural language queries about your data schema and assets
* **Migration Assessment** — Analysis of migration complexity and readiness
* **Data Validation** — AI-assisted validation configuration
* **Translation** — SQL query translation between database dialects
See the [Agents](/agents/overview) section for full details.
## API Key
An API key authenticates programmatic access to the Datachecks API. You can create and manage API keys under Settings. API keys are scoped to a workspace.
# Quickstart
Source: https://ploutosaitechnologiespvtltd.mintlify.app/getting-started/quickstart
Connect a data source and use the Asset Discovery agent to explore your data in minutes.
## Prerequisites
* Admin access to your Datachecks workspace
* Credentials for at least one supported database
***
## Step 1: Add a data source
Go to **Settings → Workspace → Integrations → Datasources** and click **New Datasource**.
Enter your database credentials and save. Datachecks supports Snowflake, Databricks, BigQuery, PostgreSQL, MySQL, Oracle, Azure SQL, and Sybase.
See [Data Sources](/data-sources/overview) for connection details by database type.
## Step 2: Create an Asset Discovery agent
1. Go to the **Agents** tab in the left sidebar.
2. Click **Create Agent**.
3. Select **Asset Discovery** as the agent type.
4. Select the data source you just added, and give the agent a name and description.
5. Click **Create** to save.
## Step 3: Run the agent
1. Click the agent you created.
2. Select the tables you want to explore — you can select all tables or choose specific ones.
3. Click **Run**. A chat window opens.
## Step 4: Explore your tables and data
In the chat window, ask the agent questions about your data:
* *"What tables are in this schema?"*
* *"What columns does the `orders` table have?"*
* *"Which columns might contain PII?"*
* *"Write a query that shows the top 10 customers by order value."*
The agent retrieves schema context and generates SQL queries on demand.
***
## Next steps
Analyze stored procedures and views to assess migration complexity.
Set up data quality checks on your assets.
# Introduction
Source: https://ploutosaitechnologiespvtltd.mintlify.app/index
Datachecks is an AI-powered data migration copilot that helps you migrate to cloud platforms faster and with confidence.
Datachecks accelerates data migrations to Snowflake, Databricks, and BigQuery by automating the three most time-consuming phases: understanding your legacy system, translating SQL and ETL logic, and validating data integrity between source and target.
## How it works
Datachecks is built around four AI agents that guide you through the full migration lifecycle:
Analyze stored procedures, views, and SQL complexity to get a full readiness report before migration begins.
Explore your data assets using natural language. Understand schemas, relationships, and data profiles before you migrate.
Set up and run validations to ensure data accuracy and integrity throughout your migration.
Automatically translate SQL queries from your source database dialect to your target platform.
## Get started
Learn the key terms and concepts used across the platform.
Connect a data source and run your first comparison in minutes.
# API Keys
Source: https://ploutosaitechnologiespvtltd.mintlify.app/reference/api-keys
Create and manage API keys for programmatic access to the Datachecks API.
API keys give you secure, programmatic access to Datachecks. Use them to integrate with CI/CD pipelines, data orchestration workflows, or any external tooling that calls the Datachecks API.
## Create an API key
1. Go to **Settings → Tokens**.
2. Click **Generate New Token**.
3. Enter a descriptive name (e.g. `prod_pipeline_key`).
4. Optionally set an **expiry date** and an **access scope**.
5. Click **Create** and copy the generated key immediately — it will not be shown again.
## Use an API key
Include the key as a Bearer token in the `Authorization` header of every request:
```bash theme={null}
curl -X GET https://YOUR_BASE_URL/sources \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Workspace-Id: YOUR_WORKSPACE_ID"
```
## Manage keys
From **Settings → Tokens** you can:
* **Disable** a key to immediately invalidate it without deleting it
* **Enable** a previously disabled key to reactivate it
* **Delete** a key to permanently revoke access
## Security best practices
* Create separate keys for development, staging, and production environments
* Set expiration dates on keys used for temporary access
* Rotate keys periodically
* Revoke any key that is exposed or no longer needed
* Store keys in a secrets manager (e.g. AWS Secrets Manager, HashiCorp Vault) rather than in source code
# Audit Logs
Source: https://ploutosaitechnologiespvtltd.mintlify.app/reference/audit-logs
Track all user activity in your workspace for security, compliance, and troubleshooting.
Audit Logs provide a chronological record of every action taken in your workspace. Administrators can use them to monitor user behavior, investigate incidents, and meet compliance requirements.
Navigate to **Settings → Audit Logs** to access the log viewer.
## Log table columns
| Column | Description |
| ----------------- | ------------------------------------------------------------------------------------ |
| **Activity** | The type of action performed (e.g. datasource created, validation run). |
| **Status** | Outcome of the action — `Success` or `Error`. |
| **User** | Username and email of the person who performed the action. |
| **User Agent** | The operating system or platform the action originated from (Windows, macOS, Linux). |
| **IP Address** | The IP address the request came from. |
| **Workspace** | The workspace where the activity occurred. |
| **Created At** | Timestamp of the action. |
| **Response Time** | How long the operation took to complete, in milliseconds. |
## Filters
Narrow down the log using any combination of the following filters:
| Filter | Options |
| ----------------- | ----------------------------------------------------------------------------- |
| **Status** | `Success`, `Error` |
| **Activity Type** | Select a specific action type |
| **HTTP Method** | `GET`, `POST`, `PUT`, `DELETE`, `PATCH` |
| **Workspace** | Filter to a specific workspace (in multi-workspace environments) |
| **User** | Filter by a specific user, showing their role and active/disabled status |
| **Time Range** | Today, Yesterday, Last Week, Last Month, Last Quarter, or a custom date range |
Click **Reset** to clear all active filters.
## Export
Click the **Download** button in the top-right corner to export the currently visible audit logs for offline analysis.
# Error Codes
Source: https://ploutosaitechnologiespvtltd.mintlify.app/reference/error-codes
Reference for all error codes returned by the Datachecks platform.
Error codes follow the format `DCS---`. Each code is accompanied by a short name and a human-readable message.
## SDK user errors
Errors caused by invalid configuration or input provided to the SDK.
| Code | Name | Message |
| ------------------ | ---------------------------- | ----------------------------------------------------- |
| `DCS-SDK-USER-001` | `SDK_SOURCE_TARGET_MISSING` | Source or target columns are missing. |
| `DCS-SDK-USER-002` | `SDK_UNSUPPORTED_PK` | Unsupported primary key. |
| `DCS-SDK-USER-003` | `SDK_COLUMN_NOT_FOUND` | Column not found in source or target. |
| `DCS-SDK-USER-004` | `SDK_INVALID_FILTER` | Invalid filter query provided. |
| `DCS-SDK-USER-005` | `SDK_INVALID_TRANSFORMATION` | Invalid transformation rule. |
| `DCS-SDK-USER-006` | `SDK_TRIM_INT` | Cannot apply trim on integer columns. |
| `DCS-SDK-USER-007` | `DB_CREDENTIAL_ERROR` | Authentication failed. Please check your credentials. |
## SDK extraction errors
Errors encountered when the SDK attempts to connect to or read from a database.
| Code | Name | Message |
| ------------------ | ----------------------- | ------------------------------------------------- |
| `DCS-SDK-EXTR-003` | `DB_HOST_UNREACHABLE` | Database host could not be reached. |
| `DCS-SDK-EXTR-004` | `DB_CONNECTION_TIMEOUT` | Connection to the database timed out. |
| `DCS-SDK-EXTR-005` | `DB_DRIVER_ERROR` | Database driver configuration issue. |
| `DCS-SDK-EXTR-006` | `DB_NOT_FOUND` | Target database does not exist. |
| `DCS-SDK-EXTR-007` | `DB_SCHEMA_NOT_FOUND` | The specified schema could not be found. |
| `DCS-SDK-EXTR-008` | `DB_PERMISSION_DENIED` | You don't have permission to access the resource. |
## Job errors
Errors that occur during job creation or execution.
| Code | Name | Message |
| ------------------ | ---------------------- | ----------------------------------------------- |
| `DCS-JOB-COMM-001` | `JOB_HTTP_ERROR` | Failed to create job due to connection error. |
| `DCS-JOB-EXTR-001` | `JOB_DATASOURCE_ERROR` | Datasource connection error while creating job. |
## Datasource errors
Errors related to datasource connectivity and access.
| Code | Name | Message |
| ------------------ | ----------------------- | ---------------------------------------- |
| `DCS-SBR-USER-001` | `DS_CREDENTIAL_ERROR` | Invalid datasource credentials. |
| `DCS-SBR-USER-002` | `DS_PERMISSION_ERROR` | Insufficient permissions for datasource. |
| `DCS-SBR-EXTR-001` | `DS_DB_NOT_FOUND` | Datasource database not found. |
| `DCS-SBR-EXTR-002` | `DS_SCHEMA_NOT_FOUND` | Datasource schema not found. |
| `DCS-SBR-EXTR-003` | `DS_HOST_RESOLVE_ERROR` | Datasource host could not be resolved. |
## Generic errors
| Code | Name | Message |
| ------------------ | --------------- | -------------------------- |
| `DCS-COM-COMM-000` | `UNKNOWN_ERROR` | An unknown error occurred. |
# Data Generators
Source: https://ploutosaitechnologiespvtltd.mintlify.app/reference/functions
Reference for all synthetic data generation functions available in Datachecks test data.
Datachecks provides a library of functions for generating realistic synthetic data. Use these functions when configuring test data generation for your datasets.
## Personal
| Function | Description | Example output |
| -------------- | ------------------------------------------------------------------ | ------------------ |
| `first_name` | Given name | `Emily`, `Brandon` |
| `last_name` | Family name | `Smith`, `Johnson` |
| `name` | Full name | `Emily Johnson` |
| `prefix` | Name title | `Mr.`, `Dr.` |
| `suffix` | Name suffix | `Jr.`, `Ph.D.` |
| `gender` | Gender value | `Male`, `Female` |
| `birth_date` | Date of birth. Accepts `minimum_age` and `maximum_age` parameters. | `1990-04-12` |
| `age` | Numeric age | `34` |
| `phone_number` | Phone number. Accepts optional `country_code` parameter. | `+1-555-867-5309` |
| `pan_id` | Permanent Account Number (India) | `ABCDE1234F` |
| `aadhaar_id` | Aadhaar identity number (India) | `2345 6789 0123` |
| `uae_id` | United Arab Emirates identity number | — |
| `sbn9` | 9-digit unique identifier | `123456789` |
## Address
| Function | Description | Example output |
| ----------------- | -------------------------------- | ------------------------------ |
| `address` | Full address | `123 Main St, Springfield, US` |
| `building_number` | Building number | `42` |
| `street_address` | Street-level address | `742 Evergreen Terrace` |
| `city` | City name | `Springfield` |
| `state` | State or province | `California` |
| `postcode` | ZIP or postal code | `90210` |
| `country` | Country name | `United States` |
| `country_code` | ISO country code | `US`, `CA` |
| `latitude` | Latitude coordinate | `37.7749` |
| `longitude` | Longitude coordinate | `-122.4194` |
| `geo_coordinate` | Combined latitude/longitude pair | `37.7749, -122.4194` |
## Currency
| Function | Description | Example output |
| --------------------- | ------------------ | ---------------------- |
| `currency_code` | ISO currency code | `USD`, `EUR` |
| `currency_name` | Full currency name | `United States Dollar` |
| `currency_symbol` | Currency symbol | `$`, `€` |
| `cryptocurrency_code` | Crypto asset code | `BTC`, `ETH` |
| `cryptocurrency_name` | Full crypto name | `Bitcoin`, `Ethereum` |
## Finance
| Function | Description | Example output |
| ---------------------- | --------------------------------- | ------------------------ |
| `iban` | International Bank Account Number | `GB29NWBK60161331926819` |
| `credit_card_number` | Payment card number | `4111111111111111` |
| `credit_card_expire` | Card expiration date | `12/25` |
| `credit_card_provider` | Card brand | `Visa`, `MasterCard` |
| `price` | Currency-formatted amount | `$12.34`, `€56.78` |
## Company
| Function | Description | Example output |
| ---------------- | ------------------------------ | ---------------------------------- |
| `company` | Organisation name | `Acme Corporation` |
| `company_suffix` | Legal designator | `Inc.`, `LLC` |
| `catch_phrase` | Marketing phrase | `Seamless end-to-end solutions` |
| `bs` | Business buzzwords | `synergize scalable architectures` |
| `ein` | Employer Identification Number | `12-3456789` |
| `job_title` | Professional role | `Software Engineer` |
## Time
| Function | Description | Example output |
| ---------------------- | --------------------------------- | ------------------------- |
| `date_time` | ISO datetime | `2023-11-25T14:35:20` |
| `iso8601` | ISO 8601 datetime with timezone | `2023-11-25T14:35:20Z` |
| `date_time_this_year` | Datetime within the current year | — |
| `date_time_this_month` | Datetime within the current month | — |
| `year` | Numeric year | `2023` |
| `month` | Month name | `January` |
| `day_of_week` | Weekday name | `Monday` |
| `timezone` | Timezone identifier | `UTC`, `America/New_York` |
## Identifiers
| Function | Description | Example output |
| -------- | ----------------------- | -------------------------------------- |
| `uuid4` | UUID v4 | `550e8400-e29b-41d4-a716-446655440000` |
| `ean` | European Article Number | `4006381333931` |
| `isbn10` | 10-digit ISBN | `0-306-40615-2` |
| `isbn13` | 13-digit ISBN | `978-3-16-148410-0` |
| `swift` | SWIFT/BIC banking code | `NWBKGB22` |
## Internet
| Function | Description | Example output |
| ----------- | -------------- | ---------------------- |
| `url` | Website URL | `https://example.com` |
| `user_name` | Login username | `john_doe` |
| `email` | Email address | `john.doe@example.com` |
# Jobs
Source: https://ploutosaitechnologiespvtltd.mintlify.app/reference/jobs
Monitor and manage comparison and validation job executions across your workspace.
The Jobs page is a centralized activity log for all validation and comparison executions in your workspace. Navigate to **Jobs** in the left sidebar to view run history, inspect errors, and manage in-progress jobs.
## Comparison jobs
Comparison jobs evaluate differences between two datasets — for example, production vs. staging. They are used for pre-deployment checks, regression detection, and cross-environment consistency validation.
### Columns
| Column | Description |
| ------------------- | ---------------------------------------- |
| **Run ID** | Unique identifier for the job run. |
| **Status** | Current state of the job. |
| **Comparison Name** | The name of the comparison that was run. |
| **Source Dataset** | Name of the source dataset. |
| **Target Dataset** | Name of the target dataset. |
| **Time Taken** | Duration of the run. |
| **Created At** | Timestamp when the job was created. |
### Statuses
| Status | Description |
| --------- | ------------------------------------ |
| `PENDING` | Job is queued and waiting to start. |
| `RUNNING` | Job is currently executing. |
| `SUCCESS` | Job completed successfully. |
| `FAILURE` | Job encountered an error. |
| `REVOKED` | Job was cancelled before completion. |
### Actions
Click the **⋯** menu on any comparison job row to:
* **Info** — View a summary of the comparison including metadata and execution time.
* **Stats** — View query performance metrics: total time, min/max/average latency, and P90 for source and target.
* **Clone** — Duplicate the job configuration to create a similar run.
* **Revoke** — Cancel a job that is pending or currently running.
***
## Validation jobs
Validation jobs run data quality checks within a single dataset. Each row captures the outcome of a group of validation checks.
### Columns
| Column | Description |
| ---------------------- | -------------------------------------------- |
| **Run ID** | Unique identifier for the job group. |
| **No. of Validations** | Total number of checks performed in the run. |
| **Status** | Current state of the job. |
| **Time Taken** | Duration of the run. |
| **Started At** | Timestamp when the job started executing. |
| **Created At** | Timestamp when the job was triggered. |
### Statuses
| Status | Description |
| --------- | --------------------------------------- |
| `PENDING` | Job is queued and waiting to start. |
| `SUCCESS` | All validations completed successfully. |
| `FAILURE` | One or more validations failed. |
| `REVOKED` | Job was cancelled before completion. |
### Actions
Click the **⋯** menu on any validation job row to **Revoke** a pending or running job.
***
## Filters
Both comparison and validation job lists support the following filters:
* **Status** — Filter by `PENDING`, `RUNNING`, `SUCCESS`, `FAILURE`, or `REVOKED`
* **Time range** — Choose from Today, Yesterday, Last Week, Last Month, Last Quarter, or a custom date range
Click **Reset** to clear all filters.
## Error details
When a job fails, click the expand icon on the row to view the full error message and failure details.
# Masking
Source: https://ploutosaitechnologiespvtltd.mintlify.app/reference/masking
Hide sensitive column values in the Datachecks UI without modifying your underlying data.
Masking lets you conceal sensitive or personally identifiable information in the Datachecks interface. Masked values are hidden in report previews, dashboards, and comparison results — but the underlying data in your database is never modified or deleted.
Masking operates at the presentation layer only. Metadata-level monitoring (row counts, null rates, validation metrics) continues to work normally on masked columns.
## When to use masking
Masking is useful in collaborative environments where visibility must be restricted for compliance, auditing, or data governance purposes. Common use cases include:
* Customer names and contact details
* Account numbers
* Financial figures
* Any column containing confidential or regulated information
## How it works
When a column is masked:
1. The column's values are hidden in the Datachecks UI — report previews and dashboard views show masked placeholders instead of actual values.
2. Validations and comparisons still run against the column using the real data in your database.
3. No data is written back to the source — masking is a display control, not a data transformation.
# Thresholds
Source: https://ploutosaitechnologiespvtltd.mintlify.app/reference/thresholds
Configure auto or constant thresholds to trigger alerts when a validation metric crosses a boundary.
Thresholds define when a validation result becomes an incident. Datachecks supports two types: **Auto** and **Constant**.
| | Auto | Constant |
| -------------------- | -------------------------------------- | ----------------------------------------- |
| **Setup** | Automatic | Manual |
| **Adapts over time** | Yes | No |
| **Best for** | Dynamic data with natural fluctuations | Fixed compliance or operational standards |
## Auto thresholds
Auto thresholds use a machine-learning engine to establish baselines from your historical data. The system analyzes patterns — including seasonality and trends — and continuously updates its expected ranges as new data arrives.
**How it works:**
1. Datachecks runs preliminary statistical tests to understand the shape of your data series.
2. It learns the expected range based on historical values and the parameters you define.
3. Bounds are periodically refreshed to stay accurate as your data evolves.
4. When a new metric value falls outside the learned bounds, an incident is created.
Auto thresholds require no manual configuration and adapt automatically to changes in your data over time.
## Constant thresholds
Constant thresholds set a fixed boundary using a comparison operator and a value. An incident is triggered whenever the metric crosses that boundary.
**Syntax**
```
threshold: " "
```
**Supported operators**
| Operator | Meaning |
| -------- | ------------------------ |
| `>` | Greater than |
| `>=` | Greater than or equal to |
| `<` | Less than |
| `<=` | Less than or equal to |
| `=` | Equal to |
**Examples**
```yaml theme={null}
# Alert when row count exceeds 30
threshold: "> 30"
# Alert when null percentage is 5% or more
threshold: ">= 5"
# Alert when freshness exceeds 24 hours (86400 seconds)
threshold: "> 86400"
# Alert when any duplicates exist
threshold: "> 0"
# Alert when row count between two sources differs
threshold: "< 1"
```
Constant thresholds are well-suited to metrics with predictable, well-understood boundaries. They do not adapt to fluctuations in your data over time.
# Validation Types
Source: https://ploutosaitechnologiespvtltd.mintlify.app/reference/validation-types
Full reference for all validation types and their configuration functions in Datachecks.
Datachecks supports seven validation types. Each type covers a different aspect of data quality and exposes one or more functions you reference in the `on` field when configuring a validation.
| Type | What it checks |
| --------------------------------------------- | --------------------------------------------------------- |
| [Completeness](#completeness) | Missing values, nulls, and empty strings |
| [Uniqueness](#uniqueness) | Duplicates and distinct counts |
| [Validity](#validity) | Formats, patterns, value ranges, and identifier standards |
| [Reliability](#reliability) | Row counts, freshness, and timeliness |
| [Numeric Distribution](#numeric-distribution) | Statistical measures across numeric columns |
| [Custom SQL](#custom-sql) | Any metric expressible as a SQL query |
| [Delta Validation](#delta-validation) | Differences between two data sources or segments |
***
## Completeness
Completeness validations detect missing data — null values and empty strings — in your datasets.
| Function | Description |
| ------------------------------ | -------------------------------------------------- |
| `count_null(column)` | Count of null values in the column. |
| `percent_null(column)` | Percentage of null values in the column. |
| `count_empty_string(column)` | Count of empty string (`''`) values in the column. |
| `percent_empty_string(column)` | Percentage of empty string values in the column. |
**Example**
```yaml theme={null}
validations for product_db.products:
- null_count_in_first_name:
on: count_null(first_name)
threshold: "= 0"
```
***
## Uniqueness
Uniqueness validations measure data distinctiveness and identify duplication within a column.
| Function | Description |
| ------------------------- | ---------------------------------------- |
| `count_distinct(column)` | Count of unique values in the column. |
| `count_duplicate(column)` | Count of duplicate values in the column. |
**Example**
```yaml theme={null}
validations for product_db.products:
- distinct_count_of_product_categories:
on: count_distinct(product_category)
threshold: "> 5"
```
***
## Validity
Validity validations check that values conform to expected formats, standards, or patterns.
### Value-based
| Function | Description |
| -------------------------------- | ------------------------------------------------ |
| `count_valid_values(column)` | Count of values matching the `values` list. |
| `percent_valid_values(column)` | Percentage of values matching the `values` list. |
| `count_invalid_values(column)` | Count of values not in the `values` list. |
| `percent_invalid_values(column)` | Percentage of values not in the `values` list. |
Use the `values` parameter to specify the accepted list:
```yaml theme={null}
validations for iris_db.iris:
- valid_species_percentage:
on: percent_valid_values(species)
values: ["setosa", "virginica"]
threshold: "> 65"
```
### String format
| Function | Description |
| ------------------------------- | ------------------------------------------------------------ |
| `string_length_max(column)` | Maximum string length in the column. |
| `string_length_min(column)` | Minimum string length in the column. |
| `string_length_average(column)` | Average string length in the column. |
| `count_all_space(column)` | Count of values that are only whitespace. |
| `percent_all_space(column)` | Percentage of values that are only whitespace. |
| `count_null_keyword(column)` | Count of null-like string keywords (e.g. `"null"`, `"N/A"`). |
| `percent_null_keyword(column)` | Percentage of null-like string keywords. |
### Regex patterns
| Function | Description |
| ------------------------------- | ------------------------------------------------ |
| `count_valid_regex(column)` | Count of values matching the `pattern`. |
| `percent_valid_regex(column)` | Percentage of values matching the `pattern`. |
| `count_invalid_regex(column)` | Count of values not matching the `pattern`. |
| `percent_invalid_regex(column)` | Percentage of values not matching the `pattern`. |
Use the `pattern` parameter to specify the regex:
```yaml theme={null}
validations for users_db.accounts:
- valid_email_format:
on: percent_valid_regex(email)
pattern: "^[\\w.+-]+@[\\w-]+\\.[\\w.]+$"
threshold: "> 99"
```
### Contact information
| Function | Description |
| --------------------------- | ------------------------------------- |
| `count_usa_phone(column)` | Count of valid US phone numbers. |
| `percent_usa_phone(column)` | Percentage of valid US phone numbers. |
| `count_email(column)` | Count of valid email addresses. |
| `percent_email(column)` | Percentage of valid email addresses. |
### Identifiers
| Function | Description |
| ----------------------- | -------------------------------- |
| `count_uuid(column)` | Count of valid UUIDs. |
| `percent_uuid(column)` | Percentage of valid UUIDs. |
| `count_sedol(column)` | Count of valid SEDOL codes. |
| `percent_sedol(column)` | Percentage of valid SEDOL codes. |
| `count_cusip(column)` | Count of valid CUSIP codes. |
| `percent_cusip(column)` | Percentage of valid CUSIP codes. |
| `count_lei(column)` | Count of valid LEI codes. |
| `percent_lei(column)` | Percentage of valid LEI codes. |
| `count_figi(column)` | Count of valid FIGI codes. |
| `percent_figi(column)` | Percentage of valid FIGI codes. |
| `count_isin(column)` | Count of valid ISIN codes. |
| `percent_isin(column)` | Percentage of valid ISIN codes. |
### Geolocation
| Function | Description |
| --------------------------- | ---------------------------------------------- |
| `count_latitude(column)` | Count of valid latitude values (−90 to 90). |
| `percent_latitude(column)` | Percentage of valid latitude values. |
| `count_longitude(column)` | Count of valid longitude values (−180 to 180). |
| `percent_longitude(column)` | Percentage of valid longitude values. |
### Timestamps and dates
| Function | Description |
| ------------------------------------ | ---------------------------------------------------- |
| `count_timestamp_string(column)` | Count of values in valid ISO timestamp format. |
| `percent_timestamp_string(column)` | Percentage of values in valid ISO timestamp format. |
| `count_not_in_future(column)` | Count of timestamps that are not in the future. |
| `percent_not_in_future(column)` | Percentage of timestamps that are not in the future. |
| `count_date_not_in_future(column)` | Count of dates that are not in the future. |
| `percent_date_not_in_future(column)` | Percentage of dates that are not in the future. |
***
## Reliability
Reliability validations monitor data freshness, row counts, and update frequency to detect pipeline delays or data loss.
| Function | Description |
| ----------------------------- | ---------------------------------------------------------------------------- |
| `freshness(timestamp_column)` | Time in seconds since the most recent value in the column. Alert when stale. |
| `count_rows` | Total row count in the table. |
| `count_documents` | Total document count in a search index. |
**Examples**
```yaml theme={null}
validations for product_db.products:
- freshness_of_products:
on: freshness(updated_at)
threshold: "> 86400"
- product_row_count:
on: count_rows
where: "country_code = 'IN'"
threshold: "> 1000"
```
***
## Numeric Distribution
Numeric Distribution validations track statistical measures on numeric columns to detect shifts, outliers, or unexpected variance.
### Central tendency
| Function | Description |
| ------------- | ------------------ |
| `avg(column)` | Mean value. |
| `min(column)` | Minimum value. |
| `max(column)` | Maximum value. |
| `sum(column)` | Sum of all values. |
### Dispersion
| Function | Description |
| ------------------ | ----------------------- |
| `variance(column)` | Variance from the mean. |
| `stddev(column)` | Standard deviation. |
### Percentiles
| Function | Description |
| ------------------------ | ---------------------- |
| `percentile(column, 20)` | 20th percentile value. |
| `percentile(column, 40)` | 40th percentile value. |
| `percentile(column, 60)` | 60th percentile value. |
| `percentile(column, 80)` | 80th percentile value. |
| `percentile(column, 90)` | 90th percentile value. |
### Zero and negative values
| Function | Description |
| -------------------------- | ------------------------------ |
| `count_zero(column)` | Count of zero values. |
| `percent_zero(column)` | Percentage of zero values. |
| `count_negative(column)` | Count of negative values. |
| `percent_negative(column)` | Percentage of negative values. |
**Example**
```yaml theme={null}
validations for sales_db.orders:
- average_order_value:
on: avg(order_total)
threshold: "< 500"
- negative_amounts:
on: count_negative(order_total)
threshold: "= 0"
```
***
## Custom SQL
Custom SQL validations let you define any metric as a SQL `SELECT` statement that returns a single numeric value. Use this for business-specific checks that don't fit standard types.
**Configuration**
| Parameter | Description |
| --------- | -------------------------------------------------- |
| `on` | Must be set to `custom_sql`. |
| `query` | The SQL query. Must return a single numeric value. |
**Example**
```yaml theme={null}
validations for mysql_db.student:
- high_age_bangalore_students:
on: custom_sql
query: |
SELECT COUNT(*) FROM student
WHERE city = 'bangalore' AND age >= 30
threshold: "> 0"
```
Supported on all connected databases: Snowflake, BigQuery, Databricks, PostgreSQL, MySQL, Oracle, Azure SQL, and Sybase.
***
## Delta Validation
Delta validations compare a metric between two datasets — useful for migration checks, cross-environment comparisons, and change detection.
**Configuration**
| Parameter | Description |
| ----------- | ------------------------------------------------------------------ |
| `on` | `delta` followed by a supported function, e.g. `delta count_rows`. |
| `ref` | Reference asset in the format `datasource_name.dataset_name`. |
| `threshold` | The acceptable difference between the two values. |
**Supported functions**
| Function | Description |
| ------------------ | ------------------------------------------------ |
| `delta count_rows` | Compare row counts between source and reference. |
**Example**
```yaml theme={null}
validations for iris_pgsql_source.dcs_iris:
- row_count_delta:
on: delta count_rows
ref: iris_pgsql_target.dcs_iris
threshold: "< 1"
```
# Validation Window
Source: https://ploutosaitechnologiespvtltd.mintlify.app/reference/windows
Configure global or tumbling time windows to control how validation metrics are calculated.
Windows determine the scope of data that a validation runs against. Datachecks supports two window types: **Global** and **Tumbling**.
## Global windows
A global window runs the validation against the entire dataset on every execution. There is no time-based partitioning — every row in the table is included each time.
**Best for:** Small tables, or when you need a full-dataset view every run.
**Trade-offs:** On large tables, global windows perform a full data load on every polling cycle, which can cause performance degradation and high resource consumption.
No additional configuration is required for global windows.
***
## Tumbling windows
A tumbling window divides data into fixed, non-overlapping time segments based on a timestamp column. Each segment is validated independently, making it easier to identify issues in a specific batch or time period.
**Best for:** Large tables, continuous monitoring of recent data, identifying temporal anomalies, and batch-specific quality checks.
### Configuration parameters
| Parameter | Description |
| -------------------- | -------------------------------------------------------------------------------------------- |
| **Date-Time Field** | The timestamp column used to partition data into segments (e.g. `created_at`, `event_time`). |
| **Window Size** | The length of each time segment, combined with Window Unit (e.g. `2` days). |
| **Window Unit** | Unit for Window Size — `hours`, `days`, or `weeks`. |
| **Look Back Period** | How far back in time to retrieve data, combined with Look Back Unit (e.g. `5` days). |
| **Look Back Unit** | Unit for Look Back Period — `hours`, `days`, or `weeks`. |
### How it works
Given a Look Back Period of 10 days and a Window Size of 2 days, Datachecks creates 5 non-overlapping windows. Each window is validated independently. Only the data within the Look Back Period is considered — older records are excluded.
**Example:**
```
Look Back Period: 10 days
Window Size: 2 days
→ Window 1: days 1–2
→ Window 2: days 3–4
→ Window 3: days 5–6
→ Window 4: days 7–8
→ Window 5: days 9–10
```
# Fabricate
Source: https://ploutosaitechnologiespvtltd.mintlify.app/test-data/fabricate
Build synthetic datasets from scratch by defining tables and columns with data generators.
Fabricate lets you create a test database from a blank canvas. You define each table and column yourself, and assign a data generator to populate each field. There is no source datasource required.
Use Fabricate when you need a custom dataset that doesn't map to an existing production schema.
## Create a test database
1. Navigate to **Test Data → Fabricate** in the left sidebar.
2. Click **New Database**.
3. Enter a **Database Name**.
4. Click **Submit**.
## Database View
The Database View is where you build your tables and configure column generators.
### Add a table
Click **New Table** in the top navigation bar. Choose how to create the table:
| Option | Description |
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| **Table** | Import columns from an existing datasource table. Select a datasource, then a dataset, then choose which columns to include. |
| **View** | Import columns from a datasource view. Select a datasource, then a view, then choose columns. |
| **Virtual Table** | Add a pre-built virtual table. All columns are included automatically. |
### Configure a column
Click the edit icon on any column to open the column editor:
| Field | Description |
| ------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| **Column Name** | Name of the column. |
| **Generator** | The data generator to use. Search by name or browse by category. Each generator shows a description and example output. |
| **Hide Column** | If enabled, this column is excluded from the generated output. |
| **Unique** | Enforce uniqueness across all generated values for this column. |
| **Null Percentage** | Percentage of rows that will contain a null value (0–100). |
| **Format** | Optional format pattern for the generated value. |
| **Mask** | Optional mask pattern applied to the value. |
### Generator types
| Generator | Description |
| ----------------------------- | ---------------------------------------------------------------------------------------------------------- |
| **Standard generators** | Email, phone, name, address, and more. See [Test Data Generators](/reference/functions) for the full list. |
| **value\_of\_column** | Copies values from a specific column in a connected datasource. Select the dataset and column. |
| **custom\_function** | An AI-generated function based on the column name. |
| **similar\_values** | Generates values similar to a list of examples you provide. |
| **category\_value** | Generates values from a selected category. |
| **value\_of\_virtual\_table** | Draws values from a pre-built virtual table. |
### Add a column
Click **Add Column** within any table to add a new column and assign a generator to it.
### Row count
Click the edit icon next to the row count in the column table header to set how many rows to generate for each table.
## Table View
The Table View previews the generated data. Select a table from the left sidebar to see a sample of the generated rows.
From here you can:
* **Download Data** — Save the generated dataset as a file.
* **Export YAML** — Export the generator configuration for reuse or version control.
* **Regenerate Data** — Trigger a new generation run with the current configuration.
## Foreign Keys
Define relationships between tables to maintain referential integrity in the generated data.
Click **New FK** to create a relationship:
* **Foreign Key** — Select the table and column that holds the foreign key.
* **Primary Key** — Select the table and column it references.
## Data Targets
Data Targets define where the generated data is delivered — for example, an API endpoint or webhook.
| Field | Description |
| ---------------- | ---------------------------------------- |
| **Name** | A label for this target. |
| **Request Type** | HTTP method — GET, POST, PUT, or DELETE. |
| **Link** | The endpoint URL. |
| **Headers** | Key-value pairs added to the request. |
| **Request Body** | JSON payload sent with the request. |
| **Target Type** | The type of destination. |
# Test Data
Source: https://ploutosaitechnologiespvtltd.mintlify.app/test-data/overview
Generate synthetic test data that mirrors your production data patterns or build datasets from scratch.
Test data generation lets you create realistic synthetic datasets for migration testing without exposing real production data. Datachecks supports two approaches:
| Type | When to use |
| --------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [**Structural**](/test-data/structural) | You want to replicate the data patterns, structure, and sensitivity profile of an existing datasource. Datachecks analyzes your source and generates data that matches its shape. |
| [**Fabricate**](/test-data/fabricate) | You want to build a dataset from scratch. You define the tables and columns yourself, and use data generators to populate each field. |
Both types share the same configuration interface — Database View, Table View, Foreign Keys, and Data Targets — and support the full library of [Test Data Generators](/reference/functions).
Navigate to **Test Data** in the left sidebar and choose **Structural** or **Fabricate** to get started.
# Structural
Source: https://ploutosaitechnologiespvtltd.mintlify.app/test-data/structural
Generate synthetic test data that replicates the structure and data patterns of an existing datasource.
Structural test data generation creates a synthetic copy of an existing datasource. Datachecks analyzes the source schema and sensitivity profile, then generates data that matches its patterns — without using real values.
Use Structural when you want test data that mirrors production without exposing sensitive information.
## Create a test database
1. Navigate to **Test Data → Structural** in the left sidebar.
2. Click **New Database**.
3. Fill in the form:
* **Database Name** — A name for the synthetic database.
* **Select Datasource** — The source connection to replicate from.
* **Select Datasets** — Choose which tables to include. Use **Select All** or search to filter.
4. Click **Submit**.
## Database View
The Database View is where you configure which generators to use for each column.
**Left sidebar** lists all tables from the selected datasource. Each table shows the column count. Filter by sensitivity using the **At Risk**, **Protected**, and **Not Sensitive** buttons.
**Column table** shows the columns for the selected table:
| Column | Description |
| --------------- | ------------------------------------------------------------------ |
| **Column Name** | Name of the column, with source dataset shown for copied columns. |
| **Status** | Sensitivity classification — At Risk, Protected, or Not Sensitive. |
| **Generator** | The generator assigned to this column. |
| **Unique** | Whether generated values are enforced as unique. |
| **Actions** | Edit or delete the column configuration. |
### Configure a column
Click the edit icon on any column to open the column editor:
| Field | Description |
| ------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| **Column Name** | Name of the column. |
| **Generator** | The data generator to use. Search by name or filter by category. Each generator shows a description and example output. |
| **Hide Column** | If enabled, this column is excluded from the generated output. |
| **Null Percentage** | Percentage of rows that will contain a null value (0–100). |
| **Format** | Optional format pattern for the generated value. |
| **Mask** | Optional mask pattern applied to the value. |
### Generator types
| Generator | Description |
| ----------------------------- | ---------------------------------------------------------------------------------------------------------- |
| **Standard generators** | Email, phone, name, address, and more. See [Test Data Generators](/reference/functions) for the full list. |
| **value\_of\_column** | Copies values from a specific column in the source datasource. Select the dataset and column. |
| **custom\_function** | An AI-generated function based on the column name. |
| **similar\_values** | Generates values similar to a list of examples you provide. |
| **category\_value** | Generates values from a selected category. |
| **value\_of\_virtual\_table** | Draws values from a pre-built virtual table. |
### Row count
Click the edit icon next to the row count in the column table header to set how many rows to generate for each table.
### Add a column
Click **Add Column** to add a new column to a table beyond what was imported from the source.
## Privacy Hub
The Privacy Hub gives you a sensitivity analysis of all tables and columns imported from the source datasource. Use it to identify which columns contain sensitive data before configuring generators.
## Table View
The Table View previews the generated data before you export it. Select a table from the left sidebar to see a sample of the generated rows.
From here you can:
* **Download Data** — Save the generated dataset as a file.
* **Export YAML** — Export the generator configuration for reuse or version control.
* **Regenerate Data** — Trigger a new generation run with the current configuration.
## Foreign Keys
Define relationships between tables to ensure generated data maintains referential integrity.
Click **New FK** to create a relationship:
* **Foreign Key** — Select the table and column that holds the foreign key.
* **Primary Key** — Select the table and column it references.
## Data Targets
Data Targets define where the generated data is delivered — for example, an API endpoint or webhook.
| Field | Description |
| ---------------- | ---------------------------------------- |
| **Name** | A label for this target. |
| **Request Type** | HTTP method — GET, POST, PUT, or DELETE. |
| **Link** | The endpoint URL. |
| **Headers** | Key-value pairs added to the request. |
| **Request Body** | JSON payload sent with the request. |
| **Target Type** | The type of destination. |
# Configure Validations
Source: https://ploutosaitechnologiespvtltd.mintlify.app/validation/configure
Step-by-step guide to creating and configuring a validation on a dataset.
Follow these steps to create a new validation on a dataset.
## Steps
### 1. Open the creation form
Navigate to **Validations** in the left sidebar and click **New Validation**.
### 2. Select a validation type
Choose the type of check you want to run. Available types include:
* **Completeness** — Check for missing or null values
* **Uniqueness** — Detect duplicate entries
* **Validity** — Validate values against a format or list
* **Reliability** — Check row counts and freshness
* **Numeric Distribution** — Track statistical metrics (mean, median, percentiles)
* **Custom SQL** — Write a custom SQL expression to measure any metric
* **Delta Validation** — Compare metric values between runs
See [Validation Types](/validation/validation-types) for a full description of each.
### 3. Select your data source and dataset
Choose the **Data Source** and **Dataset** the validation will run against. Enter a name for the validation, or accept the auto-generated suggestion.
### 4. Configure filters (optional)
Apply a filter condition to restrict which rows are included in the metric calculation. For example, filter the `tr_volume` column to `Less Than 5000` to validate only low-volume transactions.
### 5. Define the threshold
Set a threshold to define what constitutes an incident. When the metric crosses this boundary, Datachecks creates an alert. See [Thresholds](/reference/thresholds) for the available threshold types.
### 6. Set the schedule
Choose how often the validation should run — hourly, daily, weekly, or a custom cron expression.
### 7. Create
Click **Create** to save the validation. It will appear in the validations list and run according to the schedule you set.
## What's next
* [View validation results](/validation/details) — Inspect the metrics graph and incidents list
* [Validation Types](/validation/validation-types) — Learn about each check type
* [Thresholds](/reference/thresholds) — Understand how alerting thresholds work
# Validation Details
Source: https://ploutosaitechnologiespvtltd.mintlify.app/validation/details
Inspect metric history, incidents, and run results for an individual validation.
Click any validation in the list to open its detail view. This shows the full metric history and a log of every validation run.
## Metrics graph
The metrics graph plots the validation's measured values over time. Each data point shows:
* **Value** — The metric recorded for that run
* **Bounds** — Upper and lower threshold limits (if configured)
* **Severity** — How far the value deviated from the expected range
* **Detected at** — Timestamp of the run
Hover over any data point to see the details for that specific run.
## Incidents list
Below the graph, the incidents table shows every validation run with the following columns:
| Column | Description |
| ---------- | --------------------------------------------------- |
| **Run ID** | Unique identifier for the validation run. |
| **Value** | The metric value recorded during the run. |
| **Status** | Outcome — `Healthy`, `Alerting`, or `Failed`. |
| **Run at** | Timestamp when the validation executed. |
| **Reason** | Explanation of the failure or alert, if applicable. |
## Filters
Use the timeframe filter to focus the graph and incidents list on a specific date range.
## Actions
From the detail view you can:
* **Run** — Execute the validation immediately
* **Pause** — Suspend scheduled execution
* **Edit** — Modify the validation configuration
* **Delete** — Remove the validation
# Overview
Source: https://ploutosaitechnologiespvtltd.mintlify.app/validation/overview
Monitor data quality by running checks on your datasets and tracking results over time.
Validations are rules applied to your datasets that measure specific metrics and flag anomalies. For example:
* Count total transactions in the `payments` table every day
* Track the percentage of duplicate entries in the `customer_email` column
Validations can be created manually, suggested automatically when a dataset is discovered, or generated by the [Data Validation agent](/agents/data-validation).
## The validations list
Navigate to **Validations** in the left sidebar to see all validations in your workspace.
| Column | Description |
| ------------------ | -------------------------------------------------------- |
| **Name** | Validation identifier. |
| **Type** | The validation category (e.g. Completeness, Uniqueness). |
| **Status** | Current state — `Healthy`, `Alerting`, or `Failed`. |
| **Last Results** | Visual indicator of recent run outcomes. |
| **Last Evaluated** | Timestamp of the most recent execution. |
## Statuses
| Status | Meaning |
| ------------ | ------------------------------------------------------------ |
| **Healthy** | The metric is within the defined threshold. |
| **Alerting** | The metric has crossed the threshold and triggered an alert. |
| **Failed** | The validation run encountered an error. |
## Filters
Use the filter bar to narrow the list by:
* **Status** — Healthy, Alerting, or Failed
* **Category** — Validation category group
* **Method type** — The specific check method
* **Threshold type** — Static, dynamic, or anomaly-based
* **Collections** — Custom groupings of related validations
* **Data source** — The connected database
* **Search** — Filter by name or run ID
## Actions
Each validation row supports the following actions:
* **Run** — Execute the validation immediately
* **Pause** — Suspend scheduled execution
* **Edit** — Modify the validation configuration
* **Delete** — Remove the validation
* **View details** — Open the full results and metrics view
## Collections
Group related validations into collections to run them in parallel or organize them by domain. Select one or more validations and click **Add to Collection**.
## Create a validation
Click **New Validation** to configure a new check. See [Configure Validations](/validation/configure) for a step-by-step guide.
# Manage Users
Source: https://ploutosaitechnologiespvtltd.mintlify.app/workspace/users
Invite and manage users in your Datachecks workspace.
This page is coming soon.
# Manage Workspaces
Source: https://ploutosaitechnologiespvtltd.mintlify.app/workspace/workspaces
Create and manage workspaces in Datachecks.
This page is coming soon.
Workspaces are the top-level organizational unit in Datachecks. Each workspace has its own data sources, assets, validations, and jobs.