> For the complete documentation index, see [llms.txt](https://docs.concurrence.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.concurrence.com/developer-guide/platform-api/data-world-model/intake.md).

# Customer Data Intake

Register governed intake datasets, upload and download catalog files, refresh mapped Drive sources, and monitor durable dataset processing and update runs.

These endpoints manage the **dataset intake catalog**: registered dataset contracts, direct file uploads, files discovered by mapped [intake sources](/developer-guide/platform-api/data-world-model/intake-sources.md), and dataset update runs. They require a workspace-scoped Platform bearer credential.

The separate [upload-link system](/developer-guide/platform-api/data-world-model/intake-links.md) has been retired. New file integrations should use the authenticated dataset workflow described here.

{% hint style="info" %}
The Developer Console's dataset-intake surface is feature-gated. API availability does not guarantee that the Intake tab is enabled for every production environment.
{% endhint %}

## Files

### List Files

Returns a paginated dataset-file catalog. Results can be sorted by ingestion time, filename, status, dataset, or size; filtered by status; searched by filename; and optionally include withdrawn document versions.

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/intake/files" method="get" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

The status vocabulary is `received`, `scanned`, `processing`, `curated`, `rejected`, `failed`, and `held`. Current upload paths commonly use:

| Status     | Meaning                                                        |
| ---------- | -------------------------------------------------------------- |
| `received` | Stored and waiting for, or eligible for, downstream processing |
| `curated`  | Validation or configured processing completed successfully     |
| `rejected` | The file did not satisfy the tabular contract                  |
| `failed`   | Document extraction or another processing step failed          |

`scanned`, `processing`, and `held` are valid wire values, but they do not by themselves prove that malware scanning ran. Use `error_reason` for failure context and treat unknown future values defensively.

### Upload File

Uploads one file as `multipart/form-data` to a registered dataset. The service enforces a 100 MB request cap.

`max_size_mb` is stored on the dataset contract and returned by the dataset list, but the current upload runtime does not enforce a lower per-dataset value. Enforce that value in the client if your workflow depends on it.

Before persistence, the service computes a SHA-256 hash and returns the existing file row for byte-identical content already present in the same workspace and dataset. The current upload path does not perform inline malware scanning; arrange a separate pre-submission control if your workflow requires a malware verdict.

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/intake/files" method="post" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

Upload behavior depends on the dataset mode:

* **Snapshot** - The current processing format is CSV. Depending on workspace configuration, validation can complete synchronously (`curated` or `rejected`) or the file can enter asynchronous processing as `received`.
* **Document** - The filename extension must be in the dataset's accepted set. Omit `document_id` to create a logical document, or pass an existing document ID to add a same-type version. Documents enter as `received` and require the extraction processor for a terminal verdict.

{% hint style="warning" %}
Dataset registration accepts `csv`, `xls`, and `xlsx` as snapshot type tokens, but current snapshot validators and processing read CSV bytes. Use CSV for current processing workflows.
{% endhint %}

Common route errors include:

| Status | Condition                                                                                                      |
| ------ | -------------------------------------------------------------------------------------------------------------- |
| `413`  | File exceeds the global 100 MB cap                                                                             |
| `422`  | Missing dataset contract, invalid or empty filename, disallowed document type, invalid document version target |

#### Tag a Document with Its Subject

A document upload can carry `doc_metadata`, a JSON-encoded form field of string tags that link the document to the subject it is about, for example `{"patient_id": "synthetic-123"}`. Send 1 to 32 tags. Keys are 1 to 64 characters of letters, digits, `_`, `.`, and `-`; values are 1 to 256 characters. Omit the field for no tags; an empty object is rejected.

* Tags are set when the document is created and apply to every later version. Sending `doc_metadata` with an existing `document_id` returns `422`, and so does sending it to a snapshot dataset.
* Byte-identical content deduplicates onto an existing document only when its tags match exactly, so two subjects' identical files stay separate documents.
* The Files list returns each document's tags in `doc_metadata` (`{}` when untagged, `null` for snapshot files). The upload response echoes the tags sent with that request, so read the Files list for the stored value.

Tags are caller-supplied data. They can hold identifiers about a person, so handle them like the document itself, and do not treat them as verified identity.

### Download File

Downloads the original catalog file. The response includes `Content-Disposition: attachment` and `Cache-Control: no-store`; a file outside the workspace scope returns `404`.

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/intake/files/{file\_id}/download" method="get" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

## Dataset Contracts

### List Datasets

Returns registered dataset contracts with their mode, representative file type, accepted document types, schema version, field count, and recorded `max_size_mb` value.

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/intake/datasets" method="get" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

### Register a Dataset

The server infers `ingestion_mode` from `file_type`:

* **Snapshot** - `csv`, `xls`, or `xlsx`; requires one or more primary-key columns and fields. The supported field types are `str`, `int`, `float`, `bool`, `date`, and `datetime`.
* **Document** - Any other lowercase alphanumeric type; ignores the tabular primary key and field list. `accepted_file_types` can contain up to 16 non-tabular types.

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/intake/schema/register" method="post" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

Document contracts accept `document_processing.extraction_mode` values of `text_extract`, `OCR`, or `hybrid`, but the current processor implements `text_extract` only. `OCR` and `hybrid` reach a failed processing verdict rather than performing OCR.

With `text_extract`, the current processor handles PDF, DOCX, and UTF-8 text types such as `txt`, `text`, `md`, and `markdown`. An unknown type is treated as text only when its bytes decode as UTF-8; otherwise processing fails. Registering an accepted extension does not guarantee parser support.

| Status | Condition                                                                               |
| ------ | --------------------------------------------------------------------------------------- |
| `409`  | A dataset with this name already exists in the workspace                                |
| `422`  | Invalid fields, types, primary key, extraction mode, or mixed tabular/document type set |

## Update a Dataset

`POST /v1/{workspace_id}/intake/datasets/{dataset}/update` starts the one-click workflow for a dataset backed by at least one active Google Drive source. It checks mapped folders, creates batches for newly landed files, starts preparation, and then starts publication when preparation succeeds.

This route does not refresh a dataset made only from manual uploads. If no active Drive source maps to the dataset, the accepted run transitions to `failed` with an explanatory error.

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/intake/datasets/{dataset}/update" method="post" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

The `202 Accepted` response is a durable run record, not a completion receipt. Its status is one of `checking_drive`, `preparing`, `publishing`, `completed`, `needs_review`, or `failed`. Poll the run until it reaches a terminal status:

{% openapi src="<https://platform.api-us.concurrence.com/v1/openapi.json>" path="/v1/{workspace\_id}/intake/update-runs/{run\_id}" method="get" %}
<https://platform.api-us.concurrence.com/v1/openapi.json>
{% endopenapi %}

The run response includes `source_ids`, `batch_ids`, timestamps, and optional `materialize_run_id` and `error`. Starting another update while one is active can return the existing run rather than create parallel work.

## External HMAC Upload

The legacy external HMAC upload route has been removed from the current Platform API. It is no longer a separately provisioned compatibility option. Migrate file integrations to the authenticated dataset upload workflow above; dataset registration and processing remain explicit steps.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.concurrence.com/developer-guide/platform-api/data-world-model/intake.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.