> For the complete documentation index, see [llms.txt](https://docs.concurrence.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.concurrence.com/data/customer-data-intake.md). # Customer Data Intake Workspace-scoped dataset ingestion, file versioning, source discovery, processing, and publication, with explicit security and rollout boundaries. Concurrence's current file-intake workflow uses registered datasets. It gives each upload a named contract, catalog status, and version history, with separate steps for processing and publication. | Scenario | Supported workflow | | --------------------------------------------------------------------- | ---------------------------------------- | | An operator uploads a CSV snapshot against a defined schema | Authenticated dataset intake | | A Google Drive folder should feed a registered dataset | Dataset intake source | | A source exposes a stable API that Concurrence should read repeatedly | [Connector](/data/connectors-and-ehr.md) | | A person needs to complete a structured form | [Surfaces](/channels/surfaces.md) | The Developer Console exposes dataset intake where the Intake feature is enabled. The former standalone upload-link system and external HMAC upload route have been retired; they are not current alternatives to dataset intake. ## Dataset Intake Dataset intake is the governed path for files that need a named data contract, catalog status, and version history. Authenticated workspace routes let an operator register a dataset, upload or download files, list file status, and refresh a dataset from mapped sources. ### Dataset Contracts Every dataset has one of two ingestion modes: * **Snapshot** - A tabular contract with a primary key and typed fields. CSV is the current processing format. Although the registration contract also accepts `xls` and `xlsx` type tokens, the current snapshot validators and processor read CSV bytes; use CSV for processing workflows. * **Document** - A set of accepted filename extensions plus a document-processing configuration. The current native text extractor supports PDF, DOCX, and UTF-8 text such as Markdown. `OCR` and `hybrid` are accepted configuration values but are not current processing implementations and result in a failed processing verdict. Registering a file type does not prove that a downstream processor can interpret every file with that extension. Validate representative files before production use. ### Upload and Version Behavior An authenticated upload is associated with one registered dataset and is capped at 100 MB. Dataset contracts can record `max_size_mb`, but the current upload runtime does not enforce a lower per-dataset value; clients that rely on it must enforce it before upload. The service computes a content hash and returns an existing row for a byte-identical file already present in the same workspace and dataset. Snapshot versions are allocated at the dataset level. Document versions belong to a logical document: omit `document_id` to create a document, or provide an existing `document_id` to add a same-type version. Processing is configuration-dependent: * Snapshot uploads can be validated synchronously or land as `received` for asynchronous processing. * Document uploads land as `received` and require the document processor to reach a terminal status. * A successful upload response confirms storage and cataloging, not successful parsing, publication, or world-model projection. The current intake path does not perform inline malware scanning. An allowlisted extension, MIME type, or successful schema check is not evidence that a file is safe. Workflows that require a malware verdict must arrange that control before submission. ### Google Drive Sources A dataset source maps one or more Google Drive folders to registered datasets. Discovery occurs when an operator triggers a source sync or starts a dataset update; registration alone does not schedule continuous synchronization. A source sync creates a batch per mapped folder, downloads eligible files, and places newly landed files in the dataset catalog. Unchanged content is deduplicated. Processing a batch is a separate step unless the dataset-update workflow orchestrates it. The dataset-update endpoint combines source discovery, preparation, and publication behind a durable run. It applies only when an active Drive source maps to the requested dataset. Monitor the returned run rather than assuming that `202 Accepted` means the dataset is ready. Source removal and data erasure are separate. The audited registration-deletion capability in merged source stops future discovery while retaining imported files, batches, published data, upstream files, and source credentials. Its public deployment remains unconfirmed as of September 16, 2026; see [Intake Sources](https://docs.concurrence.com/developer-guide/platform-api/data-world-model/intake-sources#remove-a-source-registration) before planning source retirement. ## Retired Intake Paths Standalone shareable upload links and the external HMAC upload route have been removed. Existing integration instructions for those paths need migration to authenticated dataset intake or a separately agreed collection workflow. Patient-facing [Surfaces](/channels/surfaces.md) remain a distinct form workflow; a form submission does not automatically create a dataset file. ## Security and Compliance Boundaries * Dataset, source, listing, upload, and download operations require a workspace-scoped Platform credential. * Content-type checks, schema validation, malware scanning, parsing, and downstream publication are distinct controls. Success at one stage does not imply success at another. * The intake routes do not, by themselves, establish a HIPAA or HITRUST compliance posture. Deployment controls, contractual coverage, retention, residency, and operating procedures remain separate concerns. See [Compliance and Audit](/operations-and-safety/compliance.md) and [Data Residency](/platform-overview/data-residency.md). For endpoint-level request and response behavior, see the [Platform intake guide](https://docs.concurrence.com/developer-guide/platform-api/data-world-model/intake). --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.concurrence.com/data/customer-data-intake.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.