Intake Sources
Register Google Drive folder mappings, trigger explicit source syncs, inspect returned batches, and coordinate secure out-of-band source credential setup.
Intake sources map Google Drive folders to registered dataset intake contracts. Files are discovered when you explicitly trigger a source sync or start a dataset update; registering a source does not create a continuous schedule.
Source-discovered files enter the dataset catalog and carry a batch association. Files uploaded directly to a dataset do not have a batch. The former upload-link system has been retired.
Current source registration creates the google_shared_drive source type. Depending on credential setup, the mapped folder can be in a Shared Drive or be a folder shared with the configured account.
Register a Source
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Register a Google Shared Drive intake source (design §3).
The credential is not part of the wire contract and nothing needs provisioning:
the source authenticates as the unified platform-infra intake service account,
impersonated from the pod's workload identity. Grant access Drive-side — share
the folder with that SA's email, or set impersonate_subject for the DWD
model. drive_id is optional (resolved at sync if omitted).
Successful Response
A registered intake source (Sources list).
source_id.
DWD impersonation subject, if this source authenticates via domain-wide delegation.
ISO-8601 timestamp.
Validation Error
POST /v1/{workspace_id}/intake/sources HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 140
{
"display_name": "text",
"drive_id": "text",
"folders": [
{
"folder_id": "text",
"dataset": "text",
"contains_phi": true
}
],
"impersonate_subject": "text"
}{
"id": "123e4567-e89b-12d3-a456-426614174000",
"source_type": "text",
"display_name": "text",
"drive_id": "text",
"folders": [
{
"folder_id": "text",
"dataset": "text",
"contains_phi": true
}
],
"impersonate_subject": "text",
"status": "text",
"created_ts": "text"
}List Sources
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
500-created_tsSuccessful Response
Validation Error
GET /v1/{workspace_id}/intake/sources HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*
{
"items": [
{
"id": "123e4567-e89b-12d3-a456-426614174000",
"source_type": "text",
"display_name": "text",
"drive_id": "text",
"folders": [
{
"folder_id": "text",
"dataset": "text",
"contains_phi": true
}
],
"impersonate_subject": "text",
"status": "text",
"created_ts": "text"
}
],
"has_more": true,
"continuation_token": 1,
"total": 1
}Trigger a Source Sync
A source sync performs discovery immediately and returns one batch summary per mapped folder.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Successful Response
Validation Error
POST /v1/{workspace_id}/intake/sources/{source_id}/sync HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*
{
"batches": [
{
"batch_id": "123e4567-e89b-12d3-a456-426614174000",
"dataset": "text",
"files_found": 1,
"status": "text"
}
]
}For each folder, the sync creates a batch, recursively lists files, downloads eligible content, and calls the same catalog upload service used by direct dataset uploads. Byte-identical content is deduplicated. A batch with no newly landed rows returns completed with files_found: 0; otherwise it returns ready.
Processing is separate from source discovery. Use the batch processing endpoint for manual orchestration, or use the dataset update run to coordinate discovery, preparation, and publication.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Successful Response
A source-sync batch (Batches list / detail).
batch_id.
discovered | ready | processing | completed | failed.
Validation Error
POST /v1/{workspace_id}/intake/batches/{batch_id}/process HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*
{
"id": "123e4567-e89b-12d3-a456-426614174000",
"source_id": "123e4567-e89b-12d3-a456-426614174000",
"dataset": "text",
"status": "text",
"files_found": 1,
"created_ts": "text"
}Current source syncs consider up to 1,000 files in a mapped folder tree per run. Split larger collections into multiple mappings or coordinate a different ingestion plan with Concurrence.
404
Source ID is not present in the workspace
422
The out-of-band source credential has not been provisioned
502
Google Drive authentication failed
Batch Lifecycle
discovered
Batch record created while discovery is in progress
ready
At least one new file is associated with the batch; processing has not been settled
processing
Batch processing has started
completed
Processing completed, or the sync found no new content to process
failed
Batch processing failed
Snapshot batches are processed in version order. Document batches can be processed together. A source sync response is therefore a discovery receipt, not a processing or publication receipt.
Authentication
Source management, sync, and batch routes require a workspace-scoped Platform bearer credential. See Authentication and API Keys.
Remove a Source Registration
Remove a source registration without deleting upstream files or imported data.
API key issued via POST /v1/{workspace_id}/api-keys. Pass the returned api_key value as a Bearer token.
Successful Response
No content
Write permission required
Source not found
Source operation in progress
Validation Error
Coordination unavailable
DELETE /v1/{workspace_id}/intake/sources/{source_id} HTTP/1.1
Host: platform.api-us.concurrence.com
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*
No content
Deletion is included in the public Platform schema checked September 27, 2026. It requires workspace Data.write permission and removes the source from active listings and future discovery. It records deletion time and actor/credential attribution. It retains upstream files, credentials, existing batches, imported files, and published data; removing a registration is not a data-erasure operation.
A successful deletion returns 204. Repeating deletion of the same retained registration also succeeds without changing its original deletion attribution. A missing source or one in another workspace returns 404; an operation already using the source can return 409, and unavailable coordination can return 503. Wait for the active operation to settle or coordination to recover before retrying. A subsequent sync cannot use a deleted registration.
After removal, verify that the source no longer appears in the active list. Handle any requirement to delete retained data or revoke source credentials as a separate, explicitly scoped operation with Concurrence.
Last updated
Was this helpful?