---
search:
  tags:
    - Corpus
    - POST
seo:
  description: >-
    Optionally redacts each record first, then splits it into chunks of at most
    chunk_chars… Reference for the POST /v1/corpus/build endpoint in the Webfold
    API.
sidebar:
  label: Chunk and de-duplicate records into corpus rows
  badge: POST
title: Chunk and de-duplicate records into corpus rows
type: openapi-operation
---
Optionally redacts each record first, then splits it into chunks of at most `chunk_chars` characters (preferring paragraph, line then word boundaries) with optional overlap, and drops chunks whose normalised text was already seen. Offsets are character offsets into the (possibly redacted) record text. Each record is metered, as `corpus.build`.

`POST /v1/corpus/build`

**Request body** (`application/json`, required)

- `records` (CorpusRecord[], required)
- `chunk_chars` (integer)
- `overlap` (integer) — At most half of \`chunk\_chars\`.
- `dedup` (boolean)
- `redact` (boolean) — Redact each record before chunking.

Request body example:

```json
{
  "records": [
    {
      "id": "string",
      "text": "string"
    }
  ],
  "chunk_chars": 2000,
  "overlap": 0,
  "dedup": true,
  "redact": false
}
```

**Responses**

- `200` — Corpus rows and statistics
- `400` — Bad request
- `401` — Missing or invalid bearer token
- `403` — Billing or plan entitlement blocked the request
- `502` — Upstream service (arXiv, YouTube, web page) failed
- `503` — The backing service is not configured or is busy

Response example, 200:

```json
{
  "rows": [
    {
      "id": "string",
      "chunk_index": 0,
      "start": 0,
      "end": 0,
      "hash": "string",
      "text": "string"
    }
  ],
  "stats": {
    "records_in": 0,
    "chunks_total": 0,
    "rows_out": 0,
    "duplicates_dropped": 0
  },
  "redacted": true,
  "findings_total": 0
}
```