Skip to content
Webfold Docs
Esc
↑↓navigate↵open⌘Jpreview

Retrieve an arXiv paper

Fetch a paper by its arXiv ID, normalize it to Markdown, optionally prune references, and chunk the result. The paper metadata is returned nested under paper along with the extracted text and chunk list.

POST/v1/arxiv/retrieve
Authorization
AuthorizationBearer token (JWT) · headerrequired

Bearer API key issued for your account. Send Authorization: Bearer <key>.

Request body
requiredapplication/json
paper_idstringrequired

arXiv paper ID

prune_referencesboolean

Drop the references/bibliography section

default: true
chunk_charsinteger

Target chunk size in characters

default: 4000
chunk_overlapinteger

Characters of overlap between adjacent chunks

default: 200
Responses
200

Paper with extracted Markdown and chunks

paperArxivPaperrequired
Show properties
idstringrequired
titlestringrequired
authorsstring[]required
abstract_textstringrequired
categoriesstring[]required
publishedstring<date-time>required
urlstring<uri>required
sourcestringrequired

Source format used for extraction (e.g. html)

raw_markdownstringrequired

Normalized Markdown extracted from the paper

pruned_markdownstringrequired

Markdown with references section removed when prune_references is true

chunksArxivChunk[]required
Show properties
Array of ArxivChunk
indexintegerrequired
start_charintegerrequired
end_charintegerrequired
textstringrequired
400

Bad request

string
401

Missing or invalid bearer token

string
403

Billing or plan entitlement blocked the request

string
502

Upstream service (arXiv, YouTube, web page) failed

string
Try it
Server
Authorization
Bodyapplication/json
Request
curl -X POST 'https://api.webfold.ai/v1/arxiv/retrieve' \
  -H 'Authorization: Bearer YOUR_TOKEN' \
  -H 'Accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "paper_id": "1706.03762",
  "prune_references": true,
  "chunk_chars": 4000,
  "chunk_overlap": 200
}'
Response
{
  "paper": {
    "id": "1706.03762",
    "title": "Attention Is All You Need",
    "authors": [
      "Ashish Vaswani",
      "Noam Shazeer"
    ],
    "abstract_text": "The dominant sequence transduction models...",
    "categories": [
      "cs.CL",
      "cs.LG"
    ],
    "published": "2017-06-12T00:00:00Z",
    "url": "https://arxiv.org/abs/1706.03762"
  },
  "source": "html",
  "raw_markdown": "string",
  "pruned_markdown": "string",
  "chunks": [
    {
      "index": 0,
      "start_char": 0,
      "end_char": 1234,
      "text": "..."
    }
  ]
}