| read_eprint takes bytes and runs unpack, main-file selection, parsing, bibliography and figure resolution, with no I/O | Read from the code | superdocuments/src/arxiv_source/mod.rs:123 | read_eprint_with unpacks the data, selects the main file, parses it with an archive-backed FileProvider, collects bibliography and figures, and returns warnings. A grep of src/arxiv_source for std::net, reqwest and std::fs finds only a test helper that reads a fixture. |
| The archive unpacker accepts gzip tar, plain tar or a single gzip TeX file, and refuses unsafe paths and links | Read from the code | superdocuments/src/arxiv_source/archive.rs:81 | unpack() returns the tar's regular files, or main.tex for a single gzip file, or NotAnArchive. read_tar refuses hard and symbolic links (:239). validate_path refuses absolute paths, .., backslashes and empty names (:297-317). |
| \input and \include resolve through a FileProvider, with the archive lookup relative to the main file | Read from the code | superdocuments/src/latex/parser.rs:1087 | include() asks the provider for the name, then the name with .tex. Without a provider it records the name in inputs and warns. arxiv_source/mod.rs find_entry tries base plus name, then name as written. |
| Macros expand in text; \usepackage is dropped silently; unknown commands warn and keep their arguments | Read from the code | superdocuments/src/latex/parser.rs:715 | newcommand, renewcommand, providecommand, DeclareMathOperator are handled at :715-723. The unknown-command warning is at :850. drop_spec (:2084) drops usepackage, documentclass and others without a warning. |
| Math is written as $...$ and $$ blocks, figures as captions and , citations as text | Read from the code | superdocuments/src/latex/render.rs:185 | inline_math and display_math (:185-222) write $ and $$ with \tag for numbered rows. image (:244) writes , the path as written. cite (:323) writes (label) text from thebibliography or a supplied Bibliography, else the key. |
| BibTeX write emits expanded field values; undefined macros warn; duplicate keys keep both entries | Read from the code | superdocuments/src/bibtex/write.rs:18 | write() prints each field as its raw (expanded) value. parse.rs:303 warns on an undefined macro, parse.rs:207 on a duplicate key, and parse.rs:263 keeps the first of repeated fields. |
| MathML returns an annotation's TeX when one exists, and keeps unknown element content with a warning | Read from the code | superdocuments/src/mathml.rs:85 | to_tex calls tex_annotation (:163), which accepts an annotation under semantics whose encoding contains 'tex'. Unknown elements produce 'unsupported MathML element' (:718) and keep their children. |
| PDF images are listed with bytes; the text output marks each image as an object-numbered marker | Read from the code | superdocuments/src/pdf/images.rs:54 | extract_images returns page, object number, size, colour space, filters and encoding, with JPEG and JPX unchanged and Flate as stored-deflate PNG. The text path writes  at superdocuments/src/pdf/parser/parse.rs:971, using the object number, not a sequence index. |
| webfold arxiv get reads arXiv HTML first and the PDF on a 404; it does not fetch the e-print | Read from the code | webfold/src/arxiv/mod.rs:264 | retrieve() gets /html/<id>. On a 404 it gets /pdf/<id> and calls document::convert_bytes("pdf") (:280). The src/arxiv tree has no e-print or tar code. |
| The webfold PDF fallback has no pdf arm in convert_bytes, so it returns an unsupported-extension error | Read from the code | superdocuments/src/document.rs:849 | superdocuments convert_bytes has no pdf case, so the error is 'unsupported document extension: pdf'. The webfold wrapper (src/document.rs:126) adds only xls and csv before delegating. |
| webfold uses superdocuments for document, pdf, epub and graph only; not latex, bibtex, arxiv_source or mathml | Read from the code | webfold/src/cli/document.rs:260 | The superdocuments paths used in webfold are document::convert_bytes and convert_path, pdf::extract_markdown and related pdf functions, epub::read and extract_text, and graph. A grep of webfold src for latex::, bibtex::, arxiv_source:: and mathml:: finds no use outside its own arxiv code. |
| Named test: an archive gives its main file, document, bibliography and figures | Covered by a test | superdocuments/src/arxiv_source/tests.rs:36 | paper_archive_gives_main_file_document_bibliography_and_figures checks the \input inlining, bibliography keys, figure label and format, and EPS detection. Not run. |
| Measured: pandoc 3.1.3 agrees on 16 LaTeX fixtures except figure*; pandoc fails on three of five BibTeX files | Measured | superdocuments/benches/latex-vs-pandoc/results.json | 20 runs per timing. figures.tex differs on figures_top_level (ours 4, pandoc 5) and figures_with_caption (ours 6, pandoc 5). pandoc fails on basic.bib (unexpected '('), malformed.bib and crossref-biblatex.bib (out of memory on a crossref cycle). The file has no date; disagreements.txt and the comparison matrix date from 2026-11-06. Hand-made fixtures. |
| Measured: LaTeX fidelity to Pandoc's reading rose from 0.857 to 0.968 over 118 documents; 179 of 179 BibTeX entries match Pandoc's CSL-JSON | Measured | superdocuments, benches/pandoc-compat (merged to main after the revision Webfold pins): README.md, results/sd-before.json, results/sd-after.json, results/sd-after-bib.json | Pandoc 3.1.3 reads each file; the score is the mean of structure, features and words in superdocuments' Markdown as Pandoc reads it back. Before is superdocuments main from before the Pandoc-compatibility work, built clean and scored on the same corpus. 10 of 118 documents are identical; the six hand-written ones average 0.779, mostly because superdocuments appends a reference list from the .bib and Pandoc does not. Pseudocode in algorithmic environments is kept as source, where Pandoc keeps only the math. Generated documents come from seeded scripts, so this is not a sample of real papers. Formatted citations were compared with pandoc --citeproc on 125 documents with 6 styles (7,454 of 7,458 citations and 4,362 of 4,362 entries identical; the processor was developed on this set) and on 4 other styles (5,079 of 5,080 and 2,214 of 2,952 after fixes; 4,929 and 1,923 on the first run). |
| Measured: PDF image listing agrees with poppler pdfimages on 7 test images | Measured | superdocuments/benches/pdf-images/results-images-mixed.txt | 6 images in images-mixed.pdf and 1 in inherited-predictor.pdf, with 0 disagreements against pdfimages (results-inherited-predictor.txt). The files carry no date. |
| The hosted arXiv retrieval reads LaTeX source and its .bib bibliography when there is no HTML version | Covered by a test | webfold-saas-api/src/arxiv_fetch.rs (tests module); live check 2026-11-07 | Nine unit tests pass in a scratch crate built against the real superdocuments and arxiv-search crates. Live, with HTML forced off: 1706.03762v1 (38k characters), hep-th/9901001 (12k) and 2401.04088v1 (38k, with 47 bibliography entries from its .bib). The change is merged into the hosted API's main branch. That crate did not build in the sandbox, so its main.rs wiring was not compiled there, and the hosted service was not checked. |