Documents

Conversion path

A confirmed upload of a supported format creates a pending document bound to the source ETag. A dedicated browser Web Worker converts the original with @firecrawl/anydoc-wasm 0.2.4, then calls documents.prepareText, PUTs the Markdown to incoming/orgs/<organizationId>/<fileId>.md and calls documents.confirmText. Cron validates and indexes that Markdown; the server never runs AnyDoc. ADR-0008 records the memory gate the server design failed.

Supported inputs are PDF with a text layer, Word, Excel, PowerPoint, OpenDocument, RTF, EPUB, CSV, plain text and Markdown. There is no OCR, external conversion service or streaming parser. The worker is created with new Worker(new URL(...), { type: "module" }) and fetches WASM only when conversion starts; no dynamic import() is involved.

Limits

Conversion caps are separate from upload limits. Sources above 1.5 MiB (PDF) or 10 MiB (other formats) are stored but not converted. Extracted Markdown is capped at 192 KiB; plain .md/.txt files above it show the limit before upload. WASM linear memory is bounded at 32 MiB, and conversion has a 45-second deadline in the browser. The caps live in the leaf module packages/db/src/schema/documents.enums.ts.

Chunks target 4 KiB and never exceed 8 KiB. document_read returns two chunks by default and at most four, plus a nextCursor. Cron indexes one document per tick, after the other jobs settle, within a 15-second budget; see scheduled jobs.

Failures and recovery

Encrypted, malformed, unsupported, oversized, resource-limited and OCR-required documents become permanent failures. needsOcr means the PDF has no extractable text; the original stays downloadable. text_too_large means the Markdown exceeded 192 KiB and will not be indexed. A timeout after 45 seconds, a cancelled conversion or a missing converter asset leaves the document pending and resumable: the file preview can restart preparation from the stored original, which the browser downloads through a byte-capped reader and checks against its ETag.

Uploads from external clients, including MCP, report awaitingBrowser until a member opens the file in Files and the browser prepares its text. Deleting or changing a source invalidates its text and fences indexing leases before the storage operation.

Reads and privacy

Only current Organization members can read chunks. The assistant and MCP use the same files_list, search, fetch and document_read operations; see AI assistant and MCP. Reading text in the assistant sends the selected chunks to the configured model provider. Document text is untrusted: previews skip raw HTML and remote images, and text cannot grant tool permissions.

Extend

Changing the AnyDoc version or raising a cap needs a new measured browser run. Moving conversion back to the server requires passing all three gates in ADR-0008. Keep prepareText, the Markdown PUT and confirmText together so the server checks ownership, size, UTF-8 and the text ETag.

Conversion path

Limits

Failures and recovery

Reads and privacy

Extend

Key files