Files upload directly from the browser to R2. A dedicated browser Web Worker converts supported files with @firecrawl/anydoc-wasm 0.2.4. The existing cron worker validates and indexes the resulting Document Text. Large originals remain usable even when they exceed the conversion cap.
We selected the browser fallback after the built server Worker failed the memory gate. A batch reached 150.48 MiB; reducing the batch to one conversion per tick still reached 142.35 MiB across repeated ticks. Fresh WASM instances were collectible, but ordinary garbage collection did not reclaim them soon enough to meet the 128 MiB isolate limit. Bundle size and startup passed. The measurement report preserves the rejected server experiment and the final browser results.
Direct byte transport
uploads.prepare checks membership, MIME type, and size before creating an upload intent. A signed PUT covers the exact size, type, key, and If-None-Match condition. Files above 16 MiB use S3 multipart with 8 MiB parts. The browser sends at most three parts concurrently, tries each part at most three times, and resumes from server-listed parts. It reads File.slice() objects rather than allocating the whole file. The Worker signs requests with aws4fetch 1.0.20 and validates completion against the expected part lengths and ETags.
Intent IDs and keys belong to one Organization. Signed URLs expire after 15 minutes. Intents last 24 hours; cron confirms completed objects through HEAD when the browser loses its acknowledgement. Explicit cancellation aborts multipart and invalidates recovery leases. Single-PUT cancellation keeps a 16-minute cleanup window because an already issued URL can still finish. Cron also aborts abandoned multipart sessions after 24 hours, with bounded pagination and work per tick.
The leaf packages/storage/src/config.ts configures storage limits: images 100 MiB, documents 1 GiB, and ZIP archives 5 GiB. The legacy proxy stays capped at 10 MiB. Downloads use the existing gateway's signed redirect or streamed response. Text previews read at most 64 KiB and cancel the reader even when an origin ignores Range. Large images are not fetched as thumbnails. PDF viewers and downloads can issue range requests against the original object.
Bounded conversion in the browser
The Node @firecrawl/anydoc package depends on napi and cannot run in workerd. The unrelated npm package anydoc is not used. The WASM package runs without an external conversion service and supports Office, OpenDocument, EPUB, RTF, CSV, and PDFs that contain text. It does not provide OCR or a streaming parser.
The current caps are 1.5 MiB for PDF, 10 MiB for other supported inputs, and 192 KiB of extracted Markdown. The browser checks the source size before reading it. Resuming conversion downloads the original through a byte-capped streamed reader and verifies its ETag. Encrypted, malformed, unsupported, OCR-required, oversized, and resource-limited documents become permanent failures. A missing converter asset remains resumable. Plain Markdown and text files above the text cap show the limit before the original upload starts. Converted output is checked before prepareText or any Markdown PUT. The distinct text_too_large failure says that the original remains stored and downloadable but will not be indexed. English, German and French copy interpolate the shared cap. Conversion has a 45-second deadline; cancelling terminates its worker and preserves the uploaded original.
A pnpm patch adds createConverter and dispose to the pinned package. Each conversion instantiates a fresh WASM module; finally drops its instance, memory views, and reference heap. The main thread terminates the Web Worker after success, failure, or cancellation. Its separate entry is created with new Worker(new URL(...), { type: "module" }); WASM is fetched only when conversion starts. There is no dynamic import(). Node test helpers load and compile the file through static imports; production server code does not import the converter.
Input size alone does not bound ZIP expansion. The build also changes the pinned WASM memory descriptor from an unbounded memory to a maximum of 32 MiB. documents/wasm-memory.ts rejects an unexpected descriptor instead of silently patching a new package layout. An exhausted memory produces a permanent resourceLimit result. This bounds WASM linear memory; it does not claim to bound all JavaScript and native allocations in an isolate.
The rejected server design required an upload gzip size at most 8.5 MiB, startup below 1 second with margin, and a cap-sized PDF inside the cron budget and 128 MiB isolate limit. Moving conversion back to the server requires passing all three gates, including repeated conversions without test-forced garbage collection. Package upgrades or higher browser caps require another measured browser run.
Persistence and reads
Original upload confirmation creates a pending document associated with the confirmed source ETag. The browser converts it, calls documents.prepareText, and PUTs Markdown to the immutable hidden key incoming/orgs/<organizationId>/<fileId>.md. The signed request fixes its size and content type. confirmText checks HEAD metadata, freezes the text ETag, and publishes the indexing job atomically. The server treats client-produced Markdown as untrusted, including when the client is an Organization member: it verifies source ownership/version, exact size, the 192 KiB cap, UTF-8, and the text ETag while reading. C0 controls other than tab and line breaks are rejected before PostgreSQL ingestion.
Documents reuse the import workflow's Drizzle outbox, expiring lease, token checks, and atomic commit. The tick first settles audits, imports, upload recovery, webhooks, billing and push through Effect.all({ mode: "result" }). Only then does it run document maintenance and indexing, with its own 15-second budget inside the unchanged 25-second tick deadline. There is still one Effect program, one scoped pool and one runEdge. A document failure or exhausted budget cannot prevent those earlier jobs from executing. A global timeout before they finish skips documents; indexing gets only the remaining global time. This ordering does not make an isolate OOM recoverable, but earlier jobs have already settled when indexing starts.
Stable Effect Queue and Stream process one document per tick. The cron downloads only Markdown, cutting chunks as UTF-8 arrives instead of joining a second full decoded string. It retains the bounded chunk collection for the atomic commit. Each insertion binds four PostgreSQL arrays of at most 32 chunks with unnest, avoiding JSONB's extra escaping and limiting protocol allocations. R2 promotes the immutable staged object through a native copy under the lease lock, without a Markdown upload body. A stale worker cannot commit chunks or overwrite newer text. Recovery and replay do not duplicate chunks. With the one-minute schedule, a backlog takes multiple ticks to index.
The default-runtime gate now asserts at most 110 MiB across the baseline, tick samples and final observation, with repeated maximum-size submissions and no forced collection. The 2 MiB, 1 MiB and 512 KiB candidates failed; 256 KiB also failed one of three repeats at 110.91 MiB. The largest tested cap that passed every repeat was 192 KiB, with peaks of 108.85, 108.78 and 108.82 MiB. The measurement report preserves every attempt. These are local inspector samples, not a continuous production memory bound. The old 96 MiB V8 diagnostic is not a gate result.
The journaled limit migration fences oversized pending submissions and marks them text_too_large. It clears obsolete staging size/version metadata before tightening the database check. Original files, existing chunks and ready derived text remain intact; staged objects retain their cleanup deadline.
Staged text is hard-deleted by cron after its recorded 16-minute upload deadline, once indexing finishes or an unconfirmed upload expires. Waiting until the signed URL expires also covers late PUTs. Confirmed pending text stays available for lease recovery. Unsubmitted documents remain visibly waiting for browser conversion; the preview can resume preparation from the stored original. Uploads made through external clients, including MCP, need this browser step before document tools can read their text.
Full Markdown lives at derived/orgs/<organizationId>/<fileId>.md, outside the user-facing orgs/ namespace. Heading-based chunks target 4 KiB and never exceed 8 KiB. Postgres ranks matches through a GIN index on to_tsvector('simple', ...). document_read returns two chunks by default and at most four, with an explicit next cursor. Preview components retain only the current page. MCP search, fetch, and the document resource use these same authorized reads.
Document deletion and source changes lock the matching upload intents, invalidate searchable text, and fence indexing leases before the original storage operation. This also serializes deletion with upload confirmation, which may still be creating the document row. Derived writes share the database lease lock with this invalidation. Invalidation hard-deletes staged and derived text without placing another copy in .trash; if deletion fails, the original operation fails and can be retried. A rolled-back derived write can leave inaccessible text until indexing retries or the original is deleted. Restoring a trashed source HEAD-confirms its current version and prepares a new browser conversion.
Privacy and assistant behavior
Conversion runs locally in the browser. Original bytes and Markdown go only to the configured R2 endpoint and the application's Worker. There is no Firecrawl hosted API, Workers AI, new R2 binding, Queue, Durable Object, or KV resource. Original files remain under the existing trash policy. Extracted text is untrusted: the preview skips raw HTML, suppresses remote images, and uses safe Markdown URL handling. Document text cannot grant tool permissions.
Only current Organization members can read chunks. The assistant also requires its existing provider configuration and feature flag. Reading a document in the assistant sends the returned chunks to that configured model provider. The MCP plan-from-document prompt reads pages and calls draft_task_plan; the resulting proposal remains inert until the user applies it. Deterministic spreadsheet-to-Todo import is deferred; the existing JSON import remains unchanged.