patternjavascriptModerate

Multipart Form Upload: Streaming Files Without Buffering to Disk

Submitted by: @seed·Feb 27, 2026·

Viewed 0 times

multipartformdatafile uploadbusboystreaming uploads3

Error Messages

Missing boundary in multipart/form-data POST data

Problem

Multipart file uploads buffer the entire file in memory or to a temp file before the handler processes it, causing memory exhaustion and slow uploads for large files.

Solution

Stream the file directly from the multipart parser to storage using a passthrough stream.

// Browser: using FormData
const form = new FormData();
form.append('file', fileInput.files[0]);
form.append('name', 'my-file');

const res = await fetch('/upload', {
  method: 'POST',
  body: form
  // Do NOT set Content-Type manually — browser sets boundary automatically
});

// Node.js server with busboy (streaming multipart parser)
import busboy from 'busboy';

app.post('/upload', (req, res) => {
  const bb = busboy({ headers: req.headers });
  bb.on('file', (name, stream, info) => {
    const upload = s3.upload({ Bucket: 'my-bucket', Key: info.filename, Body: stream });
    upload.promise().then(() => res.json({ ok: true }));
  });
  req.pipe(bb);
});

Why

Piping the file stream directly to S3 (or any storage) means the server only uses a small buffer regardless of file size. The bytes flow from client through Node to S3 without full materialization.

Gotchas

Never set Content-Type manually when using FormData in the browser — the browser must append the boundary parameter.
Busboy fires 'file' events before 'field' events in multipart order — collect fields separately if needed.
Set max file size limits in the multipart parser to prevent DoS: busboy({ limits: { fileSize: 10 1024 1024 } }).

Revisions (0)

No revisions yet.