Stream JSON data for memory-efficient processing
Streaming JSON parsers process data incrementally without loading the entire document into memory. Essential for large JSON files and real-time APIs.
- 1
Standard JSON.parse loads the entire document into memory — a 1GB JSON file needs 1GB+ memory. Streaming parsers (SAX, StAX, Oboe.js, clarinet) process the document incrementally, emitting events as they encounter values. This enables processing arbitrarily large files with constant memory.
- 2 JSON Validator →
For validation of large JSON, use a streaming validator if available. Standard validation loads the entire document — for files larger than 100MB, consider splitting into NDJSON or using a streaming parser. Validate structure incrementally.
- 3 JSON Formatter →
Formatting requires the entire document (to compute indentation). For large files, do not format — process the minified JSON directly. If you need readability, split the file into smaller chunks and format each separately.
- 4
Streaming patterns: (1) SAX-style — register callbacks for events (onObject, onArray, onValue). (2) Path-based — subscribe to specific paths (e.g., $.items[*].name) and process matching values. (3) NDJSON — split the JSON into newline-delimited objects and process line by line. Choose based on your data shape and processing needs.
Frequently asked questions
- When do I need streaming JSON?
- When the JSON file is larger than available memory (typically >100MB), when processing real-time data (WebSocket, SSE), or when you want to start processing before the entire document is received. For small JSON (<10MB), standard parsing is faster and simpler.
- What is JSONPath streaming?
- Some streaming parsers (Oboe.js, jsonpath-stream) let you register JSONPath expressions and receive matching values as they are parsed. This combines the convenience of JSONPath with the memory efficiency of streaming — you only process the values you need.
Workflows that solve this
Learn more
Want to understand why?