Avro Encoding
Avro encoding is the binary serialization scheme used by Apache Avro, supporting both Binary and JSON encodings, with the schema shipped alongside the data or pre-registered.
Purpose
It provides compact, schema-aware row-oriented serialization for the Hadoop/Spark ecosystem, suited to both RPC and storage.
Mechanism
The Binary encoding writes fields in schema order: int/long use Varint with ZigZag, strings are length-prefixed UTF-8, and records are encoded recursively. The schema is not transmitted with every record (it is pre-registered or sent once).
Use cases
- Kafka message serialization
- Hadoop data exchange
- RPC data transport