ORC Encoding
ORC (Optimized Row Columnar) is the columnar storage format used by Hive; its encoding layer combines dictionary coding, RLE, bit-packing, and stripe indexes.
Purpose
It provides high compression and predicate-pushdown scan acceleration in the Hive/Spark ecosystem.
Mechanism
Data is split into stripes (64MB by default) and each column inside a stripe is encoded separately: strings use dictionary + RLE, integers use bit-packing + RLE, and lightweight indexes (min/max/bloom) support predicate pushdown.
Use cases
- Hive columnar storage
- Spark data reads
- Hadoop data lakes