Parquet Schema
A Parquet schema uses the Thrift IDL to describe the logical types and nested structure of columnar data, supporting logical annotations such as List/Map.
Purpose
To provide type descriptions and nested-structure mapping for Parquet columnar files so readers and writers interpret column data correctly.
Mechanism
The schema is a Thrift-defined message type containing physical types (BOOLEAN/INT32/...) and logical annotations (STRING/DECIMAL/LIST/MAP). Nesting is expressed through repetition levels (REQUIRED/OPTIONAL/REPEATED).
Use cases
- Parquet columnar files
- Data lake schemas
- Spark/Hive reading