Croissant Architecture

Layer 1: Resources

The Resources layer is responsible for describing the physical distribution and integrity of the dataset assets.

FileObject

Represents individual files that are part of a dataset. It includes critical metadata such as contentUrl, encodingFormat, contentSize, and sha256 checksums for cryptographic verification.

FileSet

Handles collections of homogeneous files, typically within container formats like .zip or .tar. It allows for advanced filtering using glob patterns (e.g., *.jpg) and defines contents contained within other objects.

Key Technical Concepts

Cryptographic Integrity: Strong recommendation for SHA-256 checksums to ensure downloaded data exactly matches the description.
Flexible Distribution: Modernizes the standard Schema.org distribution model to better fit complex ML dataset layouts.
Live Datasets: Special support for non-static datasets with continuous updates using the isLiveDataset flag.

Back to Overview