Croissant Architecture

Layer 2: RecordSets

The RecordSets layer provides the logical structure, mapping raw data resources into semantically meaningful records.

Structured Mapping

Translates diverse formats—from tabular CSV rows to nested JSON trees—into a common set of homogenous data records. This abstraction allows ML loaders to treat different sources identically.

Fields & Data Types

Defines individual data points within a RecordSet. Fields can be extracted from specific file properties or columns and assigned data types like sc:ImageObject or sc:Text.

Logical Interoperability

Common Representation: Essential for combining heterogeneous data (e.g., text and images) into a single training pipeline.
Transforms & Filters: Includes capabilities for regex extraction, data joins, and cross-source references (foreign keys).
Keys & Uniqueness: Supports defining unique keys across records to ensure data integrity during ingestion.

Back to Overview