How JEDI Organizes Your Logged Data

JEDI's Data Storage is a high-performance time-series persistence layer. Your data is stored in a structured, standard CSV files, making it straightforward to retrieve historical charts, generate reports, and analyze trends across any time range.

Directory Hierarchy

JEDI organizes all stored data under a <JEDI Folder>/metrics/ root directory, using UTC time to build a predictable folder path down to the device and hour level.

flowchart LR
    A["<b>metrics</b>
    (Root Directory)"] --> B["<b>YYYY</b>
    (eg. 2026)"]
    B --> C["<b>MM</b>
    (eg. 05)"]
    C --> D["<b>DD</b>
    (eg. 06)"]
    D --> E["<b>DeviceID</b>
    (eg. vSensor)"]
    E --> F["<b>HH.csv</b>
    (eg. 00.csv)"]
    E --> G[01.csv]
    E --> H[...]
    E --> I[23.csv]

metrics: Root directory where all time-series data is stored.
YYYY (Year): Four-digit year (e.g., 2026).
MM (Month): Two-digit month (01–12).
DD (Day): Two-digit day (01–31).
DeviceID: Sanitized device ID or alias used during ingestion.
HH.csv (Hour): Hourly CSV file (00.csv–23.csv) containing all data points recorded during that hour.

Storage on Disk

On disk, JEDI stores data as hourly CSV files grouped by device and date.

<JEDI Folder>/metrics/
  2026/
    05/
      06/
        device1/
          00.csv
          01.csv
          ...
          23.csv
        device2/
          00.csv
          ...

Note

Hourly CSV files are created only if data is received.
Listed exclusions and storage policies also affect retention. Refer to Data Management for more details.

Inside the CSV Files

JEDI stores data using a narrow schema, where each row represents a single metric data point. This approach allows flexibility and scalability without altering file structure as metrics evolve.

Header Row

timestamp,source,attribute,value,datatype,semantic_type,units,metadata

Example:

1778043600131,vSensor,water_flow_gpm,10.56,float64,,,
1778043600131,vSensor,door_open,false,bool,,,
1778043600131,vSensor,bme680_pressure,1014.92,float64,,,
1778043600131,vSensor,chiller_temp_c,6.11,float64,,,

timestamp: Unix timestamp in milliseconds (UTC).
source: Device ID or alias that generated the data.
attribute: Name of the metric (e.g., temperature, pressure).
value: The recorded value of the metric.
datatype: Data type of the value (e.g., float64, bool, string).
semantic_type: Logical classification of the metric (if applicable).
units: Measurement units (e.g., °C, psi, meters).
metadata: Optional additional context associated with the data point.

Best Practices

Standardize Device IDs & Metric Names: Adopt clear, standardized naming for device IDs, metrics, or aliases for intuitive directory navigation and to prevent ambiguity.
Optimize Data Frequency: Log data at an appropriate frequency for your use case. High sampling rates can increase storage usage without adding meaningful insight.
Monitor Storage and Retention: Review retention and compression settings to ensure efficient disk usage while preserving necessary historical data.

Now that you understand how your data is organized, you’re ready to take the next step. Check out Data Management to learn how to configure retention policies, compression, and storage settings for your deployment.

Need help?

Contact our support team support@machinechat.io