Skip to content

How JEDI Organizes Your Logged Data

JEDI employs a structured approach to store your real-time data, ensuring easy navigation and compatibility with external tools. Let's break down the organization.

Directory Hierarchy

graph LR
    subgraph Main Folder
    metrics["metrics"] --> byUTC["byUTC"] 
    byUTC --> metric["airQuality"]
    metric --> date["2024-03-09"]
    date --> file1["2024-03-09_03_00.csv"]
    date --> file2["2024-03-09_04_00.csv"]
    metric --> info["airQuality.info"] 
    end
  • Base Folder ("metrics"): The top-level folder where JEDI stores data. This can be customized if needed.
  • UTC Indicator ("byUTC"): Clearly designates that all timestamps within this structure are in UTC format.
  • Metric Name (e.g., "airQuality"): Data is grouped by the name of the metric. Metrics with the same name across different devices share the same folder.
  • Date (YYYY-MM-DD): Data is further organized by day for efficient retrieval.
  • Hourly Files (YYYY-MM-DD_HH_00.csv): Data within each day is divided into CSV files representing one-hour intervals.

Illustrative Example

metrics/byUTC/airQuality/2024-03-09/
    2024-03-09_03_00.csv
    2024-03-09_04_00.csv
    2024-03-09_05_00.csv
    ... (additional hourly files)

Inside the CSV Files

Each CSV file follows this structure:

1709956247890,SmartFactory,14.348670
1709956257891,SmartFactory,43.935442
1709956267892,SmartFactory,7.499121
  • timestamp (Unix Nanoseconds): High-precision UTC timestamp.
  • device_id: Identifies the device or data source.
  • metric_value: The actual sensor reading or measurement.

Metric Schema (info File)

Each metric folder contains an info file storing the metric's metadata:

{"metric_name":"airQuality","metric_type":"num"}

Key Benefits

  • Organized: Intuitive folder structure for locating data.
  • Efficient: Hourly files balance granularity and management.
  • Accessible: CSV format is widely compatible.

Have a question? Contact our support team support@machinechat.io