QuantLake
GitHub
Documentation

Read data

Find the saved files and query them with DuckDB, Polars or pandas.

See what you have

quantlake saved
quantlake saved --exchange binance-futures --symbol BTCUSDT --from 2026-10-01 --to 2026-10-31

saved lists the saved date ranges, with the number of days, rows and the size of each data type and symbol. With --from and --to, it counts only the days in that period. If some days are missing, a symbol has more than one range.

Get the file paths

quantlake files --exchange binance-futures --data-type trades --symbol BTCUSDT --from 2026-10-01 --to 2026-10-07

files prints the path of each saved file in the range. It also prints a wildcard pattern that you can pass directly to DuckDB or Polars.

Read with DuckDB, Polars or pandas

The files are plain Parquet, so you can read them with any tool that supports Parquet. When you turn on Hive partitioning, the folder names become columns: source, exchange, data_type, symbol and date.

DuckDB

SELECT date, count(*) AS trades, avg(price) AS avg_price
FROM read_parquet(
  '~/.local/share/quantlake/data/source=tardis/exchange=binance-futures/data_type=trades/symbol=BTCUSDT/date=*/data.parquet',
  hive_partitioning = true)
WHERE date BETWEEN '2026-10-01' AND '2026-10-07'
GROUP BY date ORDER BY date;

Polars

import polars as pl

trades = (
    pl.scan_parquet(
        "~/.local/share/quantlake/data/source=databento/exchange=glbx.mdp3/data_type=trades/symbol=ES.c.0/date=*/data.parquet",
        hive_partitioning=True,
    )
    .filter(pl.col("date").is_between(pl.date(2026, 10, 1), pl.date(2026, 10, 7)))
    .collect()
)

pandas

import pandas as pd

trades = pd.read_parquet(
    "~/.local/share/quantlake/data/source=tardis/exchange=binance-futures/data_type=trades/symbol=BTCUSDT",
    filters=[("date", ">=", "2026-10-01"), ("date", "<=", "2026-10-07")],
)

Folder layout

~/.local/share/quantlake/
├── AGENTS.md, README.md       # explains why files are read-only
├── metadata.db                # catalog of saved data (SQLite)
└── data/
    ├── source=tardis/
    │   └── exchange=binance-futures/
    │       └── data_type=trades/
    │           └── symbol=BTCUSDT/
    │               └── date=2026-10-01/
    │                   └── data.parquet
    └── source=databento/
        └── exchange=glbx.mdp3/
            └── data_type=trades/
                └── symbol=ES.c.0/
                    └── date=2026-10-01/
                        └── data.parquet

Each file holds one UTC day of one data type for one symbol. QuantLake converts the columns to a common format, but keeps the rows exactly as the provider sends them: it does not add, remove or reorder rows.

The files and folders are read-only. You can read them with any tool, but writes fail with Operation not permitted. To change the lake, use the CLI: see Delete and download again.