Understanding a Time Series Database
A time series database is a specialized database engineered to handle data that consists of a value paired with a timestamp, collected repeatedly over time. In observability, this describes metrics: a counter for HTTP requests, a gauge for memory usage, a histogram for request latency, all sampled every few seconds and tagged with labels like service name or region. Unlike a general-purpose relational database, a TSDB is optimized around two dominant access patterns: extremely high-throughput sequential writes as new samples arrive, and range queries that scan a window of time for one or more series.
How a Time Series Database Works
Most TSDBs, including the one built into Prometheus, organize data into a series per unique combination of metric name and label set, then store each series’ samples in time-ordered chunks on disk. Because consecutive samples in a series tend to be similar, TSDBs apply aggressive compression techniques, such as delta-of-delta encoding for timestamps and XOR-based encoding for values, which can shrink storage by 90 percent or more compared to naive storage. An in-memory index maps label combinations to the chunks that contain their data, so a query like the ones written in PromQL can quickly locate only the relevant series instead of scanning the entire dataset.
Common Time Series Databases in the Observability Stack
- Prometheus ships with its own local TSDB, ideal for single-cluster deployments with a defined retention window, typically 15 to 30 days.
- Thanos and Grafana Mimir extend Prometheus’s storage model to support long-term retention, global querying across many Prometheus instances, and horizontal scalability by offloading data to object storage like Amazon S3.
- InfluxDB, VictoriaMetrics, and M3DB are alternative TSDBs used outside or alongside the Prometheus ecosystem, each with different trade-offs around ingestion rate, compression, and query language.
A Practical Example
Consider a fleet of 500 Kubernetes pods, each exposing a /metrics endpoint that Prometheus scrapes every 15 seconds. Over a day, that is roughly 2.9 million samples per metric across the fleet. A TSDB needs to ingest this volume continuously, compress it efficiently enough to keep storage costs manageable, and still answer a query like the 99th percentile request latency over the last hour in well under a second so that a Grafana dashboard feels responsive. This combination of high write throughput and fast time-windowed reads is exactly what general-purpose databases struggle with at this scale, and exactly what a TSDB is designed for.
Downsampling and Retention
Because raw, high-resolution samples are expensive to keep indefinitely, most TSDB deployments implement downsampling: aggregating older data into coarser resolutions, for example keeping 15-second resolution for 24 hours, 5-minute resolution for 30 days, and 1-hour resolution for a year. Thanos and Mimir both support this pattern natively, letting teams query recent data at full fidelity while keeping years of historical trend data at a fraction of the storage cost.
Trade-offs and Limitations
- Cardinality is the primary constraint: every unique label combination becomes a new series, and TSDBs degrade sharply once the active series count grows into the tens of millions without careful sharding.
- TSDBs are not designed for arbitrary relational queries, joins across unrelated datasets, or storing large text payloads, which is why logs and traces use different storage engines entirely.
- Long-term, globally queryable retention typically requires an additional layer, such as Thanos or Mimir, on top of Prometheus’s local storage.
Best Practices
- Keep metric cardinality bounded and monitor active series counts as a first-class operational metric of the TSDB itself.
- Set retention and downsampling policies based on actual query patterns rather than keeping everything at full resolution forever.
- Separate short-term operational querying from long-term trend analysis using tiered storage, offloading older blocks to cheaper object storage.
Frequently Asked Questions
What is Time Series Database?
A time series database (TSDB) is a storage engine purpose-built to ingest, compress, and query data points indexed by timestamp, such as metrics from Prometheus. It optimizes for high-volume writes and time-range queries rather than the general-purpose querying a relational database provides.
How does Time Series Database work?
Time Series Database works by combining the components described in the sections above. The main page walks through the architecture, the typical use cases, and the trade-offs to weigh before adopting it.
Why does Time Series Database matter?
Teams adopt Time Series Database to ship faster, run more reliably, and reduce the cognitive load on engineers. The benefits, limits, and adjacent tools are covered in the body above.
When should you use Time Series Database?
Use Time Series Database when the problems it solves match what your team is hitting today. The page above outlines the signals that mean you should adopt it now, and the cases where a simpler approach is fine.
