Budget saving Sub-Second Observability with qryn + ClickHouse Cloud

ClickHouse Cloud just introduced a new product tier: Development Instances

ClickHouse Development instances provide users with 1TB of data storage, 16GiB of memory, 2 replicas and 2 vCPUs for less than $200 per month, with a 30-day free trial. This is a true blessing for those in need of flexible Clickhouse computing but allergic to bottomless, hard-to-calculate cloud billing options 🎉

Let's see how they perform with qryn powering logs, metrics and traces observability without learning or writing a single line of SQL and without any plugins using qryn's polyglot API capabilities transparently compatible with Loki, Tempo and Prometheus.

Requirements

ClickHouse Development instance (30-day free trial)
qryn instance (Docker example included) or qryn.cloud (free trial)
Deploy everything in the same region for optimal results

Demo Setup

To get started in no time, we'll use the qryn-oss docker demo, which includes sample data generators and offers an out-of-the-box platform experience:

git clone https://github.com/metrico/qryn-oss-demo
cd qryn-oss-demo

Modify the docker-compose-cloud.yml to use your ClickHouse Cloud host.

Replace YOURHOST and YOURPASSWORD with your correct details:

 qryn:
    image: qxip/qryn:latest
    container_name: qryn
    restart: unless-stopped
    expose:
      - 3100
    ports:
      - "3100:3100"
    environment:
      - CLICKHOUSE_SERVER=YOURHOST.clickhouse.cloud
      - CLICKHOUSE_PROTO=https
      - CLICKHOUSE_PORT=8443
      - CLICKHOUSE_AUTH=default:YOURPASSWORD

Once ready, start your demo stack using docker-compose:

docker-compose -f docker-compose-cloud.yml up -d

Access the preconfigured Grafana instance on port 3000 as admin/admin

The dummy-server demo generates correlated logs, metrics and traces with preconfigured data sources, dashboards and derived field correlation.

You can add your own data agents by following the instructions

Demo Logs 👍

Demo Traces 👍

Demo Metrics 👍

First Impressions

Everything just works right out of the box. The system feels super snappy with all standard queries, with sub-second resolution and a 400-700ms average response time for cached queries (great for dashboard updates and multi-user access).

Prometheus Range Query (6h)

Prometheus Range Query (24h)

LogQL range aggregation queries perform nicely even in single-threaded mode:

Console Access

The built-in ClickHouse Console (ex-Arctype, recently acquired by ClickHouse) provides an additional layer of access to all of our data without limitations.

Full scans of qryn samples are quite fast even when accounting for latency. The storage appears to be S3-based and leverages local caching to increase query performance on repeated datasets resulting in excellent speed for a cloud service.

Performance

How does it compare to Grafana Cloud?

With the same demo dataset qryn and grafana cloud (loki) perform very similarly, with effective response time mostly depending on network latency and congestion.

In the above example, a filtered LogQL query with full JSON parsing completes in just 863ms vs. 873ms. Head-to-head performance. You would never be able to tell one of the two specimens is a tiny single-threaded Javascript application talking to a remote ClickHouse service versus a multi-million dollar state-of-the-art golang project using zero latency storage. The comparison holds strong as time expands.

💲Check, Please!

The usage seems to cap around ~$6.4/day or ~**$193/month - as advertised 👍

🆓⚡ The very generous cloud trial covers 30 days of happy evaluation.

Price Comparison

Let's compare the potential costs of our solution against the industry leader:

Grafana Cloud (Hosted)

1TB of logs and 100k labels with Grafana Cloud would cost ~$1,200/mo

~$480 for storage + ~$720 for labels
qryn Cloud + ClickHouse Cloud (Hosted)

1TB of logs and 100k labels with qryn cloud would cost < $600/mo

~$600 including qryn, ClickHouse Cloud for storage
qryn + ClickHouse Cloud (Hybrid)

1TB of logs and 100k labels with qryn + ClickHouse would cost < $300/mo

~$200 for ClickHouse Cloud + ~$80-100 for self hosting qryn

Conclusion

ClickHouse Cloud came up with a great value product for Developers and Startups.

Combined with qryn.cloud (Powered by our very own Gigapipe, with Bring-Your-Own-ClickHouse options) it transforms into a super powerful and incredibly cost-effective observability storage engine 🚀

Special thanks to Alexey Milovidov and ClickHouse for thinking of Developers!

ClickHouse Cloud + qryn Cloud