Clickhouse is winning the Observability Wars
The author reflects on a decade working in observability across multiple companies and platforms, asserting that logs have consistently been the most problematic component. They trace the issue to developers' formative experiences with simple setups like syslog boxes or local containers, where grep and jq worked perfectly. These early successes create unrealistic expectations for large-scale systems with forty or more services, where issues like schema drift, cardinality explosion, and cross-team dashboard requirements emerge. The author notes that logs were the worst part when they started and remain so today, expecting this to continue indefinitely.
Logs remain the hardest part of observability at scale, despite years of tooling evolution.