Observability has rapidly evolved into a necessity in today’s dynamic business landscape. These tools empower companies to monitor their systems’ performance in real-time, offering the insights necessary to proactively address issues. At Ansa, we view the toolchain in three major buckets: Observability platforms that collect diverse data types, observability pipelines that process and aggregate this data, and tools for AIOps, troubleshooting, and, beyond that, leverage this data. Traditionally, vendors focusing on observability have been distinct from those in more downstream areas. However, the market landscape is continuously evolving, presenting many opportunities for innovation.

What’s Shaping the Future of Observability?

Convergence of Data Types

Vendors traditionally specialized in one of the three pillars of observability: metrics, logs, and traces. Companies like Datadog and Grafana have expanded these boundaries and now embrace all major telemetry data types. Emerging startups like Signoz are also adopting a unified observability approach from the start. This convergence leads to full-stack observability, providing a comprehensive view of system performance and operational health, which enterprises find valuable due to quicker insights and reduced complexity.

Consolidation of Vendors

The observability stack for most organizations is often a composite of various solutions, and this doesn’t even account for downstream vendors that rely on the data/insights from observability platforms, such as incident response and troubleshooting tools. However, as seen in New Relic’s 2023 Observability Report, there is a growing desire from engineering leaders to consolidate this toolchain, aiming for simplified management and fewer vendor relationships.

Reducing Costs

Attention to operational efficiency and cost optimization is critical as IT expenditures are scrutinized more than ever. The spotlight on Coinbase’s reported $65M spend on Datadog within a single year exemplifies the broader industry trend where businesses are seeking to maximize their return on investment in observability. This has led to a closer examination of the build vs. buy equation and exploration of alternative strategies to achieve better cost efficiencies while maintaining or improving the quality of operational insights.

OpenTelemetry

The rising popularity of OpenTelemtry is partially a response to the dynamics mentioned above. As a vendor-neutral framework, OpenTelemetry equips developers with tools, APIs, and SDKs to collect and export telemetry data in a standardized manner, enhancing interoperability and flexibility in observability strategies and reducing the risk of vendor lock-in.

Areas of Opportunity

Observability Pipelines

Cribl’s achievement of $100M ARR in under four years signifies that observability pipelines are becoming indispensable for enterprises. Positioned downstream to collectors, these pipelines perform essential tasks: filter out unnecessary data, enrich and transform crucial information, and direct data efficiently to various endpoints. This not only helps in cost reduction but also improves the quality and actionability of the observability data. Furthermore, it enables businesses to maintain a best-of-breed approach due to a reduction in cost concerns.

Bridging Feature Gaps

Data from observability tools is foundational for the effectiveness of downstream tools in the stack. There is a significant opportunity for vendors to extend their reach across this toolchain. We’ve seen the beginnings of this trend with solutions like Datadog’s Observability Pipeline and Grafana’s OnCall product. Startups that can provide unified platforms to bridge these feature gaps may define the next wave of innovation in the space.

AI in Observability

AI is set to take the observability stack to the next level. In addition to areas like simplifying interactions with observability data through natural language processing or streamlining tasks like report generation, AI’s strength lies in its analytical prowess and knack for spotting patterns, which means it’s getting increasingly good at detecting anomalies and boosting troubleshooting processes. As AI technology advances, it will further enhance and automate our ability to identify and resolve issues swiftly.

We’re on the lookout for forward-thinkers who are shaping the future of observability platforms and tools. If you’re building in this space, we’d love to hear from you and share perspectives. Reach out to Josh Chin, josh@ansa.co.