How is Observability different from Product Analytics ?

Context to Product Analytics 🔍

About a decade ago, tools like Mixpanel and Amplitude began revolutionizing product analytics. They transformed how product teams operate today - shifting focus from SQL to a culture of data capture. But here’s the problem, it hasn’t evolved any further since, and products or fast-growing app companies have become much more complex over the same decade.

The data landscape has changed quite dramatically today:

• There’s a vast increase in data capture, making it difficult to keep a tab on everything using dashboards

• Data is scattered and stored in silos across the numerous tools used by Product and Growth teams

• We now have clear patterns in how to interpret and get value out of the data for typical apps

All of the above changes have created a need to push beyond product analytics given its limitations.

Limitation 1: Only the what, not the Why

Product Analytics is good at telling you What happened for metrics that can be constructed using only product usage data (user events basically). It can tell you things like:

• Conversions dropped

• The DAU is up today

• Time spent on your app is down, etc

But, product analytics fails to automatically answer why this happened, it expects you to do work to get to the “why”. This brings us to our next limitation…

Limitation 2: They don't have full context

Key product growth metrics like CAC, Activation, Retention, Revenue, LTC, etc need the context of data that is sitting in multiple tools and not just the event-based data. Here’s what a broad bucket of data tools across data types looks like if you need to root cause product growth metrics in typical consumer apps:

• Performance Marketing data: Appsflyer, Branch, Singular 
• Product Analytics data: Mixpanel, Amplitude 
• Product communication data: Braze, MoEngage, Clevertap 
• Engineering data: Sentry, New Relic, Datadog, Logrocket 
• Transactional data: Redshift, Snowflake, Databricks, BigQuery

So for companies where data is stored in such silos, it’s extremely difficult to root cause metrics like retention, as it could have happened because a Facebook campaign stopped spending as much today, or a segment of users are facing an error in a critical step or someone forgot to send a notification campaign, or all three of these scenarios and more. 

This means someone from the product growth team needs to scan through all necessary data points in these data silos and try to arrive at the correct root cause. Seems quite time-consuming and cumbersome right? 

Limitation 3: They were not designed for this work 

The “why” boils down to segmenting a metric by all factors that could have potentially influenced it and finding further granular segments that could have caused this metric to move. 

A typical answer for a clothing brand looks like this - 

Revenue is down today because users coming from an XYZ TikTok campaign who are visiting the oversized t-shirts category page are not getting drop-off notifications from Braze.

Now to arrive at this statement, you need context from many data sources and someone lucky enough to chance upon these exact combinations of hypotheses causing the impact. This is extremely hard to do and time-consuming on product analytics tools because:

• they don’t have full context (as discussed above)
• their workflows are optimized for reporting metrics and not hyper-segmenting them
• they are built to have a human dependency for guessing their way to the exact specific answer. 

Enter Observability 🕺

Observability tools go beyond the What, they answer the Why. As the name suggests, they can observe your data across multiple tools/data sources and give you value from your data without you doing any work.

They are the answer to all the limitations of Product Analytics! 

Observability tools are not new, tools like DataDog and NewRelic have existed in the Engineering function for the same amount of time as Product Analytics has in the Product function. These tools also succeeded in Engineering Analytics tools like Grafana, and we believe the time has come for this in the Product Growth function as well since our systems have become as complex as Engineering, which now creates a need for Observability. 

Here’s how observability works:

Observability tools can integrate with all data sources, analyze data to identify root causes of key metrics, and proactively highlight issues needing attention or growth opportunities, without any human intervention.

Hence, observability tools also tend to be built in a vertical-specific fashion as they need to have integrations with all the data sources that matter and have built-in intelligence to be able to understand data and how to use it to make it useful for a desired use case! 

Life with Product Analytics vs Life with Product Observability 👀

A product observability tool like Segwise.ai integrates with all your product growth data tools and lakehouses in just a few minutes, without taking engineering bandwidth. By doing so Segwise.ai can proactively explain why your metrics are changing, find user segments that see great value in your product, and address any issues in user journeys.

All of this is automated for you daily, to save your product growth teams from spending hours on dashboards and writing SQL queries!

In case you’re wondering if users from a certain ad campaign, city, and referral code retain more? Any hypotheses or ideas you may have, can all be validated with Segwise’s Playground feature, within a matter of seconds!

Long story short with an analogy, imagine if your car was giving you trouble, would you spend hours, days, and maybe months trying to figure out and fix the problem? Or, would you rather have a mechanic look at it and tell you exactly what’s causing the problem along with a fix? 

That’s the difference between product analytics tools and Observability!