Transforming traditional BI with semantic layers
A podcast interview with Steep’s CEO and co-founder, Johan Baltzar
Podcast Special
The ultimate goal of data analytics is to drive sound and data-backed decisions across the organization. For most, however, this goal is still a North Star objective. Traditionally, companies have hired data teams and analysts to help business users work with analytics software and make data-driven decisions. Unfortunately, this approach doesn’t scale as the company grows. Thankfully, there’s a new generation of BI tools out there that are helping transform the role of data teams from internal service providers to platform builders. Adopting a semantic or metric layer enables data teams to focus on what they do best – building and maintaining infrastructure and doing deep data analysis. Business users work with centralized metrics and truly easy-to-use self-serve analytics tools, alleviating the need for data teams to “service” them with dashboards and reports.
In this interview, Allan Wille, CEO at Klipfolio, sat down with Johan Baltzar, co-founder and CEO of Steep. Klipfolio and Steep share the same vision. They believe in a new generation of analytics products that enable business users to work with centrally-defined metrics in a self-serve environment.
Their conversation explored the ever-evolving landscape of data analytics, the role of the semantic layer in simplifying data accessibility, the changing relationship between data teams and business users, metrics literacy, and the integration of semantic layers with existing BI tools.
Note: This article is a shortened and edited version of a podcast, highlighting the most important insights. Listen to the entire conversation on the Metric Stack Podcast.
What's wrong with traditional BI?
Johan Baltzar is the co-founder and CEO at Steep, where they’ve created a new and faster way for teams to make better data-driven decisions. Johan and his co-founder, Nino Höglund, have years of experience working with PayPal, Spotify, and other tech companies. As an analytics leader, Johan has seen the impact of traditional BI being stuck in the past.
Traditional BI faces challenges when it comes to serving diverse users within a company. Analysts have to handle numerous data requests, resulting in bottlenecks. Data inconsistencies make it hard for decision makers to trust the data. To overcome these issues, a more modern, user-centric, and flexible approach to data analytics is needed.
Allan from Klipfolio: Johan, why don't we start with a little bit of context? Who are you? Tell us a little bit about Steep, and then we'll get into some of the meat of today's episode.
Johan from Steep: I am an analytics leader. I started at Spotify ten years ago when it was launched in the US. I built an analytics team there and worked closely with product development teams. Then, I went to iZettle, which eventually got acquired by PayPal for 2B. Later, I joined a Europe-based unicorn called Kry, a telemedicine company. At Kry, I built a team of 30 people doing everything from data engineering to internal machine learning with a lot of forecasting and staffing optimization. We also did a lot of business analysis. I hired 25 analysts to serve everyone across the company, from marketing to operations.
Allan from Klipfolio: Every company is investing in data and business intelligence. But, only 20% of users are adopting BI. And, that’s probably an exaggeration. So, something's not quite working with traditional BI…
Johan from Steep: I agree 100%. So, on the one hand, we have the data stack. Compared to ten years ago, it’s amazing. It's cheap. It’s efficient. It’s scalable. Big data is a solved problem. But, on the other hand, now we have a bunch of data and we want the entire company of 100-500 people using it. That's the dream and that's what everyone wants. But how do you do that? How do you get it out of your data warehouse and make it useful for everyone?
Allan from Klipfolio: So you and Nino were like, enough is enough. We've got to do something about this. What was the kernel of inspiration that got you two moving?
Johan from Steep: I tried all the traditional BI practices in professional settings. What I realized was that the whole paradigm is broken and we’re still stuck in this old way of doing BI. You have a ton of specialists who are smart people, and they are building dashboards and reports for everyone else. When the end users need to get their hands on some data, they have to go look through old dashboards and reports and try to find the one that suits their needs. It's not flexible, it's not user-friendly, and it's usually quite a mess.
Over time, you end up adding a lot of dashboards and data consistency starts to slip. This old method of data management is low-level and manual. Compare it to engineering practices in other areas, where we say ‘Don't repeat yourself.’, ‘Put definitions centrally.”, ‘Try to be more efficient with everything that you're doing.’ Unfortunately, BI is stuck in the past. We are 10-20 years behind other fields and we need to catch up.
Allan from Klipfolio: This is Steep in a nutshell. This is the problem that you're trying to solve.
Modern data teams should be platform builders, not service providers
The role of modern data teams is evolving from service provider to platform builder. Instead of solely serving the organization's data needs, these teams are focused on creating a centralized semantic (metrics) layer and building a robust data infrastructure. With access to governed metrics, business users are now able to independently explore and analyze their data. This significantly reduces ad-hoc requests to the data team for reports and dashboards.
Not only does this shift empower data professionals to establish a strong foundation and metrics framework, it also allows end users to explore data flexibly, collaborate more effectively, and make informed decisions.
Allan from Klipfolio: During our email exchange, we talked about the relationship between the end users and the data teams and how the decision makers don’t trust the data. How do you see this semantic layer or metrics layer changing this relationship?
Johan from Steep: The analysts are not happy about being the bottleneck. They want to be more efficient and do cooler stuff with their time – like working on the system or doing deeper data investigation. We've spent more than two years just doing the semantic layer and using it with our early customers.
What we're seeing is the shift in roles. As a data specialist, when you go all-in on the semantic layer, you build up the data stack, the architecture, and you define metrics for your company. What are the key and second-tier metrics? What are the drivers of those metrics? What do we set targets on? Instead of serving folks, you set the system in place for everyone else to do more, independently.
Allan from Klipfolio: Do you mean the data teams can build more of a platform as opposed to being a services organization?
Johan from Steep: Exactly. On the end user side, the best part is you don't have to dig through a pile of dashboards or learn SQL. You can go to a nice catalog, a library of predefined metrics, and you can start digging into the data with all the flexibility you need. It becomes a common language across the company, so you can easily find what you need to solve your problems. This becomes a platform for collaboration across different roles in the company.
Allan from Klipfolio: We notice that when we're talking with decision makers, they're not thinking about metrics. They’re still thinking about ‘I want a dashboard’ or ‘I want a report.’ How do we educate end users to think about metrics as first-class citizens?
Johan from Steep: We’ve had this paradigm for so long. Executives want dashboards. Analysts are trained to build reports. We’re all a little bit stuck in our ways. I'm not sure I have a great solution yet. I think when people experience the catalog for themselves, they realize they can just click on everything and explore. They don’t need to request new reports or dig up old ones. They can just do stuff by themselves. Seeing is believing.
Allan from Klipfolio: Business users are going to keep asking for dashboards. But, the new generation of dashboards are better. They’re made up of metrics that are trusted, with common global definitions, and consistent results across reports. The semantic layer increases the relationship between the data team and the business end user. This is what we’re aiming for.
Johan from Steep: Yes. The end result will be that you’ll need fewer data analysts servicing requests and doing basic things, like writing queries and building visualizations, over and over again.
Centralizing metric definitions and supercharging teams
The new generation of BI tools is transforming the analytics landscape. We know that different teams use different BI tools, depending on their role. One of the key components in a semantic or metrics layer is the catalog - a central place where metric definitions live and to which all BI tools in the organization connect. This catalog gives everyone access to single-source-of-truth data, supercharging data and business teams and enabling them to collaborate efficiently and effectively.
Allan from Klipfolio: Based on your experience, and considering what the semantic and metric layer is doing, do you think these tools are going to replace traditional tools? Or, do you think they’re going to live alongside Sisense, Tableau, and Power BI?
Johan from Steep: When we started two years ago, I was on the fence. I thought this is a nice addition, a flexible way for managers to solve some of their problems. Now, after using it for two years, it’s very clear to me that this is the new way of doing BI. The powerful thing is that we're unlocking all the business users so they can now solve their own questions.
Allan from Klipfolio: I'm going to push back on that. Do you think that's true for a nontechnical business end user as well as a data analyst? A data analyst wants to use tools they’re familiar with (like R and SPSS) but they still need trusted data. Do you think they're going to use this new generation of tools? Do you think there's going to be a new generation of R that reads in trusted metric-level data? Or, will there be different audiences that use different tools? What does that consumption tool future look like?
Johan from Steep: One question is the difference between business analysis versus data science analysis. When do you use a BI type of tool? What kind of analysis is that good for? And, it’s usually good for 95% of all the questions in your company. Then, you get to deeper and harder questions where you're looking at correlation, forecasting the future, etc. In this case, you will have a proper data scientist using R, Python, or Hex.
It’s hard to build a tool that does both of those jobs well. How does the semantic layer fit in if you have different tools for different roles? We need to converge on a system where you have your definitions in one place and hook up different tools to those definitions. But, we still have some work to do when it comes to aligning standards and building up better APIs to make that happen.
Allan from Klipfolio: The word ‘standards’ is a massive piece of the puzzle. Where do those definitions live? Is it a standard? Looker may have a certain standard, and dbt may have a certain standard. Are they compatible? Klipfolio and Steep are both very good at that end user layer. How do we make sure that whatever the end user is consuming is also consumable by the data team and everyone else who might like to use a different tool? Let's go even a little further. How important do you think machine consumption is going to be in the future?
Johan from Steep: I think it's going to be huge. There are awesome possibilities of connecting all sorts of services to trusted definitions and being able to automatically trigger things or have models running and feeding off the same source of truth.
Our passion lies in building and using the machine capabilities or the models to build better tools for making people super efficient.
Allan from Klipfolio: I like the sound of supercharging people. BI has been trying to do that for the last 30 years but hasn’t cracked it yet. There's still a lot of people that have to make good decisions every day, and right now they're struggling.
Johan from Steep: The way we see the future is that using data is becoming a new standard. If you're in a modern tech company, this is part of your everyday work. It doesn’t have to be super-advanced stuff - just basic stuff like: What is the impact of what I'm doing? What are our users doing or how is our business behaving? The old paradigm is not working today, and it will continue to get worse and worse. So, we have to move to something more efficient and better.
Implementing a semantic layer as part of the modern data stack
The semantic layer emerges as a pivotal component within the modern data stack, bridging the gap between raw data and actionable insights. Traditional data warehouses lay the foundation, structuring data to some extent, but the semantic layer takes it a step further by providing a unified, user-friendly platform for defining and accessing metrics.
Allan from Klipfolio: Let’s ignore the huge companies, because I think they're building their own tools, and the super-small companies, because there's not enough data maturity. For midsize companies, do you find that their stack is ready? Do you think they’re technically ready to put a semantic layer in place? Do they have the rigor or is their data layer a complete mess? What are you seeing?
Johan from Steep: When you start talking about the semantic layer, there’s a perception that they need to restructure their entire data warehouse before they can even start thinking about it. That’s not necessarily true. The cool thing (about the semantic layer) is that you can start where you're at. You can just sync over your backend data, sync it over to BigQuery or Snowflake, and then start defining your metrics from there. It doesn't have to be a super-engineered perfect structure, because you will probably never get there.
With the semantic layer approach, you automatically end up with a nice structure. So, the metrics, even if their technical definition is a little bit wonky, are, at least, all in one place. They’re cataloged, described, and the same everywhere. My recommendation for most data teams out there is don't wait for the perfect data architecture, start providing definitions and providing value from there.
Allan from Klipfolio: I think most companies have a messier data infrastructure than they'd like to admit. Many qualified data teams consider the data warehouse as that single source of truth. Is this a point of resistance to moving to a semantic layer? Or, do you think this is a stepping stone that will make things a lot easier?
Johan from Steep: There’s a process that goes from taking raw data from different systems and collecting and structuring it, then making sure you can keep it fresh. The last step is getting it out there to the organization. The semantic layer has been a missing piece in that process. It doesn't replace the data warehouse practice, rather it’s an add-on in that process. The gap has been in getting it out to normal people who are busy doing their jobs.
From data literacy to metrics literacy
While data literacy has often been a hurdle, requiring individuals to grasp technical concepts and data structures, metrics literacy simplifies the entire process. It enables users to focus on understanding and using the key performance indicators (KPIs) that matter most to their roles and objectives. Instead of navigating complex data models and databases, users can access a catalog of predefined metrics powered by a semantic or metrics layer, fostering a deeper understanding of what these metrics represent and how they contribute to overall business goals. This transition empowers everyone in the organization (including executives) to ask the right questions, explore insights, and make informed decisions, ultimately making data-driven practices more accessible and actionable across the organization.
Allan from Klipfolio: Is the semantic layer going to help with data literacy?
Johan from Steep: Yeah, I think so. Once you get this into the hands of people, they can explore themselves and do more stuff. People are smart and capable, but they have a job to do. They're busy professionals. Data literacy previously has been too hard. Now that we have tools to help people get what they need, data literacy will come along. It will be less about data literacy and more about metric literacy and analysis literacy.
Allan from Klipfolio: For a team that is currently using a traditional approach, what is the first thing they should start thinking about?
Johan from Steep: I often advise companies that want to get better at using their data to start thinking about metrics and start defining their metrics framework. What are the top five metrics they can use to understand their business and understand if they’re growing profitably? Start from the top, define three to five top metrics, then start to think, we want this metric to move, but how do we move it? So, go one layer deeper. What are the health metrics that are driving the metrics I want to move?
As a next step, start putting some technical definitions around that. Approach this from the human and the business side of things, not just from the data side of things.
Allan from Klipfolio: I love it. Johan, I am sure your insights will definitely inspire and help our readers and listeners to make data-informed decisions, which, as we discussed, is a crucial stride towards achieving a data-driven enterprise.
Johan, CEO & co-founder at Steep on supercharging humans so they can make better decisions with metrics.
Super happy to have you on the show. Thanks so much.
Johan from Steep: It's been a pleasure, Alan. Thanks for having me.
Brought to you by Klipfolio
Klipfolio PowerMetrics is a metrics catalog with built-in metric-centric analytics. To learn more about our platform, visit klipfolio.com.