What does "self-serve" in analytics mean to you?

Nearly every client we work with at Fishtown Analytics talks about their desire to create a self-service culture around data. But this means a lot of different things to different people. I’m doing some initial research for an article that will eventually live on the dbt blog. If you have opinions on this topic, I would love to hear from you!

Some questions I’m thinking about…

What does “self-serve” in analytics mean to you?

  • Really good reporting in a BI tool that most of the company has access to and views regularly?
  • Power users that are not on the data team, but are comfortable doing some slicing & dicing on top of existing charts?
  • End users who are comfortable creating charts & dashboards from scratch and even writing their own SQL?
  • Something else?

How self-service is analytics currently at your company? Do users have the tools but aren’t adopting it? (or still just exporting data to Excel to do their own analysis?) Are they pressuring for more self-serve capabilities/training?

How self-service do you want analytics at your company to be?

  • Which roles in your organization should be able to self-serve their analytics?
  • What should stakeholders be empowered to do, and what should the data team be responsible for?

What’s “normal”? How self-service do you think a company should be? Are there companies doing an outstanding job on this? Do you think your company is normal-ish? Above average? Is it something you’re actively trying to improve?

What are the barriers you’ve encountered to creating an empowered, self-service culture? Are end users resistant to using a new tool? Is data too messy, complicated? Are you struggling to get buy-in from leadership?

How important is data documentation in creating and maintaining a self-service culture? Do your stakeholders currently read data documentation? Or they still come to analysts for quick answers?

Is there anything you’ve found to be particularly useful in creating a self-service culture? Training? Finding internal “data champions?” What’s working for you?

6 Likes

To me the important thing to remember is that Self Serve analytics is a spectrum of different activities, but ultimately its anything that takes work, particularly repeating work off of the plate of the data team.

The simplest. but sometimes most effective way to enable self serve analytics is to build great dashboards and make sure your team knows about them. If there is a question you are getting asked frequently and you find yourself pulling the same data for multiple users, then it almost always is a good use of your tine to set that up as a dashboard (and schedule it out if appropriate)

The next level is simple user engagement with your BI tool. The most common use case for this is being able to manage existing filters on a report (ie change the region from Americas to APAC). Basically everyone you work with can get to this level of fluency and it can help make your life a lot easier to train people to use filters. Another aspect that’s along the same level is changing date aggregates (being able to switch a report that is quarterly to a monthly one without having to ask us). A half step up from this would be switching out a single dimension in a looker report. This level is really great because it cuts down on a large amount of requests and is also really difficult for end users to mess up.

The next level is the first level of what I’d call “power users” and its when users feel comfortable building their own reports in your BI tool. THIS IS A DANGEROUS LEVEL. While its great to have end users building out their own reports, you’ll often find nuances in the data that they have missed purely by not being as close to it as you are. Our process for this is to allow users to build their own looks and dashboards, but we ask that they are all run by us before being put into use. It’s best to focus on domain expertise here - so to get your marketing team comfortable using a few different dimensions and measures that particularly affect their team and have them come to you with anything else. In the long run you gain some modest time savings in the actual report building, but a lot of the true value at this stage is in increasing the data fluency of your end users. If you’re using looker, it’s helpful to have some stripped down explores for users on this level.

The final stage are your true “data champions”. These are the people who know the data model almost as well as you. Not only do they build reports right - they surprise you in finding new insights in the data and new avenues for exploration. Treasure these people.

As for what level of self serve to expect and how to get it going, I’ve found that it all comes back to trust and relationship building. People won’t really read data dictionaries. It’s much more important to have good example dashboards and reports for them to build off of. Training people in the basics of your BI tool can go a long way towards them being able to organically pull their own metrics.

As far as what’s normal - I don’t think there is really any “normal”. Whatever level works best considering your team, the technical literacy of your business users and the complexity of your data will inform what’s right for your organization. Ultimately, the best thing you can do for self serve data is the same as the best thing you can do for non-self-serve analytics, which is focus on having a comprehensible, well maintained data model (if only there was a tool that was good for that…)

1 Like

What does “self-serve” in analytics mean to you?
In my view, it’s imperative for domain owners to “own” their own data. As in, they need to know how to access the basic performance metrics that relate to their domain and be able to perform simple analysis to understand how to make decisions that they encounter on a day-to-day basis. Of course, you can’t expect a marketer to run a regression analysis, but they should be able to find basic data about the performance of their campaigns (how many transactions did my email drive?) and be able to dig at least one layer deeper (how much was driven by different pre-defined customer groups?).

Ideally, business users would be able to construct their own basic reports and dashboards, but in reality, there will likely always be a few “power users” across functional teams who can assist their teammates - with final review from the data team. And, of course, there will be times where an analyst is needed to set up new reporting, which the end user can then access whenever they need it.

All of this depends on having an analytics team to make sure the data that’s surfaced to end users is as clear and easy to use as possible, is accurate and reliable, and is maintained as the business changes. The priority should be automation and iterative improvement, so self-service for the basics can be maintained into the future.

How self-service is analytics currently at your company?
Not very self-service at all. We have an unfortunate combination of data access issues combined with not a lot of data literacy.

We’ve been stuck in the era of Google docs that aggregate data from Looker reports and compare to targets that are housed in Excel. We have Looker built out, but it’s overwhelming for end users, so only a handful of people actually use it. This is partially from poor set up and maintenance within Looker and partially because of the poor schema design of our analytical data warehouse in combination with compounding pressure from a changing business. We’ve recently launched Metabase, but our operational databases are too complex for a non-coder or non-analyst to navigate, so we’re stuck waiting for our new data models to be built before we can really open it up.

Another side-effect of being stuck in the era of excel is that there have been only a few people who have built and maintained reporting over the years, as there are only a few who know how to work in Excel. This has meant that the company as a whole has developed a reliance on these few people to get all of their data and reporting from. Lots of “where did this data come from?” “why don’t these numbers match?” “why are we up/down compared to target/LY” that those few people have been answering because they’re the only ones who knew how.

How self-service do you want analytics at your company to be?
Very - for the simple stuff. Retail is not rocket science. Every person at the company should be able to build a basic report if they need to. Key people/teams (product, marketing, social, planning, merchandising, operations, tech) should be able to do surface-level analysis to inform future testing/improvement.

An example - marketing should be able to analyze creative performance reporting and assess the success of their campaigns (potentially in partnership with planning). A marketing analyst/data scientist can help with things like building an attribution model and identifying new triggers for marketing automation (things a marketer can’t do on their own).

The value analytics can bring is in creating data models that are easy to understand and use, maintaining view-level platform(s) for end business user use, doing analysis that requires a bit more than slicing the data via an existing attribute, and helping to connect the dots across the entire org (facilitating collaboration and communication through the use of data). An analytics team can’t do that if they’re bogged down doing simple analysis that a marketer (for example) could be doing (and I would argue should be doing).

What are the barriers you’ve encountered to creating an empowered, self-service culture?
Once someone has developed a dependency on someone else to do something for them, it’s really hard to break that habit, especially if you are pushing them to learn a new skill set. You just have to keep advocating for data, sending them the pre-made reports that you’ve created, and showing them how to do things instead of sending them the end result (if it’s simple enough). Then they’ll build confidence, you’ll have built a collaborative relationship with them, and the trust will be there to build on going forward. Delegation is also important - if you’re doing something that isn’t driving value (copying numbers into a google doc), then try to find the relevant domain owner and give them the link to the Looker report or find a way to automate it away so no one is spending their time doing that.

Another challenge has been understaffing. Due to the difficulties with our existing ETL (among other reasons), our data team has slowly dwindled to two (soon to be one) analysts and a lone data engineer. There’s only so much you can do with a data team that represents 1% of your global company and supports three markets, while you’re trying to simultaneously prop up your dying ETL and build a new one. We’ve been hard at work building out the team roadmap, laying the foundations of understanding the value of data and good infrastructure, and advocating for additional resources.

Beyond that, for us, it’s really just been about the difficult data models. But that’s in the process of being fixed thanks to dbt + Fishtown and increasing support from leadership to shift focus toward the project.

How important is data documentation in creating and maintaining a self-service culture?
It’s very important at Birchbox, as our operational databases have nuances. We’re trying to find a balance though, as not everyone likes to use documentation. Keep the documentation within the data models rich for more in-depth users and future team members, but keep it light and simple for end business users.

Is there anything you’ve found to be particularly useful in creating a self-service culture?
Persistence. Advocacy. Friendliness. Compassion. Trust. Integrity. Transparency.

Tell people about the work that your team is doing and how it will benefit them. Listen to people when they’re telling you about the questions they have or the problems they’re trying to solve. Be empathetic toward them. Admit when mistakes are made and be clear about how you’re correcting them. The people relationships are as important as the data itself in getting people to buy in. Constant reminders that you exist and are delivering value alone is powerful.

2 Likes

I’ve been working on “self-serve” at Dialogue a lot over the past few months and I think designing for it really comes down to this question. The other questions are really interesting, but for brevity and poignancy, this one covers a lot of ground. Ultimately, this question is what has helped us separate the problem into two parts:

1. Technical expertise with SQL and data modelling: These users know what data they’re looking to analyze, they have all the definitions and business knowledge to extract insights, but they don’t know SQL. We’re solving this with tools like Metabase and more and more denormalization of data. Medical team members, for example, have lots of expertise but no experience with joining data. What we do to serve these members is take applications events and enrich them with dimension and facts about the care the patient had previously received.

2. Domain expertise, an inability to discover data: the more complex problem is when a user, such as a product manager, has a business question that they want to translate into a data question. For this case, we’re still testing solutions but what we know is this has to do with surfacing the right data and making it usable for the end user in a seamless way. Right now we’re thinking about Lyft’s Amundsen and DBT Docs and a few others, but this problem is much harder to solve with a tool. In the mean time, data office-hours and trainings are helping fill this hole.

2 Likes

What does “self-serve” in analytics mean to you?

As mentioned it means different things to different people and I recognise its ambiguity and definitional issues.

Currently, I think of it as infrastructure that allows business users to easily access, engage and query appropriate data from the warehouse (semantic or reporting layer) and customise or augment data from warehouse with their own data to create a new curated set.

I also see it as a spectrum as a opposed to a binary.

The infrastructure I spoke of consists:

  • Analytics tools (e.g. Tableau, Amplitude)

  • Context/documentation (e.g. data dictionary, analysis wiki pages)

  • Resources for learning (e.g. courses, workshops, office hours)

Thinking about data culture and where it fits into self-service is an interesting question. Perhaps we can say a strong data culture increases the take up rate of self service analytics.

How self-service is analytics currently at your company?

Given the companies maturity and stage, confident it’s above average.

Currently we are using Tableau and Amplitude as our two key self serve tools.

In terms of other infrastrucutre we run Tableau workshops, provide data office hours, and hold a community of practice every 2 weeks for all business users interested in data.

We have light metadata on our data (on Tableau Online and YAML files) and interestingly no data dictionary as yet. Although it is something we are starting to consider.

How self-service do you want analytics at your company to be?

Make progress in getting business users comfortable so that a portion of ad-hoc requests can be done by themselves.

Also getting more predictive with our work. That is a number of functions in the business are in a good position to predict what will happen based on their data.

What are the barriers you’ve encountered to creating an empowered, self-service culture?

  • Lack of data literacy and skills amongst business users

  • Unaware of data that is available

  • Keeping with the status quo

How important is data documentation in creating and maintaining a self-service culture?

As mentioned before have done little documentation, so perhaps a little unaware of the importance. Will have a better

Sense when we do increase documentation and notice (if any) difference.

But we have paid close attention to naming of events, dimensions, measures, tables and columns very very carefully.

Is there anything you’ve found to be particularly useful in creating a self-service culture?

  • Data Community of Practice

  • Weekly Business Reviews with leadership

  • Onboarding workshops for BI tools

  • Office hours

Currently working on a Data 101 course that covers data warehousing, data sensemaking (asking the right questions, systematically analysing the data from a number of angles, presenting results etc.), experiments, BI tools and more. Thinking it could be a mixture of in person delivery and also online course content.

1 Like

TLDR: self serve analytics is important but is insufficient on its own. It is not a promised land of data informed decision making.

Agree with the thinking that has been shared here so far.

For me the shift in my thinking that has taken place over the past few months as our self serve infrastructure has come together, has been that it doesn’t remove the need for data analysts and the skill of data sensemaking.

With my data warehousing background I have always valued well structured data sets and quality dashboards, and mistakenly felt that if we could achieve that, a deluge of data analysis and insight would flow from that.

I still believe they are important and necessary however what I’d acknowledge is that most “business users” like product managers have a lot on their plate already, and beyond understanding / reporting on their domain, it’s unlikely most of them will be able to dedicate the time necessary to deeply explore their data or make sense of it. That is where dedicated data analysts with specialised skills in the space can play a role, to produce valuable insights that won’t pop out of your standard dashboards or simple cross tabs and line charts.

My thinking has been informed by Stephen Few’s thoughts on the matter, but I’m only really grokking it now.

See https://www.perceptualedge.com/blog/?p=2467

1 Like