Real Life Examples of How To Blend Survey & Analytics Data with Matt Bahr from EnquireLabs
Learn how to blend survey data with analytics data, where customer survey data fits in attribution analysis, and real life examples of brands that have extracted unique insights from survey data to shift marketing messaging that led to unanticipated growth.
Brad Redding sits down with Matt Bahr, CEO & co-founder of EnquireLabs, a post-purchase survey solution for Shopify merchants, applying speed and scale to survey data. They take a deep dive into blending survey data with analytics data, where customer survey data fits in attribution analysis, and real-life examples of brands that have extracted unique insights from survey data to shift marketing messaging leading to unanticipated growth.
Q: Can you give us a breakdown of what is survey data, the role it plays and in DOC brands?
Bahr: Survey data is another data source that customers have been getting excited about over the last few years, given all the limitations of tracking and capturing data in general. We actually call it Direct from Consumer (DFC) data. It’s part of that flywheel where you’re marketing to customers. So, on the DFC side, we talk about survey data, first party data, whether it’s the data you’re capturing or not, under this umbrella of DFC data. That’s data that you’re capturing directly from your customers. The most exciting thing about it for us is that it’s very resilient to changes – whether it’s privacy regulations, the way people just essentially operate and scale brands, or a new platform is introduced – it works across every channel.
That’s how we view survey data. Our market differentiation is that we want people to think about the data that we’re capturing through our question stream, which we can get into, comparable to data that you’d see in Google analytics. Traditionally, we run a survey quarterly, wait 3 weeks to get any of the data, have someone slice and dice it in Excel, then we get a PDF report and we go make a decision. That’s essentially where survey data has always lived. We want our customers and market to think about survey data more as an ongoing timeline, a data stream of insights that can we can use in real time.
Q: Before the iOS and privacy wave, what was the primary need for the post-purchase survey? What was the driving force and need that you were trying to fill?
Bahr: Initially, we built Enquire V1 for a handbag company in New York called Kara Sport because they have a high average order value. For the path to purchase, you couldn’t use direct response marketing where somebody clicks on an ad and buys and you have a really clear linear path to purchase. We were looking at the time lag in GA (Google Analytics) which doesn’t really tell you anything – it tells you 80% of people buy right away, which you know is true. So that was the initial use case – how do we help this brand solve attribution? This was in 2018 so privacy issues weren’t really at the forefront at this time. We were trying to figure out how to give clarity into influencers and word of mouth for this brand.
We didn’t just rely on Google forms, which was the initial implementation we recommended. A lot of these existing platforms lived in an eye frame on the page and didn’t connect any of the order data. So, you’d get this meaningful data, but you’d have no idea who submitted it. Those were the two main things that was the driving force for us to build, at the time, a very basic post-purchase survey application.
Q: So, you look in Google analytics and see an order come from Meta paid social and you look at the post-purchase survey. You ask the customer, ‘how did you hear about us?’ Are you trying to validate that the user said Meta or potentially a different channel?
Bahr: Yes, exactly. For example, you have your Meta prospecting campaigns, which essentially means a lot of the targeting is ‘the customer has never visited our site.’ Meta loves to take the credit for those conversions that they provide because it’s prospecting and they’re getting users. But at the end of the day, a large portion of customers actually came from word of mouth. Meta is finding them because somebody is searching something and there’s some signal within the Meta platform that’s putting them into this prospecting campaign. And without asking customers, you have no clarity. It might not be the most actionable aspect, like this was word of mouth instead of a Meta awareness, but very good from allowing marketers to build that mental model of what is actually happening, how are customers actually discovering and finding our brand in purchasing?
Q: Fast forward to where we are today, a significant change has happened in the last 10 months. What are you seeing brands do with the data they’re collecting from Enquire compared to what they might be doing in GA or other attribution platforms? Has that changed? Are they still asking to validate that their prospecting campaigns are working and where they’re coming from?
Bahr: Yes, it’s a similar question, but I think brands have gotten more sophisticated in how they use our data. It used to be we open the Enquire dashboard, look at it once a day and get directional data. Now it’s ‘how do I pipe this data into my data warehouse and allow my data science team to dive in and pull key insights?’ So, it’s definitely evolved, which is the path we always wanted to go. We’re in the process of releasing the responses endpoint to allow people to programmatically use this data anywhere to help fill in the data gaps and provide some clarity.
Q: Let’s use that use case where you have an analyst that’s piping your data in and they’re trying to extract the insights. What are the data dependencies that they have? I assume they’re not just looking at your data alone, they’re potentially blending or pulling other data sources.
Bahr: Some of it depends on a per channel basis. If you’re trying to dive into influencer, podcast attribution, our data is definitely at the forefront of those models. It’s probably the most heavily weighted source of data and that’s mainly whether there’s a link or not, it’s just very difficult to track with clicks. So, it’s definitely channel dependent. GA is still the holy grail. I think how people are marrying data types or data sources, especially with our data, our Google analytics integration is by far the one that’s enabled. We encourage people to enable it from the start to get the data in there.
The other component, just thinking about the data sources, is where a brand is in scale. Brands that are over $75 or a $100 million in top line revenue, their data stack looks very different from somebody that’s doing $15 million. It runs the gamut of what kinds of dependencies there are, but at the end of the day, there’s no secret data source. Elevar is a key player in increasing the accuracy of the data, but it’s not like we’re pulling data that didn’t necessarily exist before. Everyone has access to the same data for the most part. It’s really how you can analyze it and formulate your own directional opinions with what it’s telling you.
Q: With our customer, we crossover 80%. Why should qualitative data be used for attribution analysis when it’s set up with 80%? And where there might be data gaps?
Bahr: It’s really just to fill in the data gaps and the nature of a path to purchase. Someone might come to your site 10 times before they purchase and they might do it on multiple devices. It’s just not linear where you can say these were the 10 sources and there’s no way to solve some of these things. If somebody recommends a product to you via a word of mouth, there’s no click trail. So that’s definitely the highest utility point – to fill in these data gaps to paint a clearer picture. The other component is self-reported from your customers. If you think about Meta’s attribution, they’re using a probabilistic model to tell you, you can’t see the probability of a conversion at this point, but most likely what the ROAS is. And that sometimes feels so random, where this data doesn’t have any probabilistic nature in the output. You’re able to sample your entire customer base and deduce confidently without having to rely on third parties. Survey data is just very easy to read as a marketer. You’re making hundreds of decisions a day- building out that mental model is so important to allow you to make micro-decisions throughout the day. And the survey day is so easy to understand and read. That’s the other component of where it fits in the stack.
Q: What about multi-touch as that’s a big attribution question – where do I use my top of funnel dollars, should I be investing in remarketing? How do you consult your customers to leverage your data and their multi-touch analysis?
Bahr: From an attribution standpoint, we definitely focus on that moment of discovery – how do you hear about a brand? From an attribution modeling perspective, I always recommend people look at position based – what’s the first touch, what’s the last touch? Given the path to purchase could be quite long. It could involve ten or twenty touches. It’s very difficult to take action on that as a marketer, unless you’re doing it programmatically. That’s where you guys are doing more of the multi-touch attribution that will give you a weighted number per channel. But it gets very hard to interpret and to make decisions from. We stay out of multi-touch – we like asking questions like, ‘how long ago did you hear about us,’ which would probably signal some accuracy components of your attribution survey. From a multi-touch standpoint, let them solve those problems and do it programmatically and you can take action from that. And let us focus on first touch. In our platform, we show you first survey response and then we compare it to referral source or last click and UTM parameters that actually drove the order.
Q: What’s that number where you’d say, ‘hey, if you’re doing less than X, here’s what we recommend. Take the ‘how did you hear about us,’ recency questions and blend that with this report in GA, and here’s how to take action on that. Can you start with the revenue number you see most and then how do they take action?
Bahr: Think about attribution as a step function manner – marginally. When do you need to think about it marginally? When you’re using an MTA solution when you’re trying to increase accuracy by-5-10% on the margin. Do you have the foundation internally without using a software to understand and build out some form of an internal attribution model? That’s what we always recommend. From $0-5 million, you could definitely get away with doing that and having it all internal. As you’re looking to level up on the margin, that’s when you start thinking about different solutions because if you’re not spending enough, you’re not going to be able to extract enough from these platforms to actually get a high ROI.
From a revenue perspective, it depends on the level of success on a per channel basis. If you’re doing $10 million and you’re not excited about attribution in your return, it might make sense to lean into one of these. We see brands doing $30 million who don’t have anything, use a post-purchase survey platform, use platform reported metrics in combination with Elevar and they’re scaling just fine. It really depends on a per store basis and not necessarily a revenue standpoint.
Q: A customer might have their Google Analytics data, EnquireLabs data and they might be using another attribution tool or their own data warehouse. What friction do you see when their survey data doesn’t match at all?
Bahr: It’s extremely complimentary. Everybody wants their survey data inserted into their multi-touch attribution (MTA)solution. Fortunately for us, we don’t see, ‘do I use Enquire or do I use an MTA solution?’ People are always asking how to get data into the multi-touch attribution solution because we’re pushing the data into Rockerbox, Northbeam and others. These platforms are excited to get our data because everyone is aware that they can’t track podcasts well with click or pixel data. But these attribution solutions are transparent and excited to get our data because they know they can’t solve it any of their way.
It’s interesting to see what TikTok reports from an ROI perspective and how that compares to Enquire. It’s usually under reported. Enquire is going to tell you a better story from your customers so it’s not using some black box. How is it getting this better number? People are just telling you or they heard you from this platform so it’s always interesting to compare.
Customers are thinking about Meta as a billboard in the sense that we just spend to spend. If we were to run a regression on Meta spend and revenue that are correlated and just treating it more so in that sense versus the direct response strategy. That’s how everyone did it two years ago, and I think that’s going to continue to move in that direction, given the changes that are happening with Meta.
Q: So, a customer takes a hundred dollars that they were spending, $80 on Meta, $20 in TikTok. Now they’re taking that money and spending $40 on Meta, $30 on TikTok and $30 on content or influencer content generation that can enable that virality on TikTok.
Bahr: Exactly. We see customers creating programs where ‘if you go viral, we’ll pay you X.’ And anyone can go viral on TikTok, meaning more than a half a million views on a video. That’s the early areas of success – we saw people using Enquire to solve attribution on TikTok without spending a dollar on the TikTok ad platform.
Q: It sounds like brands still need Google analytics or an analytics platform. They still need their quantitative data to use, to compare to their survey data and potentially what they’re getting from their attribution platforms.
Bahr: Exactly. It’s an interesting note about quantitative versus qualitative data. A lot of the data we’re capturing, we categorize as quantitative. Qualitative to us is essentially an open-ended text box that will help tell a narrative versus the quantitative survey data is more or less like ‘select this input, select this option on the Ford’. Then it starts to look very similar to your Google analytics data. I advise brands to look less on the conversion number in GA and look at all the other directional data points.
The best example is when I could tell much how much you’re spending on Meta based on a $2 cost per click, which is probably a little conservative. Let’s say you have 20,000 users or sessions coming to your site, let’s call it $40K. Without looking at the conversion numbers, if you’re bounce rate is over 75% for paid traffic, you should really think about how your targeting customers or improving your landing page experience, which typically isn’t the issue initially. GA is so important. I could tell you how a brand is doing without even looking at any revenue numbers and by just looking at bounce rate and channel sources. That’s when it becomes an invaluable tool, when you can understand user engagement on your site as accurately as possible, without worrying about funnels.
Q: Any other real-life examples that you want to share from brands that you’ve seen take action between their data and decision making?
Bahr: What gets us most excited about attribution is understanding which channel is driving the orders for our customers. Everyone thinks about attribution from a return perspective. Understanding channels is one thing, but how do you improve a channel? How do I actually get my ROI up on a channel?
The path for the last five years has always been creative and bid strategy. We get super excited when customers ask questions that you wouldn’t necessarily categorize under attribution like, ‘how would you classify yourself? What are you using our product for?’ Something that provides much more clarity so they can use that to increase their ROAS because now they’re going to talk to their customer in the way the customer wants to actually be spoken to.
One of the best examples we have is 4 by 400. One of their brands was a car wash for dirt bikes. Their CEO, Andrew, asked his customers what they were using our product for? He learned that all of his customers were families who owned ATVs. But all of their ads were about dirt bikes, motocross type content. He had no idea and all he did was change the creative and then the ROAS went through the roof. It’s a different strategy versus keep test and creative, keep optimizing bid strategies. This is even more important now. One of the brands that we’re working with on the TikTok side, their customer used to be 32- to 40-year-old males. TikTok is now their largest source of traffic and it’s 18-22-year-old females, same product.
Q: Any last tips on how brand should approach decision making? If they’re getting started, maybe they’re using you or potentially another tool, but not really using it. They have you installed, they’re collecting data, but they’re not using it. Do you have any tips on how to get started without being overwhelmed?
Bahr: I think the easiest way to think about this is to start with some open-ended questions that you didn’t even think about asking like ‘why did you buy today?’ Take the ATV example. Even if it was a single response or just open ended, it allowed people to say I’m a family. I go riding on the weekends and it’s an ATV. For brands just getting started, that’s what we really encourage – adding a couple more questions, just get some insights and allow yourself to get this other data source you never had before. Our product is very sticky when someone first starts using it because it’s a whole new data source that they’ve been trying to understand. So, that’s usually our recommendation.
We also have something called ‘customer level attribution framework.’ When you think about attribution, don’t just jump into a platform to tell you what your ROI is. That’s definitely one method and what all your competitors are most likely doing. But what you should think about is what you can learn from each customer and each order that will allow you to build a model from the ground up. That’s similar to mentioning age and gender. If I knew the age and gender of every customer, how would that help with attribution? It might not help with the accuracy of attribution, but it might allow you to target channels differently, which allows your ROI to increase. It’s more of an optimization mechanism and thinking about a bottom up versus top-down approach. When thinking about modeling and building that competitive advantage with data, it’s so important to think about it versus just thinking about in terms of monthly ROI. Think about how you can take action from what you’re seeing.
Q: That sounds amazing. That’s probably another hour episode on its own. But Matt, thank you so much. You provided so much amazing knowledge. Anything else you want to share before we wrap up?
Bahr: Yes! If you ever want to chat or want us to help build some models, dive into a platform or Google sheets or Enquire, we’re always happy to do that. We encourage our customers to send us whatever they’ve built internally so we could help build a better product on the survey side of things. Feel free to email me at [email protected] or hit me up on LinkedIn or Twitter.