How To Measure Attribution Beyond Meta & Google + Data Team Organizations with Ron at Rockerbox

Brad Redding chats with Co-Founder & CEO of Rockerbox, Ron Jacobson. They go deep into the world of OTT, CTV, podcast and other data and attribution lessons that brands like ROTHYS, FIGS and other Rockerbox customers are solving for and when to start moving marketing budgets outside of Meta and Google.

Q: Give us a quick overview about Rockerbox.

Jacobson: Rockerbox is the leading attribution provider for direct-to-consumer brands. We aim to help companies from seed to IPO to measure their marketing, whether it’s paid, organic, digital or offline. They come to Rockerbox to be a source of truths across all of their marketing. And there’s a lot of work that goes into that to make it happen.

Q: We have many mutual customers. Where do you feel like your primary customer cohort is? Who do you serve?

Jacobson: It’s definitely evolving over time. When we first launched Rockerbox, we were going for that mid-tier direct-to-consumer brands who were spending $5-20M a year in marketing across multiple channels. We realized that those brands scale, they get larger and larger and we needed to keep up with their growth. So, our product needed to evolve. We needed to become good for them as they’ve scaled. We spent a large portion of our time and energy making sure we can go upmarket with those brands and things like data warehousing, customizability and transparency. Over time, we’ve also had a quarter of customers that are coming to us earlier at a million or lower, even at a couple hundred thousand.

Internally, we think about it from seed to IPO. Our goal is to be able to have marketing management technology that works for brands throughout their life cycle as they scale. And that changes, it varies what they need during the different parts of the life cycle. That’s how we think about our customer base. We’re focused on B2C digital brands, companies like Figs and Rothys to Bergen and 1-800 flowers are the types of companies that would work with Rockerbox.

Q: Let’s look at Rothys, Figs or a similar company. What does their team structure look like? Do they have in-house analysts, digital marketers?

Jacobson: Great question! That’s really what correlates the most to where they are in their journey. When we’re working with a brand like an early-stage Shopify brand, they’re primarily on Meta and Google, maybe  doing a little bit of influencer or affiliate. It’s really just a marketing team. You’re talking about two or three people who are doing a bit of everything. There’s a head of marketing, maybe someone focused on paid, organic, but it’s a small team. There’s no data science org, analysts or BI framework. That’s the foundation and they’re spending a lot of their time in platforms like Meta, Google Analytics and Excel.

As they scale, the marketing team gets more sophisticated. It gets more verticalized inside the marketing org across different channels. Then you start getting an analyst who can do sequels, DBT, APIs and trying to automate what they’re doing.  As that scales, you’re talking about data science and formal BI organizations. We even start to expand into finance and operations, because they get to a point where understanding the impact of marketing has real business implications for how they operate as an organization. What does spending an extra million dollars on TV mean for our sales in one week, two weeks and beyond? That’s when

Finance and Operations start getting involved. As the organization scales the scope and scale of their marketing analytics team increases as well. So being able to support that is difficult, but it’s what we aim to do at Rockerbox.

Q: From a marketing spend perspective, at what point should a company be thinking of hiring an analyst who can start building this infrastructure?

Jacobson: As early as possible would be the most beneficial. The later you start to build the necessary infrastructure, the more you’re putting in place bad practices that can’t be undone without additional expenses and time. The longer you don’t have conversions flowing properly through something like Elevar, the longer that happens the longer have bad historical data. So, it really depends on the business, but I would recommend as soon as possible.

Q: One of the main questions we get from our customers is how they should get started on building their own data warehouse. What does this process look like in your experience?

Jacobson: It’s really about time and resources – a company that’s raised $20 million and is in hyper-growth mode is very different than a company that’s growing 10-30% a year. I think Rockerbox is really helping brands get their marketing data infrastructure in place so they can measure and attribute on top of it. But if you don’t have that foundation in place, you can’t do that. So, there are different approaches. For some companies, it’s getting up and running when someone like a Funnel.IO or a Fivetran, if they’re more technical or Supermetrics, just to get data flowing from the platforms into Google sheets, which is a fine starting point.

Eventually, they’re going to need a database. In the past, that was dropping off files and SFTPs and S3s and people manually processing data and putting it into a database. Now, there are synchronized databases and warehouses putting data into their data warehouse. The vendor will synchronize it and ensure that it’s constantly updated. That’s the preferred way, so brands don’t worry about getting the data into their warehouse but instead think about what to do with it. That’s where we see all brands arriving at. It’s really a question of whether they can do that today.

Q: Pitch me on what Rockerbox does technically. What does onboarding look like? What are the nuances and how does Rockerbox compare to other solutions?

Jacobson: The moment brands go beyond Meta and Google, things get complicated. Actually things can get complicated just within Meta and Google. But, you’ve scaled to a certain point on Meta and Google. Now you’re launching OTT, CTV and direct mail or linear. You’ve spent a million dollars last month on a TV campaign, you log into Google analytics and you don’t see TV there. That’s disconcerting for brand – you don’t really know what’s happening, you might see that organic search, direct traffic or paid search went up, but it’s hard to attribute what the correlation was between the TV spend and what’s actually happening and what the business impact is from that. That’s where Rockbox comes in. We provide you with the underlying data that can help answer all these questions. We provide an approach to measure channels that are difficult to measure or channels that don’t have a concrete way to measure it.

And TV would be an example of that. Even though there are pseudo ways to do this, you can’t click on an ad on TV, you can’t know for a fact that someone who saw a TV ad, came to your site or app and converted.  As a brand, you can say, ‘I’m going to go spend the next quarter hiring an engineer and a data scientist. I’m going to task them with figuring out how to measure TV.’ They’re going to start that process by figuring out how to even get that data? How do I get the data of where the TV ads were served? How do I get the session data from my site? How do I get that in one location? How do I start to think about marrying that data?

You’re talking about months of work before you can ask that question. So now I have all the data, how do I connect the dots? What’s the best approach? This is why brands come to Rockerbox. They rely on us to have the expertise, maintain that pipeline and do that work for them. We construct those underlying data sets that are needed to be able to run a TV analysis and go beyond that. They’re still going to need the underlying data to eventually run more sophisticated analysis themselves. Brands that work with us have a vendor that will  construct all the right data sets for them, they’ll have an approach to measure whatever channel they’re spending money on. Nothing is perfect, but at least they know that they’re spending dollars on something and they have somewhere to look at. And I think that’s really critical.

Q: Can you break apart TV versus video streaming (like Roku) and how the measurement actually works? And clarify some of the other acronyms you use?

Jacobson: Sure. You have OTT and CTV, which are two terms that actually mean the same thing – over the top and connected TV. That would include platforms like Hulu and Roku. Then there’s linear TV, that’s classic TV where you’re logging your TV, changing the channel – classic cable. This is a really good example of somewhat comparable places to spend your ad dollars – very different data sets available and very different ways to measure it.

For example, the most basic way to think about TV is just getting exposure data – where and when was the TV ad served? Let’s say at 12:30 in Woodstock, New York an ad was served on BBC. Getting that data set is actually a bit of a challenge because there is a delay – TV vendors often provide it one or two weeks later,  they’re called post log reports. But once you actually figure out how to get that data on recurring basis, you have to figure out how to process it and connect that to what’s happening on a client’s website. That’s where you need full session data to start to understand who’s arriving on the site. There are different ways to measure that. Did that TV ad spot drive more people to my site than normal? Was there lift? Was there incrementality in terms of the visitors to my website? Is it helping my top of funnel? That doesn’t mean those people actually converted though. This is one type of measurement.

There’s also a much more bottom of funnel type of measurement. Is it actually leading people to convert? And there’s a world where people also engage with other channels in between those different areas. That’s when you think of something like linear TV. In the OTT space, there’s more deterministic data sets available. You can get some form of impression-level data like IP addresses, user agents or device types that this would serve. If you have relationships with vendors, which Rockerbox does, we’ve done 150+ integrations the past four years since launching Rockerbox. So, if you have integrations with these vendors, you can get those data sets and do a better job of measuring that channel.

If you’re a brand thinking of exploring OTT CTV, you’ve never bought it before, you have no idea what data sets are available or how to get that data set or what the cadence would be. You have no idea. How do you connect an IP address of a TV with an IP address of a desktop or mobile phone? These are all questions that you just don’t have expertise and that’s okay. It’s not your job. If you rely on a measurement provider, they can do that for you so you can focus your time on scaling versus focusing your time on trying to get to a point where you have data to answer your questions.

Q: What are you seeing in general in privacy? What constraints are you seeing if there’s a data set from Hulu and a data set from the site and trying to marry them?

Jacobson: Over the past couple of years, there have been a lot of changes from Apple’s iOS changes to ITP, deprecation of third-party cookies – the writing is on the wall on where the world is going. We have to accept it and figure out how to deal with it. Fortunately for us, we’ve been at this for so long that we’ve seen the writing on the wall for years. For example, Safari got rid of third-party cookies around 2-1/2 half years ago. We’ve realized that we need to expect data to be taken away from us.

When we first launched Rockerbox, I was always concerned about things like how do we measure the impact of views on Meta? Meta used to give feeds of view-based data to certain legacy attribution providers. And they didn’t want to talk to me at all! They couldn’t care less. But we still have to measure that. We realized that we need to build systems to take aggregate data from platforms like Meta and leverage that to measure the impact of things that we might not be able to get deterministically, particularly view-based data. How can we build models and leverage machine learning to better understand the impact of a channel, even if we don’t have user-level data?

In a way, launching Rockerbox and not having access to data that we would’ve wanted, forced us to become really good at leveraging aggregate data sets to measure impact of marketing. I think what’s going to happen is, you go on a per channel basis, figure out what the best data is that you can get, figure out the best way to measure it and recognizing it’s imperfect.

You have a channel like TV and that’s not user-level at all. Something happened in a location, measure that. We can figure that out all the way down to paid search, which is bottom of funnel, click oriented. You can connect with dots really well. But identity resolution, figuring out how to connect different dots with different data sets, levels of aggregation, different data cadences, it’s just a really difficult technical challenge. And  Rockerbox has become really good at it over the past couple years.

Q: Do you have customer use cases that you can share? Once a customer has all their data and started to dabble TV or Hulu and Roku advertising, how are they taking action on their insights?

Jacobson: I’ll give you outlines of what has happened. A lot of customers will do multiple versions of this. Any brand that’s using measurement providers, there’s a big need for help with testing. If you’ve been steady on a couple of channels and you’re dipping your toe into something new, that’s a big challenge! The worst thing you could possibly do is spend that money and come back internally, the next week, show reporting and have nothing to show for it. Dipping toes into things like offline channels is big area where clients find Rockbox helpful.

They can suddenly say, ‘hey, I spent money in this area, I have data I can use to show the impact of it and we can make decisions. That’s definitely happened with CTV, linear to direct mail, even podcast advertising.

Another use case is around budgeting. How do we think of where to spend our marketing budget for the year? How do we think of setting that at the beginning of the year and changing that on a daily, weekly, monthly, quarterly basis afterwards? Our data sets become highly involved in that budgeting process. We’re not a services organization, but our underlying data does feed into that. We have other clients actually build their own models based on Rockerbox data to gain the impact of their marketing. They use Rockerbox to get raw session data, impression-level data and to figure out joining of a direct mail send and a conversion.

They can really start to build on top of Rockerbox. We have clients that have built their own models on top of our data sets, built their own scripts to automatically adjust bid prices and budgeting. It really depends on the use case. A huge one is Meta and iOS. How do you act on a day to day, week to week basis to get the most out of that channel? The way we measure Meta for clients at Rockerbox has been very important, especially for the past couple of years, given the iOS changes.

Q: What do you mean when you talk about testing, analyzing and making decisions?

Jacobson: There’s multiple levels of testing. You have your AB testing or inter-channel tests. Is this new set of creatives inside Meta giving me lift?  Another type of testing is more around, ‘I want to do some type of geo hold out test. I’m going to serve in a certain area and I want to understand what the cross-channel impact is.’  I’m not going to just rely on the platforms reports. Meta is viewing their own data, conversions and impressions. There’s a world where serving media in one location, actually improves your search, emails or affiliate performance. These companies are already running tests, they just need help evaluating it.

Rockerbox can say, ‘give us the input of the test you’re running and we’ll help tell you the results of that test.’ So, you don’t have to understand it. Brands are good at being creative, coming up with tests, taking results and making decisions. They don’t have to be good at figuring out what a statistically significant test is  – that’s where Rockerbox can add value.

Experiments, attribution, path conversions using post-purchase surveys, promos are a really great way to think about measurement. They’re all signals that help a brand understand what is driving revenue for them and none of them is perfect. Our goal is to give the best way to use each of these inputs to understand how to grow your business.

Q: You recently launched the Marketing Data Starter Pack. What is that designed for and how does it work?

Jacobson: These are free templates for early-stage brands spending $5-20K a month who want to be better. It may be taking them too much time to log into five different providers, get data centralized and come up with initial templates to help them understand their data and make decisions from it. It’s designed to help brands get their data into Google sheets automatically. We have prebuilt templates that answer questions for them where we enable them put in fields that are specific to their business – what are your CPA goals? What’s your budget? We can tell them, based off their inputs and what we’re seeing, where they should scale and divest. It’s great for brands very early in their life cycle. We’re happy to support that and to work with them. But I know for a fact, that if a brand is scaling in a year, that’s not going to be sufficient in two years. So, we work with brands through their entire life cycle.

Q: Do you think brands & marketers should still be logging into each platform, like Meta, Google Ads, TikTok, and trying to pull data out?

Jacobson: We definitely get to a point with our customers where our numbers serve as their guiding light and  they use Rockerbox to make decisions. I don’t want to pretend that those companies don’t also log into platforms. That’s not realistic. Even if our numbers guides their decisions. We have clients that on a weekly, monthly, quarterly basis, they give their CMO the Rockerbox CPA. But they’re still logging the platforms and it makes perfect sense. No measurement is perfect. As a brand, you need all the data so you can triangulate and make the right decisions. If you see something in Rockerbox, like orders of magnitude often in the platform itself, that is something to dive into. A brand should do that.

I’m very skeptical. I think any marketer, at any company should be skeptical as well. On my desk, I have the book by Andy Grove, ‘Only the Paranoid Survive.’ As a brand, you need to be paranoid – if something feels off, look into it. Those things that feel off sometimes are really great nuggets to dive into. If you find a really big delta between what a platform reports and what Rockerbox is reporting, that can be one of those really big signals where it’s an arbitrage opportunity and you double down. Cause for whatever reason, not same, what what’s actually happening.

Q: If you remove the need for view through data, and it’s just very heavy UTM tracking, it’s almost like we went back 10 years ago where every link needs to be tracked and building that story between these different links across channels. Should we use a combination of this plus what you can get out of GA? Can that can get us 80% of the way there? Or is that not the best route to go again,  just being very diligent on UTMs and making sure you’re piecing that story together and leveraging other metrics? Not sure what the question was here.

Jacobson: Views can matter depending on the channels and brand. If you’re a brand that has a much more visual offering, then it makes a big difference. If you’re selling apparel, beauty, jewelry or things like that, the view component definitely makes an impact. In particular, YouTube views are huge and you can go far with just clicks. Broadly speaking, think about how you connect the dots that are difficult to connect. Somebody clicks through on their mobile device then a week later, they click through on their desktop device. How do you think about ingesting data after those clicks through emails, phone numbers or whatever it is to stitch those dots together. It just gets more difficult over time. What do you want to own internally? Do you want to own internally building the processes, doing identity resolution, trying to ensure you’re properly tracking, leveraging zero and first party data to connect the dots? You can try to do that. But I think that there are companies like Elevar and Rockerbox that you should be using to give yourself leverage. That’s where we can help. If you told me, I’m either going to do no measurement, measurement with really well structured UTMs or for measurement provider, I’d rather well structured UTMs than no structured UTMs all day long.

Q: What’s your outlook on e-commerce, measurement and marketing in 2-3 years?

Jacobson: I think measurement is only going to get more difficult and privacy changes are not going away. Good brands are going to have to diversify their mix more than ever and have to spend more money on trying to capture that first party data so they can re-engage their customers and pseudo free channels, email, SMS, and things like that. And to connect dots. I think it’s going to be incumbent on brands to invest money, capture data that enables them to measure and scale more effectively. But brands are also getting smarter, becoming more data oriented and looking out in the future.

All good brands are going to need underlying data sets that enable them to be data driven marketers and organizations. I think working with technology that enables you to do that better, be it by having better underlying clean data, conversions flowing properly between different platforms or more consistent data sets, is going to be critical. The sooner you can start to invest in getting your marketing data infrastructure in place so that you can make decisions that are based off data, the better, and it’s going to become more important in the years to come.

I also think companies need to be adaptable. The methods that you use to measure today might not be in the same measure method you use to measure in 2 years. And that’s okay. You want to work with partners that can guide you through that path. Even the market’s more competitive. It’s harder for a brand to grow and scale through the tried and tested techniques they used to use.

Excerpts from The Conversion Tracking Playbook podcast hosted by Brad Redding, Founder & CEO of Elevar. Listen to the full podcast here.