Transcript:
Hi, I’m Dan Elman. I’m a senior solution architect at Stitch, and today I’m going to show you how to use data inside of Braze together with Claude to investigate a lifecycle problem.
Meet Naughty Nuggets, a fictional chicken-serving quick service restaurant that offers delivery and pickup via their app, integrated with Braze. They recently launched a new acquisition campaign, new members are pouring in, and first orders look very healthy. But the CRM team has flagged something problematic: their second order rate for new customers at the 14-day mark has fallen from 34% to 26.4%. So what’s the underlying issue? Is it the message? The offer? The creative? The canvas? Something else? Let’s put our detective hats on and figure this out with Braze data and Claude.
To get to the nuts and bolts of the issue, we need to know what evidence Braze can give us. Braze offers many different data extraction points, but I don’t have access to Naughty Nuggets’ data warehouse, so Currents and Braze data sharing are off the table. Braze MCP and REST API endpoints will give me raw data, but only channel, campaign, and canvas data. Similarly, Braze reports and dashboards give me campaign, channel, and canvas data, albeit in processed form — but they don’t give me the purchase-level behavioral data, the event data, that I really want to access.
So the obvious extraction point is Query Builder. Here we can access order events as they stream into Braze via Naughty Nuggets’ app SDK. Every order event includes properties such as order value, item count, bundle type, and time of day or day part. We can query these directly to understand customer behavior, and we can ask very precise questions using SQL against this raw event data. Let’s take a look at my first prompt for Claude to see how I’m going to attack this.
You can see this is my first prompt for Claude. It’s quite sizable, but it has to do quite a lot. I’m not going to write the SQL by hand, because Claude these days is very adept at writing SQL — much faster than me. So I’ll tell Claude in plain English exactly what I want to know, and let it write the queries for me against Braze’s raw order data.
I’m asking for three things. First: has this decline suddenly fallen off a cliff, or has it been more of a slow slide? We’ll look at this week by week since the campaign started. Second — and this one is very important — break down customers by their first purchase traits. Here I want to know how many items they bought on their first order, whether it was a solo order or a group bundle of some sort, how much they spent, and when they ordered in terms of time of day. The third query splits customers into two groups, comparing the first half of the acquisition campaign versus the second. This shows me whether the journey — the canvas, the onboarding flow — is actually broken itself. If all four ways of defining the customers show equal behavior, the flow is fine. If they’re not equal, something happened at that point in time in either the first half or second half of the campaign.
Claude ingested this prompt, understood what I wanted, and gave me a caveat: Query Builder can’t see past 60 days, which is a known limitation within Braze. Now I’m going to switch into Query Builder to show you what I did with this SQL.
Here we are inside of Braze — this is Query Builder. If you don’t know how to access it, go to Analytics and Query Builder. I’ve created a custom query — this is query number one, weekly trend of the 14-day second order rate, and there are 63 lines of SQL code within this query. I’ve run the query and have an output I can literally copy and paste, since it’s only seven rows. I’ve done the same for query two and query three. Let’s go back into Claude to see how it reacted.
Here you can see the results I pasted back into Claude — query one, query two, and query three. Claude has given me insight and analysis for all three queries. It produced a scenario table with a breakdown across the blended 14-day rate for the early half and the late half, and gave me, in plain terms, its diagnosis plus two things worth chasing next. I’m not going to go through everything line by line, but here’s a high-level summary.
The first query shows that onboarding hasn’t fallen off a cliff — it’s been a gradual decline. So it isn’t any single incident at any point in time that caused the second order drop. The second query demonstrates that there are four groups of customers converting poorly by day 14 for a second order: people who ordered group meal bundles, dinner buyers, those with four-plus items, and those who spent more than twenty dollars. That indicates there’s perhaps been a change in the mix of customers coming through the acquisition offer, and that’s affecting the overall rate for second order timing. The third query confirms there’s been a shift in customer mix: new customers are increasingly starting with a group meal dinner, four-plus items, or spending more than twenty dollars, rather than the typical solo order that’s been the hallmark of Naughty Nuggets’ acquisition campaigns in the past.
So we’re seeing a new phenomenon, and we have our suspect: acquiring more group dinner customers than is typical. But before we jump straight to a recommendation or a fix, let’s find out a bit more about these customers. For example, how long does it take, on average, for them to make a second purchase? If we know that, we can provide recommendations to adapt the onboarding approach in a canvas, and also adapt the analytical approach going forward.
So I asked Claude about their purchase cadence — here’s my follow-up prompt asking for two different ways to look at that. Claude gave me some SQL to use, I ran it in Query Builder, and pasted the results back. Here’s that result: the answer was “well, it’s both, but mostly the first.” Let’s unpack that, because there’s a lot in this response.
Claude found that these customers are slower to order a second time, and they also convert at a lower rate than solo buyers — which is interesting. Group dinner customers take a lot longer than solo customers to place their second order, but it’s not purely a timing story. When we tracked both groups all the way out to 35 days, the gap between them didn’t close. Only about a quarter of the group dinner customers had reordered by then, against roughly half of the solo buyers. So they’re slower by half, and genuinely less likely to come back at all inside that 35-day window.
There’s actually a name for what we’ve just found: it’s a paradox — Simpson’s Paradox — where every individual group within a cohort is doing fine, or even better than you’d expect, and yet the overall number you’re tracking falls. That’s exactly what we’re seeing here, because the makeup of the group changed underneath it.
So now we have a diagnosis, and a better understanding of the customers, which puts us in a position to make some recommendations. Let’s go through those for Naughty Nuggets.
We have eight recommendations in total, covering measurement, data, journey, and experimentation. First, on measurement: we’d advise devaluing the historically blended 14-day rate as the measure for second purchase, and instead splitting out customers by their first order occasion. If you can identify that a customer made a group dinner purchase, record them as such, and extend the window for evaluating second purchase conversion out to 35 days.
On the data side, it would be great to apply custom attributes to customers who are “group dinnering.” If you can assess, through event properties, that they made a group purchase, put that on their profile — then you can segment them into that group and personalize based on dinner ordering, four-plus orders, or ordering for family and friends, and use that attribute effectively. Also, if they want to evaluate the current acquisition campaign against previous campaigns, it’d be great to use Braze Currents to send that raw tracking data to the data warehouse, or use Braze data sharing for a similar purpose — giving them an extended evaluation against longer-term history.
In terms of adjusting the acquisition onboarding canvas, they ought to consider sending messages at different times for different groups. Since group dinner customers take much longer to make their second purchase, they should probably be nudged around day 20 — though that’s certainly worth testing. We recommend doing some A/B testing around message timing, and also putting in place a holdout group or control group who receive no marketing, using their behavior as the baseline for measuring the impact of marketing through Braze messaging.
So that’s how you can play detective with Braze and with Claude. If you’d like to learn more about how we can help you uncover the behavior of your customers, please reach out to us at stitch.cx/contact.
Meet the speaker
Dan Elman is a Senior Solutions Architect at Stitch who helps brands turn customer data into relevant, effective engagement through Braze, designing the integrations and personalized journeys that connect strategy with technology. He holds 5 Braze certifications, including Technical Marketing Expert, Developer, and Marketing Strategy Expert. Outside of work, he’s the proud owner of two Bernese Mountain Dogs. Connect with him on LinkedIn.
