Friday, September 18, 2015
Gone Monetate : Personalizing Marketing at 100K events/second
Internet advertising has gone the way of junk mail. There are tons of impersonal advertisements sucking up screen real-estate and mailbox space. The majority of the advertising industry leverages only inane intelligence that assumes, "because I clicked on a rubber ducky once, I must want to buy a rubber ducky". In fact, I probably want to see rubber duckies on every site I visit, which is the internet equivalent of a mailbox full of refinance offers the second I have my credit checked. I'd much rather be shown things I care about. Even better, show me stuff I didn't know I cared about!
That is the reason I joined Monetate. Well, it was that combined with the opportunity to join a rock-star team of passionate geniuses, working on an incredibly challenging real-time data and analytics problem, hell-bent on changing the world.
I've only been on the team for a week, and I've been blown away. Directed by Jeff Persch, the development teach has architected a phenomenal platform: scalable and resilient, with simplicity as a core tenant. With Monetate at the center of many of the e-commerce interactions, influencing almost 1/3 of the dollars spent on Black Friday, that is no small feat. We are gearing up for another black friday, for which peak traffic may hit 50-100K events / second!
Doing some math on that, the numbers get big really fast:
100K / s = 6M / minute = 360M / hour = ~8B events / day
Yep, that ended in the B word... per day. And like everyone else, we are pushing to deliver better, faster, and more insightful analytics on those user activity streams... in near real-time.
And because of that, I hope to have many useful/fun blog posts coming.
Right now, we're in the process of rolling out Druid, a favorite technology of mine. (and something to which we dedicated a whole chapter in our Storm book -- (shameless plug ;)) With Druid in place, we'll be able to slice and dice the real-time events.
And we already have the ability to ingest user context in real-time. When you combine that user context with stream processing that allows us to identify behavioral trends, our decision engine can make contextual decisions, adapt on the fly, and deliver relevant/interesting content to the end-user!
Imagine if we could only do that for the real world too. Instead of coming home to a mailbox filled with junk mail that I immediately throw awawy, I'd have updates on the latest gadgetry and fishing paraphernalia. Hmm, maybe I can get that on the roadmap. ;)
Anyway, I couldn't be happier with my choice to join Monetate. The people, culture and technology are astounding.
Stay tuned -- likely my next blog post will show how to leverage Amazon's Kinesis for 1/100th the cost. (by aggregating events into a smaller number of Kinesis records, decreasing the overall message rate, you can save mondo dough =)