Debunking Five Analyst Relations Myths

Analyst relations is easily the most misunderstood function in marketing.

I’ve been involved with analyst relations — or AR — for over a decade, working on dozens of Gartner Magic Quadrants and Forrester Waves. I’ve experienced the impact that analyst relations, when done well, can have on growth. And I know how much time and effort it takes to do it right. It’s not witchcraft nor is it a simple “spend more / do better” formula.

It’s time to set the record straight, so in this post I’m going to debunk five of the most common myths I’ve come across. Well, turns out this ex-mathematician is not great at counting, so I’ll be dubunking a bonus 6th myth as well 🙂

Myth #1: Analyst firms like Gartner are “pay to play”

Okay, so let’s start with the big one. No, they aren’t. Let’s kill this myth once and for all.

No matter how much money you spend on things like research subscriptions, strategy days, webinars, or events — you can’t buy your way into analyst reports and rankings. Companies who complain about “pay to play” are just bad at analyst relations. There, I said it. I’m going to focus on Gartner to debunk this myth, but the same concept applies to all of the major analyst firms I’ve worked with.

Consider the story of NetScout, who once tried to sue Gartner using the “pay to play” myth. NetScout wasn’t happy with its placement in a Gartner Magic Quadrant(MQ) a few years ago, claiming that:

Gartner has a ‘pay-to-play’ business model that by its design rewards Gartner clients who spend substantial sums on its various services by ranking them favorably in its influential Magic Quadrant research reports (‘Magic Quadrant reports’) and punishes technology companies that choose not to spend substantial sums on Gartner services.”

NetScout argued that competitors who spend more with Gartner ranked higher, and that Gartner salespeople implied that spending more money would improve their position in the Magic Quadrant.

Here’s how the Magic Quadrant works in a nutshell: Gartner evaluates vendors using a proprietary methodology honed over decades of research. Analysts apply this research methodology to form conclusions about vendors and markets, in a peer reviewed process. Magic Quadrants categorize vendors into one of four quadrants based on Gartner’s assessment of them in two dimensions: Ability to Execute and Completeness of Vision.

In this case, Gartner identified NetScout as a Challenger, pointing out feature gaps in their product and negative customer feedback. NetScout, of course, thought it should be a Leader and sued Gartner. The case never went to trial and eventually the lawsuit was dismissed for the obvious reason that Gartner is protected by the first amendment and Netscout couldn’t prove any malicious intent. Gartner is paid very well by its customers for forming a strong opinion based on its methodology.

Gartner analysts could care less how much you spend with Gartner, and there are firewalls in place to make sure even the appearance of a conflict of interest are minimized. Could a Gartner salesperson have hinted at a connection between investment and MQ positioning? Of course. I’ve never experienced it, but even if it happened, an organization of NetScout size knows better.

Now, time to contradict myself. There is indeed one case where it does help to pay — you should really consider purchasing an subscription, sometimes called a seat.

Every analyst firm I’ve worked with will encourage you to provide regular briefings — whether or not you are a customer. These briefings are a monologue not a dialogue, but they are a free way for startups / category creators to get some mindshare without the cost of a subscription. The minute you have evidence of product-market fit you should be doing this at least once a quarter.

But if you are in a market with a Gartner Magic Quadrant and/or a Forrester Wave, or if there’s likely to be one, then you should consider purchasing a subscription. A subscription provides you with access to all the written research from the analysts, and more importantly it lets you schedule “inquries” with them to get direct 1–1 feedback.

The single best thing you can do to improve how analysts view your company is to help them map your company and products to their vision for a market. To do this, you first need to understand what’s important to each analyst you speak with — the language they use, trends they see, and specific product capabilities they deem important. Once you know this, you can frame your communication in a way that directly speaks to each analyst, using customer references as validation points (more on references later).

Any vendor in a big enough market to warrant a Magic Quadrant is likely able to afford a Gartner subscription. Yes, subscriptions are expensive, but so are many marketing investments, including paid acquisition and events. Doing well in an analyst report can be one of the best ROI investments you’ll make.

The TL;DR is this: there’s value in paying for an analyst subscription to gain more access to analysts. It’s not required, but probably a good idea for a lot of companies — just like investing in Adwords or events is probably a good idea for a lot of companies. But beyond a subscription, paying more to analyst firms doesn’t move the dot.

What does move the dot? Your customers. Which brings us to the next myth.

Myth #2: Your PowerPoint slides matter

Wrong, sorry. Analysts see right through hyperbole laden BS messaging, Nascar logo slides, and highly produced demos with more special effects than a Michael Bay movie.

Look, analysts are see thousands of PowerPoint slides a year and unless you are Steve Jobs, chances are your slides aren’t going to impress them. What analysts do care about is the strength of your customer references. This is basically all that matters.

Sure, regularly update analysts on your company, positioning, messaging, pricing with PowerPoint slides. All good foundational stuff. But analysts are forming their opinion on you through their interactions with customers, both the references you provide directly, and the daily interactions they have with their client base. Analysts are digging deeply in your company and product through the lens of the customer. How much value have they received? How difficult was the implementation? How’s your customer support?

If you really want to really improve your position in an analyst report, stop worrying about creating better slides and instead worry about creating better customers.

Myth #3: Gartner is the only analyst firm that matters

No way. There are lots of analyst great firms who offer tremendous value to both vendors and end-user customers.

Of course everyone knows the obvious names like Gartner, Forrester, and IDC. They cover lots of markets and buyer personas, and produce the popular vendor reports that get CEOs and boards exited ie. Magic Quadrants and Waves. Vendors spend most of their time analyst relations times with them, and I think that’s a mistake.

It’s often the more boutique analyst firms who offer more insight and value. These firms often have a narrower focus and are able to dig more deeply into a specific market. For example, there’s no one who knows digital experience better than Scott Liewehr from Digital Clarity Group. Scott personally travels hundreds of thousands of miles a year to speak with the companies and agencies who implement digital experience technology. When a CEO or CMO in this market really need to get to the bottom of something, the first call they make is to Scott.

Another example is David Menninger, who covers data and analytics for Ventana Research. Dave’s background as both an analyst and (recovering) product marketer at bunch of successful tech companies gives him a unique perspective that many analysts don’t have. As a former vendor he speaks my language and provides good insight on messaging, positioning, competitive dynamics, etc.

There are lots of other firms like Digital Clarity Group and Ventana Research who serve more specialized audiences. My recommendation is start first by researching the specific analysts who are closest to the customer in your market, not the just the analyst firm they work for. Otherwise, you’ll be missing out.

Myth #4: You can move the “dot” in a Gartner Magic Quadrant

This is one of my favorites! Sorry, but this never happens. Let’s start with a quick overview of the Magic Quadrant process.

A new Magic Quadrant kicks off with an email to all of the vendors Gartner thinks are candidates for inclusion. Vendors are provided a specific set of criteria, and then asked if they think they meet it. Gartner vendor feedback combined with their own knowledge to come up with a final list of vendors to evaluate. Each of the vendors is provided with a long set of requirements and asked to provide reference customers. Each vendor is then given an opportunity to present to the analysts for 60 minutes or so. This process takes 2–3 months start to finish. Gartner then compiles all of the findings, going through an exhaustive process over an additional 2–3 months or so.

And then that moment when you get the email from Gartner with “FACT CHECK” in the subject. Sit down. Breathe. Open the email, and you’ll see where all the dots landed!

Careers can be made and destroyed by this one email. Exceed expectations and you are a forever a hero to your CEO + board. Do poorly, and well sadly I’ve seen people fired. True story: I once fell out of my chair and ran around the office screaming when I saw that Acquia had become Leader in a Magic Quadrant. Pro tip: you can’t tell anyone about it your dot position during the fact check stage, so make sure you have a good story already worked up 😉

Regardless, no matter where the dot lands, no amount of arguing, pleading, or begging is going to move it at this point. Gartner makes this crystal clear in the fact check process, but it doesn’t stop companies from trying. Take an any analyst out to dinner, and you’ll likely hear stories about all the irate phone calls they get from CEOs who are furious over the dot. I’ve been told it’s badge of honor when analysts get these calls from the likes of Larry Ellison and Marc Benioff.

Look, Gartner doesn’t care that you just crushed Q2, or that you never lose the the competitor who is ranked well ahead of you, or that you just raised a $100m Series C from every top-tier valley VC. They only care about their methodology. I’ve worked on a couple dozen Magic Quadrants over the years and I’ve seen a vendor dot move exactly once during the fact review process. Even then it was by such a miniscule amount that you’d barely notice it, unless you obsess over things like this like I do.

The time to move the dot is well before the Magic Quadrant process started. If the result isn’t what you hoped for, take the feedback, swallow your pride, and get started for next year. Whatever you do, don’t be that company whose CEO makes the angry Friday 5pm call. It’s not going to work. Send them this post.

Myth #5: Just becoming a Leader in an analyst report will double/triple/10x your growth

Hopefully, but usually not even close. Being a Leader helps revenue growth for sure, but maybe not as much as you think. Even then, it takes a lot of work to make the growth happen.

Jeff Mann of Gartner once put this fake Magic Quadrant together as an April fools joke, but there’s indeed some truth to it.

Credit: Jeff Mann

The implication is that buyers prefer Leaders, ignore Niche, and are wary of everyone else. To prevent this, Gartner and other analyst firms have standard disclaimer language that encourages buyers to look more closely.

Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

Of course buyers should look beyond Leaders, and they usually do. It’s highly unlikely that your requirements directly map to the ranking methodology used by analyst firms. Often buying from a Leader is the worst possible decision you can make.

But being a Leader in an analyst report will absolutely get you on more short lists, which itself can be good or bad if you aren’t prepared. For example, when Ektron became a Magic Quadrant Leader in 2012, it got us into a whole bunch of new opportunities against much bigger competitors like Adobe. It completely changed the dynamics of our sales funnel. Deals were bigger, but took longer to close and were much more competitive. It was overall a huge net positive, but success didn’t happen overnight, and we still had to work hard at it.

It’s great to be a Leader in a Gartner Magic Quadrant or Forrester Wave, and companies who make it in for the first time should be very proud. It’s a huge PR moment. Your CEO, board, and investors will love you. It’s a good morale builder internally. But make sure you set realistic expectations about the growth impact it will have, and make sure you are prepared for the changes that will happen to your business as a result of it.

Myth #6: Your PR firm can manage analyst relations

Probably not, it’s a different skillset.

I’m sure there are some PR firms who are better at this than others, but in my experience, treating analysts as just another influencer channel is dangerous. I recommend two things if analyst relations is really important to your company.

  1. An executive should own it directly and consistently. Too many companies delegate AR far down into the organization, especially as they get larger. In my experience, having one executive own AR over a period of many years builds the sort of trust it takes to influence how an analyst thinks about a market.
  2. Always involve founders or whoever is the actual thought leader at your company in as many analyst interactions as possible. Analysts want to hear from the most credible sources, which isn’t your AR team or product marketers. At Acquia, that meant using Dries Buytaert as often as possible when speaking with analysts.

If you do need help scaling your program, look to agencies who specialize in analyst relations like Spotlight AR. They understand the nuances it takes to run a successful AR program at scale. (Disclosure, I was once a customer of Spotlight).

Want to learn more?

To demystify the black arts of analyst relations, here are the best analyst relations professionals + resources I’ve come across. Who/what am I missing?

Beth Torrie. I competed against Beth for many years, and she’s the best. I hope to never compete against her again.

Rick Nash and Andrew Hsu at SpotlightAR. They’ve helped dozens tech companies with their AR programs.

Joely Urton of Box. I’ve never met Joely, but a good friend of mine worked for her, and said she’s great. That’s enough for me.

Institute of Industry Analyst Relations is a not-for-profit organisation established to raise awareness of analyst relations and the value of industry analysts.


What I Learned Building an App on the Drift Platform

A long time ago in a galaxy far, far away — I was a Computer Science major at the University of Illinois. I grew up programming in BASIC on a variety of hardware, including my beloved TI 99/4A and then later a Commodore Amiga. My parents came up with a brilliant strategy to get me to learn to program — you want to play games on the computer? Build them.

But after graduating college I realized I was pretty awful programmer. I was working in QA at a software company, and I found out pretty quickly that I better find something other way to make a living. Since I was better at talking about technology than building it, I ended up becoming a sales engineer and then later, a marketer.

I’ve always felt that my tech background has helped my marketing career, a phenominon Scott Brinker wrote about way back in 2008. Marketing as a discipline sits at the center of science and humanity, and I’ve tried to balance the two in my career.

But sometimes you just feel the need to build.

Building a App on the Drift Platform

The Drift platform is a new set of APIs to allow developers to create and publish apps on Drift. I wanted to find a way to make it easier for our team at RapidMiner to learn more about the people we’re having conversations with.

I’ve been wanting to get back into coding as a #sidehustle thing, so I took at look at at one of the sample applications that launched with the Drift platform and decided to give it a try.

Every time we start talking to someone with Drift, we try and look them up in Salesforce to learn about them so we can make the converation more relavant. Salesforce has all of the usual information, but more importantly for us, it has all of the information about how people are using our products.

Our process with Drift at RapidMiner was to cut+paste the email address into Salesforce to look up a user. Sounds easy, but it takes time, and disrupts the flow of the conversation especially when we are juggling lots of simultaneous conversions.

So, I decided I’d try and build an app called salesforce-lookup that would make it easier to pull data from Salesforce into Drift. It takes the email address of the current user and returns a bunch of useful information from Salesforce directly into the Drift conversation. It looks like this inside Drift:

The app will automatically generates a direct link to the user in Salesforce, and returns information about the user, including the number of times and the last time they used RapidMiner. Simple for sure, but it saves a bunch of time for the team and lets us have more relevant conversions.

Here’s how I did it, and what I learned in the process.

Creating a GitHub Account

Okay, I already had a GitHub account, but had never used it before. Wow, GitHub is simple and useful. I can’t compare it to the all the old ways to manage code because I never used them, but it sure beats vi and a filesystem.

Apparently people still do use vi (and emacs). And there’s even a Wikipedia on editor wars. (I’m team vi, for the record).

Setting up Heroku

Heroku is an obvious place to run nodeJS apps, so I gave it a try. It was super easy to setup, and connects right to GitHub. Every time I pushed my code to GitHub it would automatically redeploy the app.

Learning nodeJS and Asynchronous JavaScript

Okay, now the hard part — building the app in nodeJS.

I’ve used client side Javascript extensively during my time as a sales engineer, but I never really appreciated how much its has evolved for server-side applications.

My breakthrough was learning that Javascript can be asynchronous, and that means it requires a bit more planning when building the app.

I had to string a serious of function calls together to make sure that I had all the information I needed at each step. That meant I had to learn all about callback functions (shoutout to Joel on the Drift team for explaining this to me).

For example, here’s a function that finds the Drift Contact Id for the user in the current converation.

In an asynchronous Javascript world, you don’t really have the Drift Contact Id variable until you execute the callback function GetContactId. Once I figured this out, I was able to chain together a series of API calls to Drift and Salesforce.

The Drift APIs were easy to use, but Salesforce was a bit harder. One of the best parts of nodeJS is the ecosystem, and I found a robust library called JSforce to help with the Salesforce query.

The hardest part was logging into Salesforce using OAuth. I had never looked at OAuth before, so I had to learn enough about it to access both the Salesforce and Drift APIs. Authentication to Salesforce is a little bit more difficult as you have to handle tokens that can expire.

Once I was able to authenticate, it’s just a simple SQL-like query to return what I needed for my app — the users name, company, and some details about their RapidMiner product usage. I can query any field we have inside Salesforce.

conn.query("SELECT Id, Email, FirstName, LastName, Company, Academics__c, Total_RM_Studio_starts__c, Last_RM_Studio_usage__c FROM Lead where Email = '" + emailAddress + "'", function(err, result) {

Then I put together the message…

// Build the Drift reply body
body = "<a target='_blank' href=" + Id + ">" + firstName + " " + lastName + "</a><br/>" + "Company: " + Company + "<br/>Total RM Studio Starts: " + totalStudioStarts + "<br/>Last RM Studio Usage: " + lastStudioUsage + "<br/>Academic: " + Academic

And lastly send the message back to the Drift conversion.

// Send the message
return + `/${conversationId}/messages`)
.set('Content-Type', 'application/json')
.set(`Authorization`, `bearer ${DRIFT_TOKEN}`)
.catch(err => console.log(err))

I was glad to learn that the best way to debug things is still to output stuff to the console. That hasn’t changed since my days of debugging BASIC. At one point, I basically had a console.log every other line.

All marketers should learn to program

When I finally made this work (at 2am last night, just like in college!) it felt really great. There’s nothing like building something and seeing it actually work. Heck, someone has even forked my code already (looking at you Brian Whalley, good luck!). I’m sure my code is awful, but it does something useful, and it was really fun to build.

Steve Jobs once said everyone should learn to program a computer, and I agree. It helps you develop logical thinking (hey asyncronous Javascript), and many of the same concepts you learn in programming apply to lots of things we see as marketers. For example, your Marketo, Pardot, or HubSpot segmentation lists are just a complex boolean expression.

The advent of applications like GitHub and Heroku plus the availability of a massive variety of training resources make getting started with programming today easier than ever. Seriously, imagine getting a CS degree today without StackOverflow — yeah, that’s what my generation had to do.

function getContactId(conversationId, callbackFn, orgId) {
// Get the contact from Drift
.get(CONVERSATION_API_BASE + `${conversationId}`)
.set('Content-Type', 'application/json')
.set(`Authorization`, `bearer ${DRIFT_TOKEN}`)
.end(function(err, res){
callbackFn(, conversationId, orgId)
// call back function
function GetContactId(contactId, conversationId, orgId) {
return getContactEmail(contactId, GetContactEmail, conversationId, orgId);

Your Best Startup Strategy is Better Execution

It’s the most wonderful time of the year. No, not that time. The other one. The time where your startup gets together and starts planning for the upcoming year. Some of you have already finished this process, but with my fiscal year ending in January, I’m currently in the midst of it.

That’s why this article from Sarah Hodges on how to run better management team offsites caught my attention.

In particular, I loved this quote from former colleague and friend Thomas Erickson, the former CEO of Acquia.

Make sure that a focus on execution remains front and center. Have a regular “execution” meeting and less regular “strategy” meetings.

— Tom Erickson, CEO, Acquia & Co-Founding Pillar


I’ve often thought that as leaders, we focus too much on strategy and not enough on execution. Peter Drucker once said culture eats strategy for breakfast, and I agree. I think organizations with a culture of getting shit done win. I’ll take a relentless executor over a “strategic marketer” any day.

I know there is a larger debate around strategy vs. execution. At larger companies, planning, strategy, and execution are seperate disciplines. But my experience comes from working at startups who don’t have the luxury of dedicated teams augmented by management consultants, so finding the right balance is important. Focus too much on strategy, and you’ll end up with a lot of great looking PowerPoints but metrics that look more like 〰 or 📉.

Set the Corporate Strategy Annually

Having worked for Tom for ~3 years, I’ve experienced how he found the right balance between strategy and execution.

Once a year in November, Tom brought the entire extended leadership team together to run a three day activity called the Goal Deployment Process, or GDP as we called it. This process was used to identify the most important issues for Acquia to tackle in the upcoming year, prioritized by their potential impact. It was a team exercise where Tom laid out the three year objectives for the company, and we worked backwards into all the things that would need to happen for us to get there.

The outcome of the GDP was a list of ten or so strategic cross-functional initiatives for the upcoming year, with clear definition of ownership, metrics, and specific actions to take. Here’s a look at what one of them looked like:

The robust planning process drove alignment on the most important items to the company. The GDP actions were often tweaked and metrics were redefined, but very rarely did we decide to abandon any completely during the year. Many of the important milestones in Acquia’s growth came as a result of the GDP process.

Drive Daily Execution

But the most important part of the GDP process wasn’t the annual meeting, it was how it affected daily priorities across the company.

Each owner of the GDP was expected to drive their actions across the company. To make an impact, the GDP leader had to structure their day to make sure they were impacting each of the cross-functional actions they owned. Every month, the extended leadership team would get together to review the metrics. We used a spreadsheet with the plan and actual result, with color coding to highlight areas we were performing under expection.

It was simple and effective. Every month, GDP leaders had to provide an honest assessment of how well (or not) we were able to drive improvements. Too much red? You aren’t executing well. Too much green? You probably set your goals wrong. Note that with Tom, I learned (the hard way) that it was better to have more yellow and red than green.

Tom ran the weekly executive leadership meeting at Acquia the same way. Every Monday morning, he brought the exec team together to review a spreadsheet containing a running list of the most important actions across each department in the company. These were usually less cross functional than the GDP, but still important to the success of the company.

This list forced each executive to provide a weekly assessment of where they were at on important deliverables. This simple format wasn’t appropriate for managing the details of the actions, but it was an easy way for the exec team to track progress, and if something were blocked, it could be addressed directly in this meeting.

Anything that fell in the strategy bucket in the weekly meetings was noted, but not addressed as a part of this meeting. Tom kept the meeting all about execution.

You need a strategy for better execution

Here’s the thing: execution is hard and its a grind. But like anything, being great at execution is daily hard work. There were times that I hated the GDP process, and all of the time it took to drive actions and report on results. But looking back on it, I realize that it was one of the main reasons why Acquia grew as fast as it did under Tom’s leadership.

Chances are at your startup, you’ve already got a good enough strategy in place. You know your customers, and you’ve built business model. You’ve got product market fit and a go-to-market in place. But do you have a strategy for getting better at execution?

If you don’t, that should be a big topic for your 2018 planning process.


Predictive Account Based Inbound Agile Marketing Automation

We’ve hit peak Account Based Marketing.

I’ve received 5 emails this week from vendors extolling the virtues of ABM. I’ve been invited to one dinner, two lunches, received a $50 Amazon gift card, and was told by Marketo that “According to 97% of marketers, account-based marketing (ABM) achieved a higher ROI than other marketing initiatives.”

Everyone is “flipping their funnel” using Account Based Marketing and I’m missing out. Or am I?

Ok. Let’s say I buy that number from Marketo (or something close to it). That still doesn’t mean that Account Based Marketing is right for me. Maybe I’m a proud member of the 3% who think ABM is probably great for lots of organizations, but not for mine? Note that none of those BDRs who invited me to dinner tried to make the case why my company (RapidMiner) would benefit from ABM in the first place.You know you are at the top of a hype cycle when its acceptable to market “the thing” — and not what the thing delivers, or why you should care.

The truth is that really great companies almost never follow someone else’s playbook, at least not verbatim. Salesforce, Slack, Atlassian, and HubSpot, didn’t follow trends, they created them. They went right when everyone else was going left. They were the 3%.

The ABM phenomenon got me thinking about what makes marketers so susceptible to trends? Maybe it’s that we like being marketed to? We appreciate masterfully executed campaigns like Flip my Funnel and Account Based Everything. We certainly like belonging to a tribe — being part of a movement. Ever been to the Dreamforce or Inbound? There’s clearly comfort in numbers. There’s nothing inherently wrong with following a trend, but it’s almost never as easy to pull it off as the trend-makers make it seem.

For example, how many “Appropriate person?” emails have you received+deleted this week? I wouldn’t know, because I created a rule to delete them a long time ago.

The book Predictable Revenue by Aaron Ross taught this approach to outbound sales teams and it absolutely worked for many of the most successful tech companies on the planet — until it didn’t. By then, the trend makers had already moved. on while the trend followers continue to blast templated emails to tuned-out audiences like me to this day.

E-Meet me?

Vendors know the power of trends and the momentum they create. It’s not a coincidence that ABM vendors are teaming up to support the movement. ABM vendor Terminus founded the “Flip My Funnel” community alongside a number of competitors. Even traditionally laggard analysts like Gartner are jumping on board the ABM hype train.

But trends come and go. Ask anyone over 40 how tech marketing worked before AdWords and marketing automation, and it will sound a lot like ABM. Tech marketers in the 90s and early 2000s didn’t have the luxury of the low cost distrubution channels that we have today, so focusing on accounts was the only way to go.

Sure, it was much harder to do ABM then, and the advent of all the new ABM companies make it much easier to do ABM at scale today. Still ABM isn’t new, and it worked then for the same reasons it works now.

Look, I’m not against ABM or any particular marketing tactic. Brandon Redlinger of Engagio lays out some great advice for organizations considering ABM. I’m just against flocking to a trend because everyone else is. There aren’t any growth shortcuts or get rich quick schemes. As I’ve said before, study these trends. Learn from them. Be inspired by them. Implement some of them. But don’t blindly follow them just because you think everyone else is.

And don’t be afraid to be a part of the 3%.


Stop The Lead Scoring Madness

I recently came across a lead scoring article on the Mattermark blog that reminded me of a post I’ve been wanting to write for a while: the way nearly everyone is doing lead scoring is totally wrong.

In the modern era of data-driven marketing where marketers boast about being able to connect every penny of marketing spend to revenue, we are using nothing more than gut instinct and anecodotal feedback to figure out the best leads to pass to sales. The main culprit for this data-driven nightmare is the way lead scoring is implemented in marketing automation systems like Marketo and Pardot.

For example, here’s how lead scoring currently works in Pardot: at the system level, you can assign points to specific actions like email opens, page views, form submissions etc.

You can also add points as a completion action to things like form submissions to weigh specific types of actions and content more heavily than others; like maybe an analyst report or a specific webinar.

The idea of course is that now we can pass the highest scored leads to sales, and that there is a correlation between the lead score and likelihood to buy.

But as Mike Volpe points out the way we assign “points” makes absolutely no sense, yet somehow it has become accepted as the gospel of modern lead management best practices.

It turns out that accumulating points is actually a terrible way of predicting an outcome, leading to marketing passing horrible leads to sales; like the serial webinar attenders (you know, those people that come to EVERY SINGLE WEBINAR you run) and habitual email openers— instead of the people most likely to actually buy your product.

It’s likely that much of the benefit of manual lead scoring comes from the placebo effect of telling sales that you are only passing them the best leads. So in turn they work them more exhausitively, and you’ll get better results than the unscored leads of the past that sales would mostly ignore. But that doesn’t mean they are better quality leads. They could actually be worse and you’d never know it.

Don’t believe me? Here’s an A|B experiment you should run immediately:

Take 10–25% of your lower scored leads using your current lead scoring model and start passing them to sales. The leads should appear like the rest of the higher scored leads(MQLs) you are currently passing to sales, and don’t tell anyone so there is no bias on how the leads are worked. Make sure you can identify leads in the experiment group against the control group.

Run the test for as long as it takes for your test to hit statistical significance, which will be based on how many of your leads convert into whatever metric you are using to gauge the health of leads e.g. SAL, SQL, etc.

In a few weeks after your sales team finishes following up on the batch of leads, compare the two groups using whatever conversion metric you are using. Now did the higher scored leads convert better than the lower scored leads? When I ran this experiment at my last company they didn’t — and the lower scored leads converted at a slightly better rate than the higher scored leads. Oops.

Building a lead scoring model by assigning random points to actions and attributes of leads is just about as un-data driven as it gets.

The solution is to build a lead scoring model based on a proven mathemetical model that will really tell you the features — or attributes of a lead — that have a statistically significant corrleation with the outcome you are trying to predict e.g. people who buy your product. Unlike the randomness of assigning points to specific behaviors and attributes of a lead, a predictive model will give you a statistical measure of which features actually matter the most.

There are predictive marketing products that will do this for you — like Infer, Mintigo, and Everstring. But they come with a hefty price and are overkill for many companies. You can get started for free using tools like RapidMiner Studio or even good old Microsoft Excel.

The first problem you’ll need to tackle is getting access to clean data. Predictive lead scoring is mostly a “wide data” problem; you want as much data as possible about your users to uncover the specific features that have the most impact on closing deals. This means combining data from your CRM and marketing automation system, hopefully your product, and perhaps even external APIs. But chances are the data from your CRM and marketing automation system is clean enough to get started.

Once you have the data, you’ll model it using a variety of machine learning techniques, and most likely you’ll start with some form of regression. I won’t attempt to explain the math behind regression here, but if you are interested here’s a good overview from RapidMiner founder Dr. Ingo Mierswa.

Here are two resources for getting started with either Excel or RapidMiner Studio for free:

  1. This video from Ilya Mirman, the VP of Marketing at OnShape, shows you how to build a predictive lead scoring model in Excel using linear regression. Okay fine, Ilya is a Stanford-educated engineer and MIT MBA, but the approach he outlines is something even mere mortal Excel power users could figure out with a little bit of effort. If you can do a VLOOKUP, you can try this.
  2. With a bit more effort you could download RapidMiner Studio, which will help you both prepare your data and build your predictive model. RapidMiner Studio provides far more options for modeling than Excel, and will help you get a more accurate predictive model. The current lead scoring model we’re using at RapidMiner was built by my colleague Thomas Ott, and here’s a webinar where he walks through how we built the model complete with a sample of our data and the model we use today.
Lead Scoring in RapidMiner Studio

Regardless of which tool you use, you’ll get a result that looks something like this — a weighted view of the lead attributes that actually result in more deals.

For example, in our RapidMiner lead scoring model. the number of product starts was the most important factor in predicting a purchase, and leads from Netherlands were more likely to close than any other country.

We used RapidMiner Server to automate adding our predictive lead score to Salesforce, but even a simple Excel output of your weighted attributes is enough for you to make those marketing automation point assignments using mathetically-sound principals instead of educated guesses. And that’s a huge step in the right direction, and you didn’t even have to spend a bunch of money on a predictive lead scoring product to make it happen.

Lastly, I highly recommend everyone read Myk Pono’s opus on How To Design Lead Nurturing, Lead Scoring, and Drip Email Campaigns. Myk goes into this and many more topics in great depth.

Whew, feels good to get that post off my chest 🙂


A Primer on Developer Marketing

TL;DR marketing to developers is hard, and you are probably doing it wrong.

The unstoppable rise of the developer

My former Acquia colleague David Churbuck introduced me to RedMonk’s Steve O’Grady and his 2014 book about how developers are taking over the world, called The New Kingmakers. The idea is that successful companies empower developers to solve problems instead of getting in their way by dictating legacy technology and process.

It sounds obvious now, but it wasn’t so long ago that everyone who sold technology started at the top of an organization, typically the CIO, and worked their way down. Important technology decisions were made on the golf course or at the steakhouse, and companies like Oracle, IBM, and SAP sold billions of dollars of enterprise software this way for decades.

But the explosion of open source and cloud changed everything. Instead of waiting for technology to be handed down by the CIO, developers could now pick whatever best product they wanted, and quick deploy them on AWS, Heroku, DigitalOcean, etc. We even coined the term Shadow IT to describe the stealth adoption of unsanctioned technology. O’Grady captures this disconnect nicely in a Venn diagram:

The New Kingmakers

So while the balance of power has clearly shifted to developers, many enterprise technology marketers continue to chase the CIO with old school enterprise software marketing tactics. Does this sound familiar:

First, we’ll create a whitepaper and put it befind a form. Then we’ll spam errr syndicate it to a bunch of opt-in(ish) email lists. Then our BDRs will call them Schedule a demo. Profit!

Ditch your traditional tech marketing playbook

One of easiest ways to fail at developer marketing is to fall in love with marketing best practices that were formed by companies who don’t sell to developers.

I’ll be honest: I’m jealous of my peers at companies like HubSpot and Marketo who market to marketers. It just seems so much easier. I know how CMOs are measured — how we buy — the language we use. I’m a good proxy for marketing to other CMOs. But I’m a horrible proxy for marketing to developers. I’m reasonably technical; I was a CS major, ran technical sales at a few enterprise software companies, and I’m currently the de-facto WordPress developer at RapidMiner. But I’ve never had to deploy code hundreds of time a day, scale a React.js application, or secure a bunch of Docker containers.

So just running the traditional one sized fits all tech marketing playbook isn’t going to work. The developer audience is particularly wary, and as O’Grady puts it:

Developers have, out of necessity, built up an immunity to traditional marketing tactics. Print ad placement doesn’t work because they spend most of their time online. Online advertising is ineffective because they block ads. Forced registration for white papers fails because they don’t care about your high level whitepapers. Media messaging is ineffective because the developers know more about the technologies than the reporters do. Analyst webinars are ignored for similar reasons. And if you throw a conference featuring your executives talking about their projects, expect your WiFi to crash as developers tune out the talk and hack their way through the sessions.

Respect the community

GitHub has millions of users. Over 100,000 users contribute to Drupal. MongoDB has been downloaded 20,000,000 times. RapidMiner has over 70,000 active users a month. The single most important factor in the growth of companies who sell to developers is the strength of the community. Even Microsoft gets this, open sourcing the hugely popular .NET framework to encourage even more adoption.

But successful developer communities aren’t built by marketing, and sometimes happen in spite of it. They almost always form organically around a product/project that developers love to use, combined with a a shared sense of purpose by the community to do something meaningful.

The danger comes in treating your developer community like a CRM or marketing automation system. Don’t expect that you can convert them with a lead form, “nuture” them with emails, harass them into taking a demo, and close. This approach works to a point, but it won’t the create the kind of sustainable value that comes from having a world-class developer brand.

Go into your developer marketing efforts knowing that many of the positive effects of building a strong developer brand are frustatingly hard to measure. But at the end of the day, developers only really trust other developers. You’ll have to have some patience to get the measurement right, and have conviction in the connection between longer term brand investments and the advocacy and revenue that comes from them.

Great marketing starts with great documentation, and deeply technical content

Your product might be the most important innovation since Sir Tim Berners-Lee’s World Wide Web, but if no one understands how to use it, you’ll quickly become irrelevant. Treat great documentation the same way you treat building a great user experience. It’s that important.

For example, Twilio, Stripe, and Soundcloud all have amazing (and up to date, that’s really important) documentation, complete with thorough tutorials across multiple languages, getting started guides, detailed API overviews, working examples, and more. Their documentation makes it easy (and fun) to use their products and APIs.

And their documentation is easy to find and use. When I ran marketing at Acquia, we had all of our excellent product documentation behind the dreaded lead form, much to the dismay of the engineering team. I argued for it, because this form was one of the best sources of “leads” without consideration for the negative impact it would have on our developer brand.

Every few months or so, Acquia’s head of engineering would ask if we’d consider un-gating the documentation. And each time I refused, armed with the data showing how many leads were sourced via the documentation form.

But I was way wrong; the kind of mistake you make when you overweight short term goals like leads with long term objectives like brand building. Eventually we did the right thing and liberated all of our great documentation and you know what? All the developers who converted via our documentation forms ended up converting somewhere else on our site — webinars, product trials, etc — if they were really interested. None of our metrics went down.

Outside of documentation, creating great developer content requires deep focus and commitment. Developers aren’t going to read your lightweight ebooks and “10 awesome ways to speed your React.js app” listicles. Your content should try to be three things:

  1. Relevant. Is your topic relevant to your developer audience? Bring your own developers / experts / and community into the ideation process. It’s amazing how quickly you can fill a content calendar by just asking your users what they would like to read.
  2. Honest. Marketers love to spin. We love to write about all the game-changing, paradigm-shifting stuff our products do. But developers don’t grade marketing on the strength of our superlatives, they want to know what you really do.
  3. Actionable. Will the reader do something different or try something new as a result of your content?

For example, before the launch of Drupal 8 last year we turned a series of blog posts from Acquia’s Angie webchick Byron into the 40-page Ultimate Guide to Drupal 8. It’s still a popular resource for developers looking to move to Drupal 8, and it checks all three boxes.

Another example comes from Codeship, a continuious integration startup in Boston. With a tiny marketing team, they have built a blog newsletter subscription of over 65,000 users by providing detailed technical content on a wide range of topics. Some content is specific to Codeship, a lot of it isn’t. But all of it has been vetted for technical depth and accuracy, and is highly relevant to their developer audience.

The type of content doesn’t really matter — blogs, webinars, podcasts, etc. all can work. RapidMiner regularly gets over 2,000 people to register for our monthly webinars on data science. Andreessen Horowitz runs a great podcast on a variety of technical topics. Just make sure your content is relevant, honest, and actionable and it will do well.

Eliminate all your bullshit messaging

Most marketing copy is awful. The aforementioned David Churbuck of Acquia has a great perspective on the straight talk that is missing from marketing. David is former journalist, and the founder of

As a corporate communicator, a so-called “content marketer,” I have a conflicted view of the disease as both a carrier and critic. Without casting stones inside my own house, let me just say that I fight a constant, daily war against the forces of derivative babble-speak, and think, after over a decade within the walls of corporate communications (after two spent in journalism), that I understand the source of the pestilence.

David blames our jargon-laden bullshit addiction on the symbiotic relationship between: 1) the press, and their need to oversimplify complex technology 2) industry-analysts who reward vendors who speak with a common language and 3) Google, and our insatiable need to rank for every possible search keyword.

If you are really marketing to developers, then you need to write for them too. In plain English. Without hyperbole. And buzzwords. Developers want specifics. They want to know what you do, and don’t do, and exactly how you do it differently than whatever approach they are using today.

Take a look at your product page. Does it cover the specific technical details of your product, or does it regurgitate phrases that you are trying to rank for on Google or industry analyst-speak?

My colleague Ingo, the founder and President of RapidMiner, reminds me of this daily: there are over 1500 operators in RapidMiner Studio, but you’d never know it because we’ve buried them on our site. Our goal is to use product pages to narrow someone quickly into detailed technical documentation, tutorials, samples, etc. It shoudn’t take more than two clicks from the homepage to learn about exactly how RapidMiner does logistic regression or deep learning.

Note that the total bullshit filter doesn’t have to apply to your your mission/vision statement. It’s okay to aspire to inspire. For example, Kubernetes is “an open-source system for automating deployment, scaling, and management of containerized applications.” Yeah, that’s exactly what it is. But Nginx provides the more visionary “Flawless Application Delivery for the Modern Web”. But get to an Nginx product page, and you are one click away from the deeply technical content developers expect, like this on the Nginx Content Cache.

Lastly, if you are really the leading platform for something, you don’t really need to say it. And lets all agree to stop using “seamless integration”. There has to be a better way.

Make events a core part of your strategy, but do them differently

Developers love events. They are happy to spend their nights and weekends networking with their peers. Decisions that used to get made on the golf course are now being made over the weekend at camps, cons, and meetups. But your developer event strategy has to be more thoughtful than just sponsoring a tradeshow and staffing it with your sales team.

In some cases, you’ll have to do the traditional big tradeshows. If you sell security software, you’ll need to be at the RSA Conference because all your competitors will be there. If you are in the AWS ecosystem, you’ll probably sponsor re:Invent because of the huge developer audience. But don’t just staff your booth full of eager BDRs looking to scan “leads”. Make sure your booth is staffed with people who can answer technical questions and connect with the audience.

But there are lots of other ways to use events to reach developers without the high cost of big tradeshows.

Hold your own conference. Holding your first user conference is scary. You’ve got to build an audience from scratch. It’s expensive. You’ll struggle to get sponsors. You’re entire marketing team will be all hands on deck for months making it happen. But it’s so worth it, and it gets a little bit easier every year.

Jason M. Lemkin of SaaStr suggests that companies hold their first conference once they get to 100 customers or so, and I agree.

Participate in community events and meetups. The easiest way to get started with events is to find and supports the ones that are already happening. Chances are, there are already well established events happening around your product, and they would love to have someone on your technical team participate. Set aside some of your marketing budget to cover travel and arm your expert with plenty of swag.

Be creative, and have fun. You probably don’t have an unlimited marketing budget with and the ability to write a $500,000 check to become the Mega-diamond sponsor of every tradeshow. And thats okay, because brainpower and creativity scale much better than a marketing budget. Here are two good examples of creative developer campaigns:

  1. New Relic has given out well over 100,000 t-shirts, and credits them for a lot of their early traction. Seriously. Never underestimate the impact a brilliant developer wearing your t-shirt can have on a movement.
  2. DigitalOcean, in partnership with GitHub, runs an annual Hacktoberfest event to encourage contribution to open source. Make 4 pull requests from a bunch of open source projects and you’ll be rewarded with a fancy limited-edition Hacktoberfest t-shirt.

Why you should find a developer evangelist

Steve Francia was the Chief Developer Advocate of MongoDB, where he was responsible for the developer experience of MongoDB and led a large team of developer evangelists. He was an early employee and his evangelism team had a huge impact on the early traction of MongoDB.

What does a developer evangelist do? Their job is to lead the grassroots adoption of technology in a quest to win the hearts and minds of developers. There are lots of ways for evangelists reach developers — ranging from speaking at conferences to blogging code samples to participating in developer communities. Evangelists address the the trust gap that developers have with traditional marketing, and they provide unique insights back into the product team as they are often the closest to the users.

Whether or not they live in marketing doesn’t really matter, and they really shouldn’t. I’ve been lucky to work with two great developer evangelists over the past few years. eGandalf aka James Stout was a developer evangelist on my team at EPIServer. James was initially a customer and active participant in the EPIServer developer community forums. We hired James on the marketing team to increase engagement with our developer and partner community, through providing everything from direct mentoring and coaching to code samples and working applications for customers. James led our highly successly Office Hours program, a weekly Google Hangout covering a broad range of topics for EPIServer developers.

Even a single evangelist hire can make a difference. And the best ones aren’t made, they are found. Chances are there is someone already on your technical team who is playing this role, and would love to make the transition. So go find yours.


When Sales and Marketing Best Practices Become Average Practices

No one got fired for buying IBM?

In the 80s, IBM dominated tech from the desktop to the datacenter.

Buyers often picked IBM because they were safe choice, not the right choice. It didn’t matter. Buying IBM became the accepted best practice for CIOs. It was an emotional decision, not a rational one. People generally don’t like taking risks, and IBM was the easy, safe choice. Of course this mentality led to high costs and failed IT projects, but at least no one got fired making a decision to buy IBM. Or did they? More on that later.

The same best practice phenomenon is happening today in sales and marketing. Most tech companies run the same sales and marketing playbook, including the Sirius Decisions lead waterfall, the sales specialization + BDR model taught by Aaron Ross in Predictable Revenue, inbound marketing from HubSpot, and so on.

Study these best practices. Learn from them. Be inspired by them. Implement some of them. But don’t blindly follow them just because you think everyone else is, because that makes them just average practices.

For example, the current BDR tactic du-jour gone wrong — the meme-laden breakup email. I know breakup emails sometimes work; they certainly got my attention when I first started getting them. But they are no longer fun or funny when I get tens of them a week.

Don’t follow the playbook. Be the playbook.

Truly great companies almost always pave their own way. They take best practices and make them even better practices.

The difference between copying a playbook and being inspired by one is understanding. If you really understand the problem you’re trying to solve, and there’s someone else doing it better than you, then by all means copy their idea but do it better. Make it yours. Good artists borrow, great artists steal.

For example, Salesforce re-invented growth for the SaaS era, as explained in great detail (111 plays!) by Marc Benioff in Behind the Cloud. Box later took the Salesforce playbook and adapted it to their land+expand freemium model. Companies like Slack and InVision succeed with a relentless focus on building beautiful, usable products, matched to a sales and marketing model that’s about eliminating friction in the buying process, not creating it.

But if you are implementing a best practice just because it’s the new thing that all of the hot startups and thought leaders are writing about, then take a step back and study the problem more or you’ll probably end up with something much closer to average than best.

In the end, people actually did end up getting fired over buying IBM. The 90s saw the accelerated rise of companies like Microsoft, Oracle, Intel, and eventually open source. Companies who held on to the IBM status quo for too long were exposed to competitors who could do things better/cheaper/faster with superior technology. The safe choice suddenly became the risky choice. The best practice became the average practice.


Why I’m Killing the Marketing Qualified Lead

Five years ago, I was sitting at my desk working on a PowerPoint pitch for an upcoming customer visit. My boss stopped by and asked me a question that would change my life.

Hey Tom, can you run marketing?

Prior to that moment, I had spent my entire career in technical sales with a few short stints in product management. I had worked closely with marketing throughout my career, but I had never been a marketer. I had never generated a lead. I had never managed a budget. I had never launched a campaign.

So I of course said yes, but then reality quickly set in. I had to scramble to learn modern marketing, and learn it fast. I started by reading everything I could find from the companies I admired. Marketo’s Definitive Guides. HubSpot’s Blog. Eloqua’s Content Grid. I read these over and over until I could nearly recite them word for word, and recreate the important visuals on a whiteboard.

I was asked to join marketing at a time when growth hacking was just becoming a thing, and it happened fit my background. I’m not a brand guy (a weakness I’m trying to address) but I love technology and math, so the language of marketing automation spoke to me. I implemented Marketo at my first CMO job at Ektron and then again at Acquia, where our team won the 2014 Marketo Revvie award for most dramatic business impact.

There’s no arguing that the MQL, and the broader sales and marketing funnel, transformed marketing. It forced alignment, requiring sales and marketing to agree on the traits and actions that made a good lead. It required a good content strategy to guide prospects through the complex B2B buying journey. It drove a consistent set of lead management processes that made it easy to measure conversions at key points. And maybe most importantly, it let marketing to prove our contribution to revenue.

I loved the MQL. I owe my marketing career to it. But now I’m over it. I’ve learned that more isn’t always better, and I think the MQL treadmill is slowly starting to suck the life out of marketing.

Enter the PQL.

I first heard of the concept of a PQL — or product qualified lead — via Christopher O’Donnell of HubSpot via a post on the excellent OpenView Medium publication. The basic idea is to combine freemium/open source product distribution with an inside-sales model to increase deal velocity. Users qualify themselves by using the product and inside sales exists to support them through the journey and set the stage for a long term relationship with the customer.

RapidMiner is the ideal candidate for a PQL model. We get over 20,000 product downloads a month. While there is some immediate drop-off between the initial product download and first usage, once someone makes it past the up-front learning curve, they absolutely love our product. For example, here’s our NPS score at two usage checkpoints:

Our clear path to growth is to get more people to use the product, guiding them through the product journey from building their first predictive model in RapidMiner to embedding the results into a business process to make or save money.

If someone doesn’t download and use our product, they aren’t a qualified lead.

The MQL + marketing automation playbook simply can’t produce the kind of high velocity leads and conversion rates we’ll get from creating happy, engaged users. If users need additional help or have questions, they can still “raise their hand” to talk to our inside sales team. And we’ll use a RapidMiner predictive lead scoring model to help identify the right free users for our inside sales team to proactively connect with.

We’re going to fully commit resources across the entire company to make this PQL model work, from how we allocate marketing resources and budget to our sales processes to the product roadmap. Everything. Instead of measuring traditional marketing metrics like MQLs, we’ll focus on product usage metrics and customer success.

Goodbye Lead Forms

The shift away from MQLs also lets me liberate content from the dreaded lead capture forms. I’ve been thinking about doing this for a while, and this post from Dave Gerhardt of Drift Why We’re Throwing Out All Of Our Lead Forms And Making Content Free pushed me over the edge.

I really don’t care how many leads our content generates. If our content is great, more people will download RapidMiner (we do capture email addresses on downloads) and more importantly, people will use it and see how we can deliver business impact. I’d rather help one user build a predictive model that generates millions in new revenue than add a bunch of poorly qualified “leads” to a database.

For example, people love our webinar content. We consistently get over 2,000 people to register, even though we ask them to fill out a complex form. How many people could we reach if we required nothing? (or maybe just an optional email address to receive the on-demand version for those who can’t make it?). I’m betting on a lot.

We’ve got great content. We’ve got a team of brilliant data scientists and predictive analytics experts. I want to get their knowledge in front of as many people as possible.

If you like this post, please recommend it so others will see it. Thanks!


Is Specialization Bad for Startup Marketing?

We’re starting to run a Scrumban process in RapidMiner marketing. Scrumban is a great way accelerate output by creating more transparency around priorities in the hectic world of startup marketing. We’ve got our Scrumban board, with all colorful kanbans beautifully laid out into various columns.

RapidMiner’s Scrumban board, blurred

As we get into Scrumban, we often find outselves blocked by kanbans that require specialized expertise. For example, our product marketing team is swamped with important messaging and enablement work. They can quickly become a blocker for the demand generation and events teams who require messaging for campaigns and tradeshow booths.

Our team structure is pretty typical of tech marketing. We’re a team of 7, and as our CMO I’m still pretty hands on with things. While I’m not particularly great at anything, I’m pretty good at lots of things. And I’m certainly not afraid to push myself out of my comfort zone to learn new things. In the Scrumban process I’m able to pickup any kanban and do my best to move it forward.

In a startup like RapidMiner, versatility is really important. Marcelo Calbucci coined the term Full-Stack Marketer, a reference to the full-stack developer who is equally comfortable working on back-end and front-end development projects. The full-stack marketer needs to be equally comfortable with writing content, optimizing for SEO, executing a campaign, expanding reach with PR, enabling the sales team, review metrics, etc.

Now it’s really hard to be great at everything, but maybe that’s not the point? Functional specialization becomes more important as companies scale, but for startups marketers being locked into the job description you were hired for limits your ability to impact the business. And maybe more importantly develop your own skills, especially if you’d like to run marketing someday.

The path to CMO is paved by versatility.