Many marketing departments use expensive business intelligence software that provides dashboards full of green arrows, suggesting that their marketing is successful.
The increase in traffic, click-through rates, and stable cost per acquisition seem to indicate success.
However, the sales are stagnant.
This disconnect between what your measurement tools indicate and what your business is experiencing is where measurement tools fail their users.
The root cause of this lies in most marketing departments today: they do not have any decision-making oversight over their metrics. They view their metrics as report cards instead of warning signs.
Consequently, there are many instances where organisations have experienced a change in their metrics but have not acted on them or understood their implications.
To fix this, companies must take a different approach to measurement.
Companies need to shift their mindset from generic key performance indicators to a more structured hierarchy of operational factors, diagnostic factors, and business outcomes.
The reasons most marketing measurement does not work
Most marketing measurement fails before the marketing department receives any of the data on a dashboard.
Marketing departments do not understand the difference between diagnostic metrics and true key performance indicators.
Marketing departments create their reports based on habit rather than on a strategy for their business.
In some cases, marketing departments are using siloed data from the platforms they advertise on and trust the ad networks' attribution data versus the actual data in their Customer Relationship Management Systems (CRMs).
The boardroom war of numbers can be ended by creating clear definitions, who owns the data, consistency in reviewing the data, and defining one single source of truth.
Define the specific action needed for each metric change.
Stop reporting generically and begin using the "When this happens, do this" process to determine what action to take based on each of the marketing KPIs.
Stop only looking at basic tracking metrics.
A more mature organization focuses on incrementality testing and marketing mix modelling as opposed to reliance on only last-click attribution.
The pitfalls of data driven decisions
When searching online for marketing KPIs, the results will be the same everywhere!
Marketing vendors, marketing agencies, and marketing platform help desks all have the same list of common metrics: customer acquisition cost (CAC), customer life-time value (CLV), return on ad spend (ROAS), and conversion rate.
These lists are not helpful on their own.
Just because CAC is important doesn't make it an effective marketing strategy.
The failure of most materials about marketing metrics is providing teams with the markers to measure but not giving teams operational insight into how those markers work with each other.
How does one know what action to take when your cost per lead decreases, but your sales pipeline dries up?
When making data-driven decisions, understanding the context of the overall landscape is critical.
For example, high conversion rates could be meaningless if the traffic quality is so bad that the conversions have no revenue produced.
If you acquire customers at a low CAC, but those customers are going to churn within the first 30 days, then you are not measuring the effectiveness of your marketing efforts.
Additionally, measuring the effectiveness of your marketing should include an understanding of sample bias, the overlap of channels, the difference between leading and lagging indicators, and the false sense of confidence that comes from having blended metrics.
Overview of the hierarchical framework for measurement
The first principle of a modern analytical framework is that equal treatment of all data in making the decision will result in decision-making paralysis.
A current framework will implement an organized hierarchy of data filters that will allow executives to view high-level data for big-picture decision-making and narrow-filtered views of data for specialists to make tactical changes to marketing campaigns.
Business objectives
At the top of this structure is an organization’s business objectives.
These are the financial successes that support the organization’s existence and survival.
For marketing practitioners, these are often viewed as too abstract or difficult to achieve due to their many dependencies, such as product pricing, sales team execution, and current market conditions.
Yet, there is no marketing department that can be successful without a direct impact on the organization’s revenue.
The core business objectives of any organization will include gross profits, new revenue generated, retention of current customers, and the expected increase in market share.
If these numbers do not trend upward, there is nothing else that matters.
Key Performance Indicators (KPIs)
The true Key Performance Indicators (KPIs) are few and far between.
They are the connection between a marketing campaign and an organization’s business objectives, and they must be both highly correlated to financial success and able to be measured with high reliability.
Customer Acquisition Cost (CAC), when calculated accurately using the total cost of the marketing and sales teams, is one of those KPIs.
The Marketing Originated Pipeline (MOP) is another KPI. Return on Marketing Investment (ROMI) is also a KPI.
These numbers indicate to leaders the effectiveness and scalability of their marketing operations.
Supporting metrics
The Supporting Metrics provide context for interpreting the findings regarding the KPIs.
The Supporting Metrics are focused on the channel marketing managers and department heads.
If the overall Customer Acquisition Cost (CAC) is spiking, the Supporting Metrics will identify the underlying issue.
Are social channels' CPMs driving this increase? Or is the sales team's close rate declining?
Metrics such as conversion rates by channel, lead opportunity ratios equating leads to opportunities, and the average cost-per-click fall into this category.
They may not be boardroom metrics, but they are vital management metrics.
Diagnostic metrics
The purpose of diagnostic metrics is solely for troubleshooting and optimising results.
They represent the earliest stages of determining whether a campaign is doing well or not.
Click-through rates (CTR), Cost Per Click (CPC), Cost Per Thousand Impressions (CPM), and Email Open Rates are all strictly diagnostic metrics.
An executive should never be exposed to a fluctuating CTR. That should never be a concern for them.
Media buyers live and die by those metrics.
Diagnostic metrics will initiate quick, tactical action: stopping an ad, changing a headline, and altering a bid limit.
Vanity metrics (In context)
Vanity metrics receive a bad reputation in the marketplace.
For example, total social followers, raw page views, and video impressions are generally considered to have no merit.
However, it's important to note that context is of utmost importance when making assessments regarding vanity metrics.
A metric becomes a vanity metric when it is used as evidence or justification for business success without ever considering whether or not it generated revenue.
In other words, if an enterprise software company uses its total number of LinkedIn followers as a measure of success in Q3, that is a vanity metric.
It is valid to consider equal numbers of video impressions to measure the initial reach of a consumer brand's awareness campaign and then measure brand lift and branded search volume before concluding how many impressions the brand has generated.
Real life scenario
The framework model is clean, but reality is usually a mess.
Delayed CRM uploads to sync, many different tracking pixels assigned to the same advertising source, and misalignment of client incentives will often compromise the integrity of the metrics reported to you from your CRM.
The true measure of maturity in measurement is the ability to interpret data that conflict and make an accurate decision.
Scenario 1: Increased website traffic with decreased revenue
An ecommerce business introduces a new digital PR effort. The result is a 40% increase in web traffic. Marketing is ecstatic.
However, Finance indicates that the gross revenues during that same period look as though they dropped by 2%.
Many people would assume that the website is broken and the checkout flow must not be performing properly.
But by looking at a range of diagnostic metrics, the diagnostics paint a completely different picture.
The conversion rate on the core product pages remained steady.
The decline in overall site conversion rates is solely due to a sudden spike of low-intent visitors arriving at the web blog through the PR campaign, reading, and leaving.
The decline in revenue was due to an out-of-stock situation on the top-selling SKU, which occurred at the same time as the PR campaign.
Workflow for making decisions
- Separate traffic into two categories based on intent (Transactional vs. Informational).
- Check stock levels and operational bottlenecks before blaming marketing channels.
- Suppress informational traffic when calculating core conversion rates to prevent unnecessary panic.
Scenario 2: ROAS has increased, but profitability remains flat
An agency contacts a client to report on an increase in Return on Ad Spend from 3.0 to 4.5.
The client looks in their bank account and cannot find the additional funds.
Classic failure of attributing to a last-click attribution option and platform-level reporting is illustrated in this example.
Ad platforms love to take credit for everything that they touch.
In this specific case the agency significantly increased the budget allocated to branded searches and subsequently began to retarget those people who abandoned their cart.
The advertisement platform took credit for sales that were going to occur regardless of their advertisements.
Instead of generating new demand, the advertisements served only to collect the intent of consumers, which meant that rather than generating new sales, the advertisements detracted from organic traffic.
Workflow for making decisions
- Pull back on branded search spend by doing a geo-holdout to determine true incrementality.
- Change how performance is reported to give consideration to blended Customer Acquisition Costs (CACs) and total net new revenue generated.
- Reallocate spending towards upper-funnel demand generation and accept that there will be lower ROAS from the advertisement platform but greater growth from the business overall.
Scenario 3: Cheap clicks on paid social equals no pipeline
A B2B SaaS start-up places a large amount of its advertising spends on LinkedIn and Meta Ads because to them, the cost per click is really low, which leads to really low form fills.
As a bonus, the performance marketing manager is ecstatic!
However, 90% of the leads being generated are being rejected by the sales teams.
The visitors to the site are composed of junior employees, students, and businesses which are outside the target market.
The low-cost clicks represent an illusion that masks the massive amount of waste incurred from the ad spend.
Workflow for making decisions
- Abandon measuring success by CPL.
- Create and mandate a strict CPQO metric as the primary KPIs for Paid Social.
- Establish Offline Conversion Tracking and feed retroactive CRM stage data back into the ad platforms to provide the algorithms with training data to focus on closed-won revenue, rather than simply cheap form fills.
Scenario 4: Webinars generate registrations, not sales
A marketing team runs a large webinar series where the number of registrations has been tremendous and the cost per registrant has been low.
However, three months after the registration period ended, the number of registrants who closed a deal was zero.
The trap of the webinar is in regard to the time-to-signal.
The problem here is that from a Business-to-Business (B2B) perspective, sales cycles tend to be lengthy.
The challenge of using registrations as a leading indicator of success is that they will only reflect how much interest there is at the point of registration.
If you wait six months for a particular deal to close, it means you have no actionable data as to whether your registration strategy worked or not. Therefore, you have to create additional evidence of success.
Workflow for making decisions
- Determine the missing middle metric that falls between registrations and closed deals. For this example, the missing middle metric is Meetings Booked.
- Examine the post-webinar nurture process of your leads to see if your Sales Reps are actually following up with leads or if the leads are just sitting stagnant in the CRM queue.
- Adjust your content for future webinars. If the number of registrations exceeds the number of intent leads, then the content may be too broad. Focus on narrowing your content down to highest-intent, lowest-funnel pain points so you can generate fewer registrations but higher-quality leads.
Establish governance
Poor data will destroy credibility much faster than poor performance.
If there are two sales and marketing directors in the same room and one states that Total Revenue is $100,000 and the other states that Total Revenue is $110,000, the meeting is dead in the water.
Any meeting time spent arguing the differences in data integrity will be ineffective.
Establishing governance is the less-pretty side of data analysis, but it's necessary to have a strong operating procedure to support the data collected.
The governance side of data is established through clearly articulated definitions of metrics. Every metric used should be clearly defined by a single source of truth.
Source of truth
Every metric needs a defined single source of truth.
Every platform has a level of bias rooted in an interest in maximizing the amount of money you will invest in its use.
Facebook would like you to spend more on Facebook. Google would like you to spend more on Google.
In terms of determining revenue, always use your billing system or ERP as the source of truth.
In terms of determining what stages of the pipeline and leads you are at, always use your CRM as the source of truth.
Advertising platforms only serve to provide you with numbers that can help you in your quest to diagnose how much you have spent on advertising relative to how many people viewed your ad and did what you asked them to do.
If the CRM reports 50 leads generated, while the ad platform reports 80 leads generated, the official count is 50. That’s it.
Frequency of review
It’s a fact: Data fatigue exists.
By reviewing high-level KPIs on a daily basis, you risk overreacting and making decisions based on temporary trends rather than seeing the entire view.
Therefore, it is essential to review high-level KPI data only every week or so, not every day.
You should divide up your review frequency into three distinct timeframes:
- Weekly Reviews: For tactical adjustment and to ensure that we continue to meet our spending objectives as well as spot any damaging issues like poor CTRs on major creatives or a dead link on tracking tags.
- Monthly Business Reviews (MBR): To monitor trends and progress towards key performance indicators (KPI) in regards to supporting metrics that we’ve established. How does our pipeline development look? What is our average blended cost per acquisition (CAC)? This is where we look to reallocate budgets between channels and make strategic changes.
- Quarterly Post-Mortems: Analyzing the quarterly success in regards to business objectives and KPIs. Did we achieve our revenue objectives? What was our true return on investment (ROI)? What are our major asset allocation and strategic plans for the next quarter?
Creating action-based workflows
A dashboard is only useful when it drives action.
It’s critical that mature organizations develop workflows and decision matrices prior to deploying campaigns.
Consider the following workflow process. If your cost-per-acquisition (CPA) increases by 20% within a rolling two-week period, you will be able to define how your team will react.
Will they put the campaign on hold? Will they check the performance of the landing page? Will they solicit new creative assets?
By developing these metrics ahead of time, you eliminate any emotional responses and remove all doubts from your team when performance is declining.
Advanced analytics: The end of basic tracking
The late 2020s will mark the end of an era of simple, deterministic tracking.
Due to increasing privacy regulations, cookie deprecation, and increasingly fragmented customer journeys, current attribution models are dangerously untrustworthy.
Leading companies are abandoning the concept of absolute tracking and instead relying on statistical models and find-learn-improve cycles.
Limitations of attribution models
Attribution software attempts to determine exactly the path that a user took prior to making a purchase.
Did they click on a paid ad, read a blog, open an email, then purchase?
Attribution is helpful in some cases; however, it is flawed at its core.
Attribution has a huge bias against trackable, bottom-of-funnel activity.
In addition, attribution software is completely blind to the role that dark social plays in influencing purchase behavior, such as podcasts, Slack communities, private Discord server conversations, and word of mouth.
Relying on a multi-touch attribution approach will inevitably lead to less funding for brand building and an overfunding of aggressive retargeting.
Incrementality testing
While attribution helps answer the question of what touchpoints were used leading to the purchase, incrementality testing answers the question of whether that sale would have occurred if no additional touchpoints had occurred.
Incrementality testing is considered the gold standard for measuring marketing effectiveness through running controlled experiments.
To provide an example of incremental testing, you can remove specific geographic locations from your paid search campaigns while continuing to spend the same amount on your other locations.
If these geographical locations exhibit a sizeable decline in revenue, that tells you that there is a high level of incrementality associated with those ads. Whereas if the revenue for those locations remains the same, then you will know that there is no return for that spend.
This approach enables marketers to determine the value of ads and the effectiveness of a variety of marketing strategies, free from the biases and constraints of the tracking limitations and biases present in current marketing practices.
Marketing Mix Modeling (MMM)
Marketing Mix Modeling was historically used solely by Fortune 500 FMCG brands, but companies in the mid-market can now use it.
Marketing Mix Models analyze the correlation between investments into every available marketing channel and the overall performance of an enterprise.
To do this, MMM looks at how much has been invested in all forms of media advertising in addition to what overall revenue has been generated through those ads.
In doing this, MMM tries to determine how well that media investment has performed, but it does not focus on individual customers and their actions.
Rather than narrowing their focus down to individual customers, MMM focuses on the macro inputs and outputs, with the added consideration of any impacts created as a result of seasonality and economic fluctuations.
Therefore, for enterprise-level marketers who want to understand the delayed and long-term fiscal effectiveness of their upper-funnel brand campaigns, MMM is an invaluable tool.
Executive dashboards for non-marketers
When providing an executive with a budget report from your marketing department, it is important to remember that they do not care about the cost per click metric.
Your executive cares about the efficiency of capital usage. How much can you accomplish with the money you spent?
If you present an executive with a 40-page report showing a bunch of metrics related to the effectiveness of your marketing, you are going to quickly lose any budget for your next campaign.
When designing executive dashboards, they should be ruthless in design and specific in purpose.
summary
The top level of any marketing report should fit on a single screen and should deliver the following three key pieces of information:
- Your total spend.
- The end results of all of the money you spent, such as revenues, pipeline, and market share.
- How effectively you used your investment, including CAC, LTV to CAC Ratio, and Return on Marketing Investment.
If the executive wants to see more details, you can provide them with more information, however, you need to remove the operational noise from their view of the report.
It should contain an easy to follow chronological view of the trends, graphical charts comparing targeted and actual, and some of the more high-level efficiencies pertaining to overall performance.
Diagnostic layer
The second level of your report should be focused specifically on your marketing leadership team.
The activities of each marketing channel must be further broken-down into regions and/or product categories and the metrics validating the direction of shifted budgets must be highlighted here.
Finally, the lowest level raw metric data should not be included on any standard executive-level dashboard.
Data thus far lives in the BI tools and platform interfaces, where those that are actually making media buys and managing campaigns have the ability to view their business' activities through a visual representation of that data.
As a result of this data overload and overstimulating environment, we are forced to ask ourselves some questions about our current practices.
Frequently Asked Questions (FAQs)
What’s the difference between a KPI and a metric?
Every KPI is indeed a metric, but there are very few metrics that serve as KPIs. A metric is simply a piece of quantifiable data that tells you something.
If I were to say to you that the bounce rate of my website has been reduced to 0%, the bounce rate is a metric.
But if I were to say that I have reduced my bounce rate from 80% to 0% through my organic search efforts, the specific metric I have identified has become a KPI.
The difference is, the KPI tells us if we are truly winning or losing, and it does this by telling us whether we are making good business decisions with our dollars.
Metrics are meaningless if they do not support board-level decision making; they serve as a means of diagnosis rather than as a KPI.
How do you identify vanity metrics?
Vanity metrics are numbers that may be impressively expressed but do not link directly to a clear and measurable business result.
The quickest way to determine if a metric is a vanity metric is to ask the question: "If this number increased by 2X tomorrow, would our revenue or profit increase?"
If the answer is "no" or if it requires numerous "if" scenarios to justify the correlation, then you have identified a vanity metric.
Common vanity metrics are total number of social media followers, raw page views, and total ad impressions unless they have a clear metric associated to measuring brand lift and downstream conversions.
What metrics are the most valuable for a B2B SaaS start-up versus an eCommerce company?
For an eCommerce company, revenue is dependent upon high-velocity transactional metrics such as Blended ROAS (Return On Ad Spend), Gross Margin, CLV (Customer Lifetime Value), and Cart Abandonment Rate.
Because of the velocity of these metrics, the time to receiving a signal is relatively fast and the transaction cycles for eCommerce brands tend to be shorter.
Conversely, a B2B SaaS company operates in a substantially different world due to the long and complex sales cycle.
The metrics that matter most for a B2B SaaS company will be focused on pipeline velocity and capital efficiency such as the payback period for Customer Acquisition Cost (CAC), Marketing Originated Pipeline, Lead-to-Opportunity conversion rates, and NRR (Net Revenue Retention).
Why is the information in our CRM consistently at odds with the information displayed in our ad platforms?
The ad platforms are designed for their benefit, with their proprietary attribution models in place to receive the maximum attribution of every conversion.
They attribute based solely on last-click or view through while ignoring all the other channels engaged along the buyer journey.
In addition, the ad platforms only track what occurs with the browser or app, which is often limited due to privacy updates, blocked ads, and cookie duration.
Conversely, the CRM tracks actual concrete business events, such as signed contracts and cleared payments.
When you are unsure about which is the accurate source of truth related to any business metric, the CRM is always the ultimate authority and the data from the ad platforms should only be used to help to inform future business decisions.