Dataprix

I am sure you have a new year’s resolution.

Perhaps it is that you’ll maximize the value you get out of your web analytics implementation (or should I say from your implemented web analytics tool since for many tools just implementing the tool is a multi year project!).

For all Analysis Ninjas this special post outlines how to get maximum value from Google Analytics. Tips, best practices, pictures, pointers, guidance.

Most people barely scratch the surface of their web analytics tools. My hope with this post is to get you beyond the Dashboard and the Search Keyword reports and the Referring Websites data.

[While this post is uniquely focused on GA these are things that you should do with any web analytics tool, be it Omniture or WebTrends or CoreMetrics or the one your mother-in-law gave you for Christmas.

Many of these features are, IMHO, easier to use in Google Analytics but they are available in other paid analytics tools.

In fact with paid analytics it is quite likely that you are also getting additional options, add-ons, drill-anywheres, etc that should exploit a lot more to get exponentially higher ROI - if you are not call your vendor and do ask for guidance to show ROI.

Net, net this post should help everyone, that's my goal.]

This a simple ten nine point checklist of Analytics Awesomeness.

If you do all of these ten nine things consider yourself an Analytics Empress/Emperor [aka Maximizer].
And if you do them well, ask for a raise, even in this economy.

If you do five consider yourself an Analytics King/Queen.

If you do three or less, consider yourself an Analytics Princess/Prince (aka the King of Newbies!).

Of course any less than that and you are a Newbie (not that there’s anything wrong with that!).

Ok that was, partly, just for fun.

Let’s go. . . .

#9: Get External Context to you Performance.

If you look at your own web analytics data you know how well you are doing. 70 miles per hour. If you have access to your competitive ecosystem then you know that while you are happy to move from 60 miles per hour to 70 miles per hour in 12 months, they have moved from 170 miles per hour to 190 miles per hour!

By enabling benchmarking in Google Analytics, you can view metrics for similar sites within your category. These benchmarks enable you to identify areas of opportunity relative to the performance of your competitors.

Benchmarks are available for Visits, Bounce Rates, Page Views, Average Time on Site, Page Views Per Visit and Percent of New Visits. Sweet!

I am loving this data because I am incessantly focused on getting New visitors, and looks like I am doing 17% better than competitors. Happy birthday to me. :)

And bonus tip, you can choose the category of your web based business.

Simply click on the Open Category List link in the Benchmarking report and drill down into the relevant available three levels.

For example: Computers & Electronics -> Internet Software -> Content Management.

Make the benchmarks unique to your own business vertical / category.

In addition to the GA data I also wanted to point out the most excellent Coremetrics benchmarks report. Very sweet data there as well, please bookmark.

Helpful Google Analytics Resources:

How do I enable benchmarking?
Data Sharing Options, Technical Goodness, Deep Details.

Finding your Benchmarking reports:
Visitors > Benchmarking

Helpful Occam’s Razor Articles:

Context Is King Baby! Go Get Your Own
Lack Management Support or Buy-in? Embarrass Them!

#8: Internal Site Search, baby! Rock it!

My undying and eternal love for internal site search data is in one word: legendary! :) I do heart it so. And it comes in at #8.

There is perhaps only one single place in your entire web analytics data where there is no question about visitor intent. Its the queries they are typing in your website’s search engine. At all other places in your data you are guessing and inferring.

Extract every little drop of nectar from this data.

It can help you understand visitor intent and identify new opportunities to: improve landing pages, cross-promote products, and adjust your creative strategy.

Helpful Google Analytics Resources:

How do I set up Site Search for my profile?
Metrics definitions, how to use, trends details & more

Finding your Benchmarking reports:
Content > Site Search

Helpful Occam’s Razor Articles:

Kick Butt With Internal Site Search Analytics

#7: Search, Organic, Get Good At It.

Pure web analysts tend to focus on referrers and campaigns (DM type) most of the time. There is usually someone who does “search” and that usually means Paid search. Organic search analysis is usually an orphan (or there is someone doing that some of the time).

You want your boss to love you? Focus hard core on Organic search. It is free, it is a long term investment and can be the foundation of a great Paid search strategy.

GA allows you to wonderful analysis of your Organic search data. Torture it every which way.

Don’t know how to quantify impact of SEO to get Management support? See this article by e-nor: Leverage Google Analytics to Monetize your SEO Effort .

Don’t know how to get your organic results page ranking shown in GA? See this article on my buddy Joost de Valk’s blog: Track SEO rankings with Google Analytics .

And that’s scratching just the surface.

But if you want to do SEO Analytics than you need to get out of your Analytics comfort zone (or Omniture or CoreMetrics or ClickTracks comfort zones) and use Webmaster Tools. [Google Webmaster Tools, LiveSearch Webmaster Center ]

Think of it this way. Lots of links show up when people search for you (oh the humanity of it! :)) and your Analytics tool only shows you data for people who show up at your site. Webmaster Tools (especially the one from Google) shows you a wealth of info that you would never see in your Analytics data.

There is a ton of good stuff but without a doubt my favorite for analysis:

Where do your website links show up? For what keywords do you actually get traffic for? Awesomely actionable data.

Compare your trends over time (click on that down arrow next to locations), compare your searches from different platforms (like Mobile Search) or for different kinds of content:

It is a shame that few Web Analysts actually know of webmaster tools and fewer still use the data to help their companies rank better.

You want to stand out amongst the sea of Analysts? Get good at things like this.

Helpful Google Analytics Resources:

Overview of Traffic Sources Reports

Finding your Organic Search data:
Traffic Sources > Search Engines > toggle “Show” option to “non-paid”
(Click on each record for an engine-specific list of keywords)
Traffic Sources > Keywords > toggle “Show” option to “non-paid”

Helpful Occam’s Razor Articles:

I clearly need to write more about SEO here, meanwhile refer my book please, lots there.

#6: Landing Pages, Landing Pages, Landing Pages.

I always get asked this: “If there is one thing I can start doing in terms of Web Analytics would would it be?”

My answer is always the same, Bounce Rates for Top Landing Pages:

Thing that suck are simply standing there, naked, staring at you. I challenge you to look at this report for your site and not find broken things.

Could be campaigns, could be landing pages, could be wrong calls to action, could be…. you find that out. What this report gives you is actionable impactful starting points.

Then fix, fix, fix (and use the free Website Optimizer!). As a good friend of mine says: “You never get a second chance to make a first impression”. Cheezy? Yes. True? You betcha!

Helpful Google Analytics Resources:

Understanding & Leveraging Content reports
A/B & Multivariate Testing with the Website Optimizer

Finding your Landing Page report:
Content > Top Landing Pages

Helpful Occam’s Razor Articles:

Measure Effectiveness Of Your Web Pages

#5: Goals, Goals, Goals [& Goal Values]!
[Bonus points for crimes against the universe if you have not assigned Goal Values to your Goals.]

You: What should I measure?
Expensive Consultant: What are you solving for? That’ll be $500.

:)

If that $500 makes you think of why in the first place you created the website then wipe your tears away on that ten second consultation. It was totally worth it.

Define your goals and measure ‘em.

And here’s something worth $1,000. It will be extremely rare that you’ll have one goal. No website exists for one exclusive purpose. Think hard, identify all the goals, and measure them.

Online clicks drive awareness, customer loyalty, offline sales, and yes even online sales! Make sure to track all actions on your website that help your business.

Don’t discount the importance of tracking even things like: store locator pages, email sign-ups, maps on site, feed sign-ups, leads into your CRM pipeline, request for quotes, etc. Even sexy stuff like Loyalty and Recency (you non-ecommerce website owners, are you listening?).

While you can’t create a “Goal” for Visitor Loyalty and Recency, you should create goals (as in “the goal of our BBC website is to have 50% of the visitors visit more than 15 times a month”) and then measure success using your GA Visitor Loyalty reports.

Goal values should be assigned to each goal. For example if you submit a qualified lead on www.cat.com for a Hydraulic Excavator M322D (starting at $300k) then that through careful financial analysis you can determine that each web lead is valued at $189 (based on past conversions, offline valuations etc).

Make sure you type that into Google Analytics as you set up a goal to track lead submission. You will get “revenue analysis”, you will get “per visit goal value”, you will get “$ Index” of each page view (!) and more yummy goodness. Do this!

[The above image is from a non-ecommerce website where the goals are both online and offline. Each goal has an assigned goal value allowing the owner to precisely compute the economic value generated from the website - $14,930 in Dec.]

Helpful Google Analytics Resources:

How do I set up goals in Google Analytics?
Goals & Funnels Overview, Troubleshooting, How-to for non-HTML pages

Finding your Goal reports:
Left Nav > Goals (overall conversion & funnel reports)
Conversion rates are also available on most Visitors & Traffics Sources reports (select the Goal Conversion tab, instead of Site Usage)

Helpful Occam’s Razor Articles:

Tracking Offline Conversions: Hope, Seven Best Practices, Bonus Tips
Excellent Analytics Tip #13: Measure Macro AND Micro Conversions

#4: Enable Deep Ecommerce Tracking & Analysis.

With the simple radio button click on your GA account profile and addition of a simple javascript tag to you ecommerce page you are set to reap benefits of some deep deep analysis of your ecommerce data.

Sadly not that many people do this.

Just look at the wealth of information you can get, about conversions, revenue, product sales, meta categories…. and even true “pan session” insightful analysis like Visits and Days to Purchase!

With GA the ecommerce tracking is actually quite feature rich. You can use it to track other things on your website.

For example if you are a non ecommerce lead generation website then you can capture rich transaction data for those leads and even track how many visits/days it takes someone to give you a lead.

Our good friend Justin shows you how to do exactly that here: Tracking Lead Gen Forms.

Of course I have not even touched in how much more magnificent your campaign tracking will be or how every advertising and sales channel can be held significantly more accountable now.

Helpful Google Analytics Resources:

How do I track e-commerce transactions?
Indepth FAQ and Troubleshooting Guide

Finding your E-commerce reports:
Left Nav > Ecommerce (overall revenue, transaction & product reports)
E-commerce data is also available on most Visitors & Traffics Sources reports (select the Ecommerce tab, instead of Site Usage)

Helpful Occam’s Razor Articles:

Conversion Rate Basics & Best Practices
Excellent Analytics Tip#6: Measure Days & Visits to Purchase
I Got No Ecommerce. How Do I Measure Success?

#3: Your Only True Analytics BFF: Bounce Rate

Even by the end of 2008 I am astounded at how few people know what the KPI (yes it is a KPI!) Bounce Rate truly means and how delightfully easy it is to understand and deploy in your war against waste.

Bounce rate is simple, easily accessible and should be used as the first point filter for “initial engagement”, “quality traffic”, “crappy pages”, “efficient campaigns” and other such key desires.

In this economy if you are not truly a BFF with bounce rates you are leaving too much of your scarce money on the table.

Helpful Google Analytics Resources:

Bounce Rate clarified

Finding your Bounce Rate:
Bounce rate is a metric on most Visitors, Traffics Sources & Content reports

Helpful Occam’s Razor Articles:

Bounce Rate Tips, Usage & Best Practices

#2: Powerfully Leverage Custom Reports.

You are unique, yes even special :), why should your reports not be? Why be saddled with 80 reports from your Web Analytics Vendor when none of them quite contain exactly what you want?

Or what about those crabby patties in Marketing who won’t look are any report and you want to give them just a two column report with just the KPI’s that they need?

For all of those reasons and more you need to use Custom Reports.

All you need are two things: 1) Some understanding of what your users want 2) Drag and drop skills. . . .

A unique report for your unique needs.

Note that I have put in one table KPI’s and Metrics that are otherwise scattered all over the place in GA. No more hunting and pecking!

Also note the other cute thing you can do with Google Analytics. You don’t have to create a massive puke of reports. You can create a tab with just metrics for the Sales Team. Another tab in the same report with Marketing Team metrics etc.

I call it creating “micro ecosystems”. You have a fewer reports, and they are significantly more targeted. Try it, you’ll love it.

Promise me, you’ll never look at a standard report ever again.

Helpful Google Analytics Resources:

Quick start guide & video for Custom Reporting
Overview, technical sweetness & troubleshooting tips

Finding your Custom Reports:
Left Nav > Custom Reporting

Helpful Occam’s Razor Articles:

Web Metrics Demystified
Eight Rules for Choosing Web Analytics Key Performance Indicators

And, drum roll please. . . . the #1 thing to do with Google Analytics is. . . .

#1: Give Me Segmentation or Give Me Death!

Were you surprised at my #1 choice? I think not.

Aggregated data rarely provides the insights you need. That’s because lots of different people come to your website for different purposes and and exhibit different kinds of visitor behavior because of different drivers and motivations.

If that were not enough you are trying to get certain goals achieved through different strategies and electric shocks.

All this makes for a potent soup and your only way to truly being a Analytics Emperor is to be really really really good at using Segmentation.

Use Google Analytics to focus on visitor persona’s you care about, measure the degree of visitor engagement, identify high value sources, magnify desired customer behavior, or simply identify the “whales” who buy tons of stuff from you.

How to? See specific actionable tips in the Occam’s Razor article linked below.

Helpful Google Analytics Resources:

How do I create and use advanced segments?
Helpful FAQ’s on Advanced Segmentation inc regex

Finding your Advanced Segmentation reports:
Left Nav > Advanced Segments (under Settings)
Segments are also available from the drop-down menu at the top right of the interface (default: All Visits

Helpful Occam’s Razor Articles:

Excellent Analytics Tip#2: Segment Absolutely Everything
Google Analytics Releases Advanced Segmentation: Now Be A Ninja!

There you are.

Nine amazing things you can do to truly leverage a free web analytics tool like Google Analytics (or a paid analytics tool like WebTrends or Omniture or your favorite).

Do this and you and your company will be fine. More than fine.

If you are not doing all of these things, you are simply collecting data. You won’t be fine.

Good luck!

Now your turn.

If you had to maximize the value you get out of a web analytics tool what would be your advice?On our suggested scale, how would you rate yourself? A Prince? King? Emperor? : )

Please share your insights, critique, love (and self ratings!).

PS: Before we go I would like to thank two beautiful people for their help: Matt Parry & Rachel Meyers. You rock!

Copyright © : 2006-2008 Occam's Razor by Avinash Kaushik | All Rights Reserved

Google Analytics Maximized: Deeper Analysis, Higher ROI & You

Admit it, you secretly live in the fear of your Senior Management finding out that your online greatness is less a result of your online campaigns and more a result of the tons and tons your company has invested in the real world.

The real world. “Offline” to you and me. :)

We tend to often overlook the pesky offline real world. Sooooo booorrring ! (Say that with a Paris Hilton’ish brush off. :)

I think most of it is not malicious.

For one thing it is really hard to measure. For another thing your “online” presence is probably three geeks (says the proud geek!) living in Seattle and your “offline” presence is 15,786 people in the company with power concentrated in New York and Atlanta. Hard to coordinate and get “them” to listen to us and pay attention to us.

But the world is not online or offline, it is nonline (hat tip to David Hughes for that magnificent term). One of these day everyone will get that.

Meanwhile you are the smart one, know that it is important that our “online analytics” morphs into true Multichannel Analytics, i.e. non-line analytics.

Recently we had covered how to measure offline impact of online activities.

In this post, dedicated to Sandra Dowker, we’ll cover the reverse: how to measure the online impact of offline campaigns / activities.

Then you’ll have a blue print for doing nonline analytics: Offline -> Online -> Offline .

Why should care about offline at all?

For most companies radio, tv, newspapers, magazines, catalogs, retail stores, call centers, etc still form a bulk of their business advertising and marketing expenditure. All these touch points have a online impact (intended or not).

The online channel often, though not always, can be a lower cost channel. Showing exactly how to use it (which campaigns, mediums, products) and for whom (visitor personas, geographies) and why (recession baby!) will ensure your job security and the channel’s security as well.

Know that pesky “direct” (or in some web analytics tools: “unknown”) bucket that you never seem to be able to understand? Many of those visitors are certainly “url typers”, but depending on your business a good chunk of them could be visiting as a result of your offline activities. Give them due credit.

Culturally there is no better way to get your company (HSN or NFL or Dell or B&H Photo & Video or 1-800-Flowers or….) to pay attention to “lowly and under appreciated” web analytics than show you can quantify the online impact of their massive (or little) offline spend.

I have never known a better strategy then to understand and align yourself with the largest money maker in your company (and no its not online). So there.

Convinced?

Sure you are.

If it is not the enticing prospect of understanding the data better then it has to be craving to be loved by your greater organization. :)

So what’s the problem here?

Like in the case of the offline impact analysis, the problem here is also one of the missing primary key (see that post please if you don’t know what this is).

Our goal is the figure out how to tie your visit to our website due to a stimulus from a offline campaign. How do I know that it was a tv ad that drove you to the site or a magazine article or a banner by the side of a road etc.

Once you have collected that piece of data you can do any of the other analysis you want for that offline to online traffic stream, be it conversion rates or site abandonment rate or task completion rates or even non-ecommerce outcomes.

Let’s get going. . . .

Tips for measuring on-line impact of the offline channel:

Here are the most common offline-to-online movement channels and how to collect data for end-to-end analysis of each:

#1 : Use redirects (vanity url’s).

The grand daddy of them all. Plaster your billboard (or magazine article or tv ad) with a easy to remember url and boom (!) you got yourself some tracking.

A magazine ad with the call to action Visit www.usequickbooks.com redirects to www.quickbooks.intuit.com/tracking_code=newsweek_dec_2008 The rest of the analysis is a piece of cake (simply segment out visits with that tracking code).

A box of 12 krispy kreme doughnuts to your IT person will ensure all redirects are coded with the correct tracking code.

If you are super cool like my former employer then you will have web based interface where the Online or Offline Marketer can attached a tracking code to any url, hit save and be in tracking heaven!

Another example:

www.dell.com/tv

(today, dec 22 here in Maldives) redirects to

http://www.dell.com/content/topics/segtopic.aspx/tv?c=us&cs=19&l=en&s=dhs&keycode=6Vc94&DGVCode=TV&dgc=TV&cid=11510&lid=985367

Note the amount of specific tracking that the folks at Dell have attached do that redirect. Bravo folks!

Contrast that to HP which has a ton of tv ads running and has a working redirect that leads to no trackable information:

www.hp.com/tv

to

http://www.shopping.hp.com/webapp/shopping/home.do

Missed opportunity.

In closing it is a crime of the highest proportions if your magazine, catalog, tv, radio, bus, billboards are not using:

1) easy to remember vanity urls and

2) these vanity urls are not permanent redirect

3) they are encoded with the correct tracking parameters (version of the ad, name of the magazine, location of the billboard, offer in the radio ad, etc etc).

That was not hard right? You are on your way to measuring online impact of your offline channel!

[I can sense the Smarty Pants amongst you snickering at the prospect of data pollution because people posting your vanity url's online. That is the reason #2 above is important. Permanent redirects (301's) will pass the referrer's data to your website. You can then split out online referrals from the offline referrals (offline will have a blank referrer).

For example the Analysis Ninja at Dell will easily be able to split out visits from my blog from clicking the links above, www.dell.com/tv, and easily exclude. Now stop snickering.]

#2 : Use unique redeemable coupons / offers codes.

Pretty much all multichannel merchants now do something similar to this. . . .

The offline mediums (a magazine in this case) carry unique promotion (or offer or config) codes that must be typed into a form on the website. This allows those visits to be tracked as being from a external “motivation”.

This works well for tv, magazines, radio, catalogs and other such mediums where it is easy for people to remember the codes.

For example the ads for www.1800flowers.com in the NYC Taxi Cab I was riding close to thanksgiving asked me to use the coupon code taxi to get $5 off a $50 order. It was easy enough code for me to remember it and use it.

www.qvc.com and www.hsn.com also tend to use these types of tracking mechanisms as one of them tools in their arsenal to track people who buy on the site while watching the tv show (or later).

Catalogs for pretty much every major or minor company, including yours I am sure, is using this exact strategy as well by providing unique coupon or offer codes.

Bonus Tip: Another great strategy is to use the same coupon code between your channels. For example you are giving $65 off the Apple Ipod Touch. Your ads / catalogs / tv campaigns can say something like “call 1800 Hot Hot Now or visit Hot Hot Now . com and use the code ipod65“.

The benefit of this is that you are providing people a choice in terms of channel preference (use the phone or the site if you want) but since the code is the same you can track it delightfully later. This helps you understand channel preference by media type (tv / catalog / radio) and also by product type (electronics / food / meds etc) and by region (what’s up with people in Florida 100% using the phone channel to order their Viagra) etc etc.

#3 : Use online surveys / market research.

On this blog we have often talked about using onexit surveys (4Q or others) to understand Primary Purpose, Task Completion Rates & Segments of Discontent (see: Three Greatest Survey Questions Ever).

You can easily adopt that methodology to ask two more questions of your website visitors:

“Which of the following were the source of your visit to our website?” [The answers can be: A tv ad, A radio spot, A google (:) search, You received a catalog etc, wordsmith your options.]

and

“What is the likelihood that as a result of a visit to our website that you’ll make a purchase?” (or sell your kidney etc) [The answers will be something like More likely, Less likely etc.]

These two simple questions, drop down single choice, will help illuminate both the drivers of visits to the website (and a real chance here that you’ll explain your very high “direct visits” number here) and also the preference in terms of the purchase channel (and of course you can use this for tech support or other non ecommerce websites as well, just twist the second question a smidgen).

In our online to offline article I had talked of the possibility of using primary market research to understand outcomes that happen offline. You can also complement surveys like the one above with primary market research to understand channel use by your customers.

#4 : Correlate traffic patterns with offline ad times / patterns.

My wife was watching HSN the other day (that is how hard it is to find something to watch on tv in the age of 900 channels!) and saw Wolfgang Puck talk about his genius kitchen knives and a “extra special deal just for you right now for only $19.99 plus shipping and handling”. (Imagine that with a Austrian accent.)

So Jennie of course went to the website instead of the phone and two minutes later (well a week actually) we were proud owners of Wolfgang Puck limited edition extra special for a limited time only with a chopping board knives (with knive covers!).

Now this is not a unusual customer behavior. Offline media stimulus causes us, lemmings like, to run to the site and do stuff.

Yet it is extremely rare that a Web Analysis Ninja sit down and overlay the company media plan on top of the traffic patterns and deduce the impact online of the offline media spend.

Why not?

Sure you have to beg, plead, and practically sleep with someone to get your hands on the comprehensive media plan for the company. But take one for the team and do what you have go.

Once you have the magazine / tv / catalog / postal mailings / radio / billboard plan for your company then do the correlations with your website traffic and see what the impact is.

My tip would be not to just look at overall traffic (or All Traffic) for the site, you may or may not detect something. Correlate it with your Direct Traffic. You might see a sudden spike in traffic there. Or with Direct and Search (organic or paid) referrals. When you do stuff on tv / radio / retail stores people search (what can I say!).

I had covered exactly this strategy in this post: Excellent Analytics Tip #12: Unsuspected Correlations Are Sweet! You can find out exactly how to execute this analysis in your company from that post.

Here is a picture of the correlations that I had shared there (for one company who ran radio campaigns and the resulting impact, not just directly from visits from the vanity url mentioned in the radio ad but also from Direct Visits, Branded Search etc, very impressive, and surprising, holistic outcome):

Please see that post for detailed explanation and guidance.

Of course it is always most optimal to identify causality as well (because correlations don’t always mean causality). The keywords you drill down when you see the search spike perfectly matching your offline campaigns for example, that has intent. Or you are perhaps running the survey I just mentioned, that will show causality. Or you sold a bunch of Puck knives exactly when, or slightly after, the tv ad ran, that will indicate causality.

Do that.

UPDATE: #5 : Use the power of controlled experiments!

Jim Novo’s comment below reminded me that I forgot to add this super awesome way to measuring multi channel impact.

I had covered it as a key strategy in my multi-channel analytics post, tip #6 for measuring impact of online to offline outcomes.

Please see that post for more details. I share stories about using newspaper inserts or conducting geographically isolated experiments in retail stores (or fast food restaurants) etc.

The strategy works for on-line to off-line, and it works even more brilliantly for off-line to on-line.

.

There you go. Four Five simple things that anyone can to to get started on their journey to identify online impact of their offline strategies.

Remember your goal is to identify the complete picture: Offline -> Online -> Offline.

Those of you who have my book, Web Analytics: An Hour A Day, will know this handy dandy reference picture from page 235… it shows how to track your nonline world efforts and capture the key pieces of data to do true multichannel analytics:

You can use this multichannel value analysis framework to plan out how you will make sure you are passing the various primary keys and forth and also use all possible techniques at your disposal.

Ok now your turn.

What are the strategies that you have use to measure online impact of your offline campaigns? Have you used any strategies above? What has worked for you?

Please share your experiences, your best practices and tips.

Thanks.

PS:
Hello from the absolutely gorgeous Maldives, more specifically the Conrad Rangali Island. It is lovely here, warm, the staff are wonderful, snorkeling is great (and I am committed enough to all of you to still squeeze in a, hopefully, valuable blog post!).

Here are some pictures. . . .

This is one of the two islands that forms the Conrad:

You take a sea plane from Mali to get there, this is one of the island resorts from the air:

A typical sunset:

Lots and lots of snorkeling, absolutely wonderful, and my one high was running into a 4.5 ft shark. It passed by my nose, I was absolutely petrified, it was not [Tweet]. Here’s a small one close to the shore:

Tiny little hermit crab, hard trying to get this close to it without scaring it:

Other than that lots of personal time (also hard to photograph):

Hope your holidays are fun wherever in the world you are.

Ok, don’t forget to share your online to offline analytics tips using the form below!

Copyright © : 2006-2008 Occam's Razor by Avinash Kaushik | All Rights Reserved

Multichannel Analytics: Tracking Online Impact Of Offline Campaigns

Michael, politely, says in an email:

“I have done web analytics for five years, I have mastered Omniture, WebTrends and Google Analytics, I provide analysis and not just reporting. I feel like am an Analytics God.

What would be your advice for me in terms of next steps for my career? My goal is to climb the ranks and increase my salary.”

Let me hasten to add two things.

Michael is not his real name.

Modesty aside, :), Michael is good at what he does.

I get many emails in the spirit of this one and thought it was about time I wrote a proper post about it.

Another reason for writing the post now is that it is always a good time to think about your career path, but never more so than the current economic circumstances. Some of you face tough times, some might get laid off [see end of this post], some might make opportunistic leaps. Either way good time to ponder, do some self reflection and make a conscious choice.

Before we get going some assumptions I am making:

1) You are an “Analyst” (Senior, Junior whatever). Or atleast 40% of the time you are a true Analysis Ninja, even if 60% of the time you are a glorified Reporting Squirrel!

2) You might have some project / task management experience, your leadership experience is limited to that.

3) I am simply assuming you are good at tools and some technical stuff and some business stuff. When Michael says he is good at Analytics his stress is on his mastery of javascript tags, his rich understanding of evars and sprops and complex 60 kb Omniture tags. He can implement anything in his sleep.

4) You realize that there is more to life than creating reports and trying to explain KPI’s. It is ok to want more money and be aggressive about your career but know that it won’t happen unless you vastly expand your horizon on the work you’ll do (and how hard it will be).

Update: 5) You are at a mid to large/bigger company. Please see this comment for context around why.

Every Web Analyst (or really Business Analyst) of any sort finds themselves at that critical point. Have been doing analysis for a while, now where does my career lead me?

The first and perhaps most important thing to realize that you have to make two very important very critical very life impacting choices:

Choice 1: Business or Technical.

Choice 2: Individual Contributor or Team Leader.

Each choice will help propel your career in a different direction (slope and length). Typically we don’t think that you have those choices (we all want to jump to Director / VP and get the chq, not so fast buster!).

Do some introspection.

Here’s the Avinash Kaushik Web Analytics Career Introspection Guide! Answer these questions:

#1. Do you like being a Individual Contributor?
Think these things through: Master of my domain. Controller of my destiny. I like setting agendas, let other people deal with people who have to do it. Truly am at peace with my introvert self. And so on and so forth. Be honest with yourself.

#2. Do you like managing people?
You rejoice at the prospect at helping mold people’s careers. Motivating them. Solving their personal problems. The prospect of collecting self reviews and getting 360 degree feedback and writing a performance review for each of your employee each quarter does not make you want to jump off the building. You see a matrixed bureaucratic organization and like President Bush you say “bring it on!”.

#3. Are your true “Analyst” skills your massive mastery of how to solve every technical problem with every tool and how to implement anything and you could decode and reconstruct the debugging tool WASP in two days? You can hack the Google Analytics tag to capture people’s underwear size and color.

#4. Are your true “Analyst” skills your understand of your company’s business strategy, your mastery at translating “measure something” from a VP to three Critical Few metrics that bedazzle her, your ability to understand the long tail and get a ah ha moment that revolutionizes how you understand and measure of your search campaigns?

I am getting to how you can increase your salary part. Please stick with me.

Answering the above four questions honestly and critically is much harder than you think. Trust me on that.

At Intuit I truly learned the value of self awareness. Steve Bennett prioritized Management development (I’ll be eternally grateful to him for that) and my friend Scott Wilder really got me going on this path by doing my Enneagram assessment (MBTI is too shallow).

To me self awareness is the process of figuring out what you are truly good at, and really truly knowing (and accepting) what you are not good at. It takes time to get going, and is more of a lifetime journey and less a destination. It comes with great benefits. For example, I have learnt to maximize my being in roles / situations where my strengths will boost success.

Your answers to the four questions will ensure that you don’t end up in a job that 1) you’ll hate or 2) where very quickly you’ll rise to your level of incompetence.

There are jobs, based on your choices above, either in companies or, many people forget this, at web analytics vendors.

If you want to have a move your career forward in web analytics (from a Metrics Analyst) here are the four options for you (and yes they all will help you make more money, some more than others):

|1| Technical Individual Contributor.

A lot of people wrongly believe that to make lots of money you have to get into Management (technical or business). This is totally wrong. For the longest time, for example, I was well compensated for being a Senior Individual Contributor.

Roles in this category would include: Sr. Project Manager. Sr. Architect. Implementation God. Sr. Tech Lead. Tech Demo God (usually at a Vendor). And more.

For this role at a client (company) world the common theme in this role is that you report up to a Director or a VP and you get to set policy, rules and regulation, only have, if at all, the barest of dotted line responsibility for project implementations, you might be the master liaison with the business team and your vendor to make sure technically all that is supposed to be happening with the technical tag hacking and tool hacking is happening.

This role in the vendor world you get to go to various clients and show off cool detailed stuff that your VP of Marketing consistently screwed up so far and answer technical questions from wise guys. You might also be the one man army tapped to do rapid prototyping to prove you are better than Google Analytics (!), or it is likely that you are the point of contact for the first sixty days for a new client when your company is trying to impress the client by providing fast help (as the payment chq has not yet cleared). Make no mistake this can be fun, you get to travel, meet new companies and people.

It is quite likely that you’ll sit in the IT (CTO / CIO) function.

Career Prospects:

Pretty sound in large to larger companies. They can afford such a person in a dedicated manner. In a vendor world (say Omniture, WebTrends, ClickTracks, CoreMetrics etc) you probably have a lot more jobs of this type. It would be harder to find these roles in medium to large companies.

$$$ Prospects:

Anything from $40k to $100k (or more, at vendors). It is hard to find people who are really really good at this. If you are one of them you are in demand.

Long Term Job Title Growth:

This ones a bit dicey. If you stick to web analytics your title might tap out at one of the titles mentioned above (say Sr. of this of that or Architect) - remember that does not mean there isn’t a long future and plenty of hay to be made.

If you really want to have your Job Title grow a lot more then you’ll have to gradually move to the world of Business Analytics (not web) and Business Intelligence roles in IT. Both of these not just provide individual contributor title growth, they provide for easier switches to other leadership roles (should you show promise).

|2| Business Individual Contributor.

If you are a Analyst today you are in a individual contributor role on the business side (if you are a Web Analyst in IT the best career move you could make for yourself is to get moved to Marketing - or a business function, it is really hard to have a strong Analyst - not reporting squirrel - role in IT).

Roles for you on the business side in this category would include: Sr. Analyst. Internal Evangelist. SPOC for CMO / CEO dashboards (supreme analysis). Central Business Liaison (for a large business, focus on getting people to implement web analytics and get going). Strategic Solutions Consultant (clearly with hype like that this a role at a vendor!). Product Genius (at a vendor, perhaps Apple :).

For this role at a company (client) the role can report to anyone from a Director to the CMO. Your job is heavily business focused - understand various businesses and their strategy and provide über analysis (pan business function) or create dashboards or be in charge of rolling Omniture across 90 business sites (beat them up until accomplished).

There are a rare few roles where you can become the internal Analytics Champion, I did this for a while. You are good at your analytics “game” but you are also a strong business person (I hate to say this but MBA / “strategic” type). You get to go around and work with VP’s, CMO’s and Sr. Leaders and identify measurements strategies for their impossible to answer questions (often they don’t know how ease these are so you totally look like a hero). As an Evangelist you pull your organization up by the bootstraps (quite gratifying).

This role in a vendor world can mean you are a product manager of the analytics product, you are a project manager for certain features, you are a professional services rep (sorry, “Strategic Solutions Consultant”) and roles like that. No one does Marketing (with a pinch of hype :) like Omniture, this excellent page on their site will help you understand what a business individual contributor role might look like at any vendor: Omniture Consulting.

For many of you there is also a option of a role I play now, an Evangelist. A business individual contributor role with significant influence over the vision and the product, as well as an opportunity to impact the external Analytics ecosystem.

Career Prospects:

Except for small companies you have lots of room to grow in this role before you hit a ceiling. Either at a company or a vendor. The only condition is that your have to be a very very strong business person. Understanding ecosystem. Business strategy. Trinity type execution of measurement. Smooth talker (sorry, “effective communicator”) etc. Your deep understanding of statistics etc is not required. Javascript hacking skills are optional. If your strength is technical see the role above, or the one below, and you won’t hit the ceiling in six months.

$$$ Prospects:

Anything from $70k to $120k (or more, at vendors or companies). From my humble experience in our little world, less than 10% of the people in our field truly have the skills to do this well. If you are one, congratulations.

Long Term Job Title Growth:

One nice thing about being a Individual Contributor on the business side is that you are afforded a lot more flexibility. To become a internal consultant on business analytics projects (beyond web analytics). To even switch to leadership role (team management). To tackle other complex things for a company, like creating a “data strategy” or becoming the chief privacy officer (a individual contributor role) etc.

On the vendor side you also have a lot more opportunities to have job title growth (remember that comes with increased responsibilities). My friend Matt Belkin at Omniture is a good example of this, over the years he has had a fantastic career there.

|3| Technical Team Leader.

Roles in this category would include: Manager, Analytics Implementation. Sr. Manager, Website Analytics. Group Manager, Web Operations Reporting. And still rare but sometimes: Manager, Web Analytics Data Warehouse (Steven I did not forget you!).

In the early genesis of web analytics, in the good old days, it was owned by IT this role was a lot more prominent in companies. Having WebTrends almost mandated that. The shift to ASP (javascript based) solutions (crediting HBX here) caused a shift of web analytics to the business side (an excellent outcome). It also eliminated the need to have a large IT staff to support web analytics.

An example is six years ago when I took over web analytics there was a four person team (one leader, three direct reports) in IT supporting just running WebTrends internally and churning 200 reports out. The shift to a asp based solution meant only one job remained and it became that of a Sr. Technical Individual Contributor (and I was lucky to have a very good one!). The other jobs were evolved or replaced with people who did analysis not reporting (an efficient use if there ever was one).

Roles in a company setting would be reporting up to Sr. Manager or Director (or rarely VP/CIO levels). Often you’ll find yourself in the Business Analytics team in the CIO / CTO function, you take care of that “web data”. :) In some companies there is also sometimes a role in the Web Analytics team that is in Marketing (/business) where you can carve out a nice technical team lead career (reporting to the Director of Web Research & Analytics :).

Roles at a vendor probably have a lot more technical team lead opportunities. Managing technical aspect of the analytics product or managing the technical army of consultants or things like that.

It goes without saying that this role requires something really really hard: Your ability to leave your leave your lone ranger mentality and the deep rooted habit of just doing all the technical stuff yourself (yes you are sooo good at this stuff and you don’t trust other people’s code).

It is harder for technically oriented people to blossom into people managers, but really that’s what you are signing up to do. You are going to have to be comfortable with some of your awesome hacker skills getting rusty as your leadership skills (and delegation!) mature.

The natural next growth in a company environment would be to

Career Prospects:

If your company is using a ASP based solution (Analytics or IndexTools or Unica etc) then be aware of the aforementioned fundamental shift and the limiting impact of that on your career if you make this choice.

Some companies have inhouse (hosted) solutions (javascript tag based or log file based). In this case there is still a need for a robust multi person technical team inhouse. The opportunities are a lot less, but in those cases you can have a web analytics technical team leader role that will last a while.

You are going to live or die with your ability to inspire and motivate people, not your ability to write code or keep systems up. That will limit

$$$ Prospects:

Anything from $50k to $100k (or maybe more for inhouse WA DW type roles).

Long Term Job Title Growth:

Not too much if your company is in the ASP based model (and remember ASP is not just for WA, it is now for testing, behavior targeting, surveys, electric shocks, everything!).

For inhouse implementations (or DW extensions) you can expect nice growth. Both if you stick with WA or moving to say taking over technical leadership roles on the CRM side or Supply Chain or ERP side of things.

Good technical team leaders are hard to find, if your technical skills today are awesome and you are willing to truly grow your people management skills you’ll be God. [Related post: Three “Spire’s” of Great Leadership.]

|4| Business Team Leader.

When people think of making more money in web analytics jobs, 99% of the time this is the role they are thinking of. [Might I just quickly again encourage you to use the Avinash Kaushik Web Analytics Career Introspection Guide first.]

Roles in this category include: Sr. Manager, Web Analytics. Director, Web Research & Analytics. Manager, Web Metrics. Team Lead, CoreMetrics Reporting. Group Manager, Analytics & Optimization. Etc.

This role in a company setting is increasingly reporting to Sr. Directors, VP’s or, in companies that get it, the CMO. Ideal candidates were Analysis Ninja’s of supreme kind and have shown streaks of good people leadership skills. They are motivators, can inspire confidence, are inherently unselfish (key for leading people) and have the ability to charm the pants of Sr. Management (though come to think of it that might be a HR violation!).

I cannot overstate this enough: Ideally you have grown from a Reporting Squirrel to an Analysis Ninja, but your hard core technical skills are vastly overrated in this role. As is your ability to be, say, a Excel Master Blackbelt. Remember, you inspire and you lead.

This role, in a pure web analytics leader fashion, is a lot less needed or visible or available in a Vendor setting (unlike the other three above). Vendors need to mostly sell. Perhaps to analyze nedstat.com or unica.com for internal use. Or perhaps as a business lead for the $300 per hour consulting arm.

Career Prospects:

You have lots of room to grow here. If anything web analytics is becoming more serious for lots of companies (damn the temporary recession!). Having had just Squirrels manage web analytics any company worth anything is looking to put solid leadership in place.

Your limitation will be if you stay with just Web Analytics (clickstream) or you have an ability to truly do Web Analytics 2.0 (move beyond clickstream). You do the latter and you won’t run into ceiling anytime soon.

If you do hit the Director of Web Research & Analytics (or VP in a large company - title inflation :)) then you might hit limits. Your option then is to shift to being a business leader and run a business. Or other such options.

$$$ Prospects:

Anything from $90k to $170k (or more). The nice thing is strong people leaders with analytical minds are the rarest of rare in corporate America (not quite as rare as a Tenrec but close). If you are good you have no limits.

Long Term Job Title Growth:

If you want to stay with web analytics you’ll tap out at Director (or in a grade inflation environment, a VP). But as I mentioned below strong business executives don’t really have a ceiling.

That’s it. Four different job families. Each unique in its job, salary and future prospects.

My fondest hope is that as you evaluate your career that you’ll now be empowered beyond the normal job stuff that you often read. What you have above is the output of my humble experience in multiple multiple roles working as a Practitioner, Author, Evangelist, People Leader.

There is one job / role / career choice I have not covered here. Becoming a Consultant. Perhaps another day. For now let me share just this advice, the most common reason for Web Analytics Consultants failing (or not succeeding as much as they should) is that they believe technical skills are enough or just being good at business (analysis / understanding / savvy) is enough. It is not. It is hard to find both is one place as well. If you want to do that find a partner. You be the strongest Analyst on earth and let her be the Technical Goddess. Now you are set for greatness.

Before I end, I had promised something about being laid off in a down turn.

My Story.

I have had three professional jobs in the US. I have resigned from only one, the last one. The first two jobs (4.5 yrs and 2 yrs respectively) I was laid off.

The first in worst of personal financial times (market going down, first baby on the way etc). It was deeply stressful.

In hindsight though each layoff was the best thing that could have happened. Allowed me to start fresh. Each bumped my career trajectory in ways that would never have happened if I had stayed at the job.

I hope your job is secure. But if it is not I hope you take some inspiration in my humble experience of being laid off twice from good jobs (that I was good at) and the longer term results.

Ok your turn now.

What has worked in managing your own web analytics career? Anything above in my Web Analytics Career Introspection Guide resonates with you? What about the four job families? What did I not consider or get wrong?

Please share your stories, feedback and encouragement.

I appreciate your attention, thank you.

PS:
Couple other related posts you might find interesting:

• Nine Rules To Work / Live By
• Three “spire’s” of Great Leadership
• Multiplicity: Succeed Awesomely At Web Analytics 2.0!
• How To Excite People About Web Analytics: Five Tips.
• Web Analysis: In-house or Out-sourced or Something Else?
• Make a Great Vendor / Agency / Consulting Pitch - Win Big Contracts

Copyright © : 2006-2008 Occam's Razor by Avinash Kaushik | All Rights Reserved

Analytics Career Advice:”I am an Analytics God, I want more $$. How?”

An Analysis Ninja, let’s call him Philip Walford, asked a delightful question. Philip wanted to know if the impact of a faith based initiative in his company, product demo videos, could actually be measured using data.

Hurray!

Faith is good. Data is better. : )

[And before you flame me: know that I love my religion more than you love yours. Wait. That did not come out right. Let me rephrase that.]

In this thanksgiving week 2008 post I’ll share Philip’s question about how to identify value of video product demos on an ecommerce site, and my answer about involving customers.

Here’s Philip. . . .

We are a large retailer with a lot of product on our site. In the past we have invested lots of dollars and time producing things like demo videos for our products, or adding other features and tools to our website to provide more information about a product. Our goal is to inspire customer confidence in their purchase (by giving them as much information is possible).

The question is, what are the KPIs of things like a demo video.

My recommendation was to measure conversion rate for the segment that views the video. If conversion is higher then the videos are bringing value. Others in my company have presented the hypothesis only customers that are a lot more invested in buying the product are likely to click on the video link and hence “pre qualified”, hence that segment would have had a higher conversion rate regardless.

I understand their perspective but I feel they are reading too much into the situation but I don’t know how to argue this point. There are several directions we could go with this but I wanted to see if you could share some guidance on this issue.

My answer to Philip. . . .

This is a complex problem, more than might be apparent on the surface.

It is also an example where it can be easy to jump into bed with your web analytics tool to get satisfaction but you wake up in the morning feeling. . . . well. . . . less than satisfied.

But before we go there I have to give a ton of credit to Philip and his crew for being skeptical of reading too much into their own opinions or biases.

I firmly believe that people who work for a company rarely (never!) represent customers. They are too close to the company and too different.

Just because I work for Microsoft and use a Zune (yes I do!) does not mean I can be a effective customer representative of Microsoft Zune customers. Company employee opinions rarely reflect those of customers. Do please be aware of that.

So when looking to make decisions, look for data (quant or qual).

I’ll present Philip with three solutions / options as he battles the challenge of figuring out if the investment of muchos dineros in creating product videos is worth it (besides the fact that these videos ooze sexiness!).

1) Use ClickTracks (Compute Contextual Influence)

There are two challenges with using clickstream data and the “typical” measure of conversion rate to determine success.

A] You might be looking at a “biased” segment (as challengers to Philip’s recommendation mentioned). I.E. Only the highly motivated people.

B] By comparing all people who converted and viewed the video with those that converted and did not see the video you are not comparing fair segments. You are also lumping all other “convince our visitors to buy” tools into one large bucket. Tools like Comparison Charts and Product Screenshots and Product information and Customer Reviews and more.

It is quite possible that those other tools might be getting people to convert at a much higher rate and by dumping them all together you are not being fair.

And of course you’ll get a wrong read on conversion impact of the videos.

So even if you use your web analytics tools (your Google Analytics or Omniture or WebTrends or CoreMetrics or whatever) try to compute “contextual influence” (value of each feature in context of the others).

It is actually very hard (damn near impossible) to do this in all those tools (even for the Paid solutions, even after you plunk down half a million dollars for the mandatory Data Warehouse “add on”).

ClickTracks is the only tool I know of that can do this out of the box, using its terribly named “funnel report”. No data warehouse. No extra tags or variables or sprops or wt_&*#$. In fact not even much IT, I just need admin access to my tool (not site, web analytics tool).

Its easy to use. Create a hierarchy of your website. Add individual or groups of pages into each stage (notice I did not say step because you can jump steps here). Add an outcome (in my case say “Thanks for placing your order” page). Click Calculate.

Boom!

[You are not supposed to be able to read the analysis, sorry, privacy dictates that.]

What I want you to note is two things.

This is a site where each stage means a view of the site (and like a traditional funnel how many people get in, get out, move on etc).

Secondly note that each box (which represents a page/’s or a tool - videos, comparisons, reviews etc) has a different stage of blue.

What this lovely report does for you is compute “the influence”of each of those pages/tools in driving the ultimate outcome - purchase here. The darker the blue the more “influential” that piece of content. [Influence is defined by the existence of that piece of content in the visitor session, regardless of what path the visitor took, regardless of when the content was seen.]

Ain’t that super sweet?

The analysis you see above is for a real ecommerce website. What it proved to us, delightfully, was that the product videos, we had created at a cost of over one hundred thousand dollars, yellow star above, was the least influential tool we had on our site.

The most influential, sexy pink star above, was a tool that had cost us $8 to produce - it was a page that compared different versions of the product (information that was handily available in the company).

We used actual customer behavior. We analyzed contextual segments. Ultimately it allowed us to put our precious few resources in the right area.

Of course it is quite likely that everyone who came to the site and did not buy (convert) might have loved the videos and rushed to stores to buy our products (one HiPPO actually said that!). There is no way to prove that using just the web analytics data.

What we did is proved impact on online buyers.

As to the HiPPO. . . . read on. . . .

2) Use Surveys (Actively Collect VOC)

When in doubt (or confronted by a HiPPO, remember don’t run) what better way to go then gather some Voice of Customer. Dare I say the voice of god? :”)

Two things I have tried (of many!) that work a lot of the times. Each covers one unique bucket of visitors to your website.

A] Consider sending a simple post purchase email survey to customers who have purchased on your site and ask them for the key influencers of their purchase.

You could share with them the various tools you have on your site (product information, comparison tools, images, videos, customer reviews etc etc) and simply ask them to rank order them in order of importance.

Don’t ask them to tell you how much they like them, or choose ones they like, they tend to pick all. :) Just ask them to rank order. Or use a tactic similar to that.

This tells you want works for those who buy.

For the 98% that will never convert on your website. . . .

B] Consider a onsite survey like 4Q (though 4Q can only be customized so much so perhaps you want to use either your own or one of the big daddy paid survey tools).

This will go to a small random sample of people who are on your site (who may or may not buy). You’ll ask them three or four questions about why they were there (primary purpose) and then what tools/features of your website they liked (rank ordered if at all your survey company can do that).

That will give you what you want.

Since this can also be thought of as a page level problem, you can also use something passive, a page level survey / poll, like Kampyle on your product pages and ask people to quickly rate the various features. There is a Site Content feedback topic in Kampyle which you can customize.

Now you have the most important piece of data you need, your customer’s. Few website owners / marketers / hippo’s can argue with this. Leverage this advantage.

Finally one last option for you. . . . hopefully one you’ll use before you write a chq for a hundred grand to create your videos. . . .

3) Use… wait for it….. Testing! (Measure Actual Customer Behavior)

I am sure this does not surprise you. Run a A/B or Multivariate Test and let your customers help inform you of the value of these features.

For 30% or 40% or whatever %, don’t show the product demo videos and for the rest show the product demo videos and see the impact on the data. Boom (!) you have your answer, without any biased opinions.

It is certainly going to take you a small amount of effort, get the Website Optimizer, talk to your IT folks, create version of the page with no product tour link etc.

But you are making a very expensive decision for your company are you not?

And here is the additional benefit of testing. You are free to use any kind of “conversion”.

You can measure success as conversions (submit order).

You can measure success (of the test) as number of people abandoning from the product page.

You can measure success as the time people spend on the product page. [There is a very cool javascript code that does this with the Google Website Optimizer, it is especially helpful for rich media / flash sites. Without a doubt other vendors can do this as well, just ask.]

You can measure success through your survey tool if it is integrated (this is some extra work sadly, but for big bets I recommend it).

You can integrate your analytics tool with your testing tool (say Google Analytics with Website Optimizer) and use other metrics to measure success such as bounce rate or electric shocks etc :).

[For GA and GWO ROI has integration instructions .]

The bottomline is that you can define success and then let the customers tell you.

That’s my answer to Philip.

Sounds exciting?

Am I the only one who thinks when you do this kind of analysis you are in a nearly orgasmic state?

Yes these methods are some small amount of work. But nothing in life worth having is easy. The tools might be free, but that does not eliminate your need to investing your time and effort! :)

And on the positive side with a recession looming people who involve customers in making decisions, rather than their opinions, will win big. The “guessers” will not win big. They might even win small. Or fail.

Plus if you do this you’ll be a Analysis Ninja, not a Reporting Squirrel.

Ok now your turn.

Have you tried to analyze the features like Video Demo’s on your website? Or perhaps other complex features you have launched? What works for you? What totally failed? In my recommendation to Philip, what did I overlook?

Please share your feedback, critique and hurray’s.

Copyright © : 2006-2008 Occam's Razor by Avinash Kaushik | All Rights Reserved

Excellent Analytics Tip #14: Measuring Value of Ecommerce Sales Tools

“Experiment or die, there is no try.”

That was my call to action, Yoda inspired, last week to a group of international C-level executives. And I meant every word of it.

There is a tendency to think experimentation and testing is optional. Ouch!

I fundamentally believe that is wrong. For a few simple reasons:

# 1 It’s Not Expensive!

You can start for free with a superb tool: Google’s Website Optimizer. It is packed with enough features that I have no qualms flogging it (even though I work closely with the team!).

If you want to help our economy and pay for your tools then that is absolutely fabulous. Both Offermatica and Optimost are pretty nice options.

[Just don't fall for their bashing of all other vendors or their silly claims, false, of "superiority" in terms of running 19 billion combinations of tests or the bonus feature of helping you into your underwear each morning.

You'll be lucky if you can come up with 5 combinations, and it is not that hard to put on your underwear.

Look for actionable uniqueness. For example I am quite fond of the fact that with Offermatica you can "trigger" tests based on behavior. That is nice, well worth paying for.]

# 2 Six And A Half Minutes. That’s it!

Tom has tried this with many many Marketers, and its so true: If you have two different pages you want to test, it takes six and a half minutes for you to configure, test (QA) and launch a A/B test.
[Please read that literally, as it is written. You have two pages already. 6.5 mis to: Configure. QA. Launch.]

You have six and half minutes right?

I cannot recommend enough the wisdom of starting with a A/B test.

You will start fast, you will find enough problems in your company, you can show easy wins.

Aim to get to the thing vendors are selling, MVT, but start with A/B regardless of the tool you use.

# 3 Show ‘em You Are Worth It.

There is a lot of pressure on all of us to prove our worth and make significant improvements to our web business.

ClickStream analysis with Omniture or Google Analytics or ClickTracks is well and good, testing will get you on the path of taking having a direct impact faster.

By the nature of it Testing is action oriented, and what better way to show the HiPPO’s that you are awesome then by moving the dial on that conversion rate in two weeks?

# 4 Big Bets, Low Risks, Happy Customers.

Very few people appreciate this unique feature of testing: You have an ability to take “controlled risks”.

Let’s say you want to replace your home page with pictures of naked people, yes in the quest of engagement . : ) Naked people are risky, even if they are holding strategically placed Buy Now buttons.

So run a test where only 10% of the site traffic sees version B (naked people).

You have just launched something risky, yet you have controlled the risk by reducing exposure of the risky idea.

Stress this idea to your bosses, the fact that testing does not mean destroying the business by trying different ideas. You can control the risk you want to take.

# 5 Tags, CMS, Reports & Regressions: All Included!

Pretty much all Testing tools are self contained, simple to launch (A/B is brain dead easy, MultiVariate needs your brain to be awake - that’s not hard is it?), they contain all reporting built in and the data is not that hard to understand.

So you don’t have to worry about integrations with analytics tools, you don’t have to worry about rushing to get a PhD in Statistics to interpret results and what not.

You will hear super lame arguments about mathematical purity or my factorial is better and the other guy’s whatever. Ignore. It will take you a while to hit those kinds of limits. And the nice thing is by then you’ll be smart enough to make up your own mind.

What’s important is you start. Do that today. This of this as dating and not a marriage. You are allowed to make mistakes. You are not going to marry the first guy you run into. Don’t take that approach here.

So agree with me? This is attractive? Right?

Think about it this way. If your analytics career is flagging then testing is the Viagra you need to take.

Seriously.

: )

So as my tiny gift for you here are five experimentation and testing ideas for you. I’ll try to go beyond the normal stuff you hear at other sources.

# 1 Fix The Biggest Loser, Landing Page. (& Be Bold.)

Now all that is well and good. But the sad thing in a common mistake people make is get excited and then go try to test Add To Cart buttons. Or three different hero images on the home page.

That’s all well and good. But honestly that’s not going to rock your boat. [Remember you are on Viagra!]

For your first test be bold, try something radical, bet big. I know that sounds crazy. But remember you can control risk.

If you start with a A/B test with some substantial difference then you can show value of testing faster because you’ll get a signal faster, you’ll start the emotional change required to embrace testing across the organization.

My favorite place to start, is the Top Landing Pages report (or Top Entry Pages if that’s what your vendor calls it) from your web analytics tool.

Find the biggest loser, the one with the highest bounce rate.

Click and look at the sites sending traffic to this page, look at the keywords driving traffic to this page. That will give you clues about customer intent (where people come from, and why).

Come up with two different (bold) ways to represent that page and deliver on that customer intent.

Your first A / B / C test.

# 2 Test a Single Page vs. Multi Page Checkout.

One of the highest ways to improve conversion is to reduce Cart & Checkout Abandonment rates. Take money from people who want to give you money!

Some websites have a one page checkout process: Shipping, billing, review and submit.

Some have it on four pages.

I have seen both work, you never know, it really depends on the types of visitors you attract.

So if you have a single page why not try the multi (if your abandonment rate is high, say more than 20% :). Or vise a versa?

I have seen very solid improvements in these tests.

Or here’s a bonus. Many shopping cart (or basket to my British friends) pages have a Apply Coupon Code box. This seems to case people to open Google and search for codes. So why not move this coupon code box to the Review Before Submit page?

It won’t send those who don’t have a coupon code looking for one, and by the Review Order page they are way too committed. For those that have a coupon code they can still apply it.

In both these scenarios you are helping your organization find value quickly by touching a high impact area.

And remember, this works for lead submission forms and other such delights.

# 3 Optimize the Number of Ads & Layout of Ads.

Ad supported sites are numerous. And the there is so little restraint, the core idea seems to be let’s slap as many ads on the site as we can.

More ads = more clicks = more revenue.

Usually this is never tested.

[I can't read Norwegian so this could be wrong, but I counted a total of 19 ads on this page! Ten above the fold. Important point: American sites are just the same.]

So test the number of ads you should have on a page. Its not that hard. It can be a simple A/B test or a MultiVariate test.

In a memorable test the client actually reduced the number of ads on the page by 25% and the outcomes improved by 40%. I kid you not, 40%. And guess in which version customers were happier.

There is a built in assumption there that you are simply not selling impression, in which case pile on the ads in the pages. You are not being held accountable for outcomes so enjoy the ad party.

Here’s a bonus idea.

There are sites were the ad is in the header, it takes up the whole header and is the first thing that loads. I have only seen one case where that worked.

The header takes up 30% of the space above the fold on a 1024 resolution.

So if that is you why not try a test with the header ad and without? See which one improves overall conversion / outcomes?

The other bonus idea is to try different ad layouts. Most people have banner blindness, top of the page and in the middle of the content (as in Yahoo news).

Why not try different layouts and formats? If not to see which one works the best then to just annoy your customers? :)

# 4 Test Different Prices / Selling Tactics.

You can of course test different pretty images, why not try to reinvent your business model using testing?

A company was selling just four products. But the environment got tough, the competitors got competitive. How to fight back? Some “genius” in the company had an idea “Why don’t we give our cheapest product, currently $15, away for free?”

CMO says: Radical idea. CEO says: Are you insane? CFO says: No way!

Now it did present a fundamental challenge, no one like to give revenue up. And people worried about how successful it would be, what would be the revenue impact, why would anyone buy a non-free version etc etc.

Rather than create prediction models (with faulty assumptions!) or giving up in face of the HiPPO pressure, the Analytics team just launched a A/B test. And they controlled for risk (after all the CFO did not want to go bankrupt) by doing a 95% control and 5% version A test.

Perhaps unsurprisingly the free version of the product sold lots of copies.

That was not surprising.

What was surprising was that free helped shift the sku mix in a statistically significant way, i.e the presence of free caused more people to buy the more expensive options. Interesting. [In a delightfully revenue impacting way!]

The other positive side effect was to cause lots of new customers to be introduced to the franchise, as they “purchased” the free version. Lovely.

Here are some bonus ideas.

If you give discounts try 15% off vs $10 off (people tend go for the latter! :)).

Try $25 mail in rebate vs $7 instant rebate (or change amounts to suit).

You get the idea.

# 5 Test Box Layouts, DVD Covers, Offline Stuff.

Let’s say you are launching a new product or a dvd or something similar. You want to figure out what layout might be more appealing to people in stores.

You could ask your mom to pick a version she likes.

You could ask your agency to ask a few people.

Or you could launch a test online and see which version is rated highest by your website visitors!

I have done tests for DVD covers and the results were surprising.

Or here’s another idea…

You are a multi channel customer. You sell bikinis. Now you want to sell Accounting Software. Why not try it on your website before you reconfigure your stores?

Or you are Wal-Mart and it is expensive and takes a long time for you to put new products in your stores. That makes it risky to start stocking the “on paper hideous but perhaps weirdly appealing” Zebra Print Occasional Chairs in your store. What if it bombs?

Well why not add it to your site, see if it sells. If it gets 15 positive customer reviews (!!), then you know you have a winner on your hands.

The actual launch process is faster, you can reduce risk, and you don’t have to rely on just your company employees (the fashion mavens) from picking winners and losers.

All done.

I hope that you’ll find both compelling reasons for starting experimentation and I have managed to stretch your mind beyond “honey let’s start testing shopping cart buttons”.

There is so much you can do. This recession season buy your CEO the gift that keeps giving, a experimentation and testing tool.

Here’s a summary for you. . . .

Five reasons for online Experimentation & Testing:

#1 It’s Not Expensive!
#2 Six And A Half Minutes. That’s it!
#3 Show ‘em You Are Worth It.
#4 Big Bets, Low Risks, Happy Customers.
#5 Tags, CMS, Reports & Regressions: All Included!

Five off the beaten track Experimentation & Testing ideas:

#1 Fix The Biggest Loser, Landing Page. (& Be Bold.)
#2 Test a Single Page vs. Multi Page Checkout.
#3 Optimize the Number of Ads & Layout of Ads.
#4 Test Different Prices / Selling Tactics.
#5 Test Box Layouts, DVD Covers, Offline Stuff.

Ok now its your turn.

What are the reasons your company is not jumping on the awesome testing bandwagon? If it did, what finally convinced them? If you are doing testing, care to share some of your ideas? Anything off the beaten path you have tried? Any massive failures?

Please share your feedback, insights and stories.

Thank you.

PS:
Couple other related posts you might find interesting:

• Experimentation and Testing: A Primer
• Build A Great Web Experimentation & Testing Program

Copyright © : 2006-2008 Occam's Razor by Avinash Kaushik | All Rights Reserved

Experiment or Die. Five Reasons And Awesome Testing Ideas.

Ideally you should only have one web analytics tool on your website.

If you have nothing and you are starting out then sure have a few different ones, stress test them, pick the one you love (just like in real life!), but then practice monogamy.

At the heart of that recommendation is a painful lesson I have learned: It is a long hard slog to convert an organization to be truly data driven.

And that’s with one tool.

Having two tools just complicates life in many subtle and sub optimal ways. One “switch” commonly occurs is the shift from fighting the good fight of getting the organization to use data to bickering about data not matching, having to do multiple set of coding for campaigns (the page tagging work itself is trivial) and so on and so forth.

In a nutshell the efforts become all about data and not the quest for insights.

So if you can help it, have one tool. Bigamy atleast in this case is undesirable. [If that does not convince you remember the magnificent 10/90 rule from May 2006 when I was but a naïve web analytics Manager.]

But.

Pontification aside the reality is that many people run more than one tool on their website (though hopefully they are all on their way to picking the best of the lot). That means the bane of every Analyst’s existence: Data reconciliation!

It is a thankless task, takes way more time then needed and the “game” is so rigged that 1] it is nearly impossible to get to a conclusion and 2] it is rarely rewarding - i.e. worth it.

But reconcile we must. So in this post I want to share my personal checklist of things I look for when going through a data reconciliation exercise. Usually this helps get things to within 95% and then I give up. It is so totally not worth it to get the rest!

* This post is a bit technical, but a Marketer should be able to understand it. And my goal is you smile three times while you read it, either from the inside jokes or from the sheer pain on display! *

So if you are starting a data reconciliation project for your web analytics tools, make sure you check for these things:

#1: Comparing Web Logs vs. JavaScript driven tools. Don’t.

#2: First & Third Party Cookies. The gift that keeps giving!

#3: Imprecise website tagging.

#4: Torture Your Vendor: Check Definitions of Key Metrics.

#5: Sessionization. One Tough Nut.

#6: URL Parameter Configuration. The Permanent