The launch of Google Instant has caused a lot of excitement and SEOs around the world are wondering how this will affect their clients.
Google’s initial attempts to develop an instant search product were met with criticism from the SEO community as the referrer string was not being passed which would have pretty much ruined web analytics as we know it.
Fortunately they have solved the problem this time by making the Google URL update in the website brower address bar shortly after the search results are updated. There is a small delay before the URL appears to update but it always updates before the new page is loaded ensuring the correct referring URL is always passed.
As you type a search into Google, the results update to Google’s suggested search term which appears in grey. If you click on a result whilst a grey result is showing up then this suggested query is passed in the referring URL and this is what will show up in your keywords report in analytics.
In addition Google also include the original search query in another parameter called oq. If you check Google’s URL string you will see something like this.
aqi=&aql=&oq=seo&gs_rfai=&
Because this is passed in the referring URL, it’s possible to extract it and include it in a report using Google Analytics filters. Here is how you can track these original search queries alongside the suggested search queries with a few minutes work.
Edit: Dan Barker has pointed out that the oq parameter is still used by the old Google Suggest so this custom profile will show the results for both until the old version has been completely deprecated. It’s possible to update this to only include data from Google Instant by adding another filter to only include traffic which contains another unique parameter such as &sclient=psy.
1. On the Google Analytics Overview screen, click the Add new profile link.
2. Select ‘Add a Profile for an existing domain’ and then find the correct account from the drop down. Give your new profile a name like ‘Google Instant’ so it’s easy to find and click Finish. This will create a new profile that will start to capture all the raw data for your site.
3. On the Overview screen, select the Edit link for the new Google Instant profile we’ve just created.
4. Select the ‘Add Filter’ link on the right hand side of the ‘Filters Applied to Profile’ table.
5. Create a new filter by entering the following details.
Filter Type: Customer Filter > Advanced
Field A -> Extract A
Referral: [?|&]oq=([^&]*)
Field B -> Extract B
Campaign Term: (.*)
Output To -> Constructor
Campaign Term: $B1|$A1
Field A Required: Yes
Field B Required: Yes
Override Output Field: Yes
Case Sensitive: No
To briefly explain is happening here, we use a regular expression in Field A to extract the original search query from the oq= parameter that has been passed in the Referral string. We then use Field B to get the final search query from the Campaign Term variable. In the final step we output both search queries back to the Campaign Term separated by a | character. We will now be able to see this showing in the Keywords report.
Click the Save Changes button to create the filter.
After a few days you should have collected some data in your new profile and you can view the results.
Go to your keywords report and search for \| using the keyword filter.
Then you should be able to see results that look like this example.
You can see that some people hadn’t even gone beyond ‘stric’ before they clicked on suggested search for ‘strictly come dancing 2010′ which is the 2nd suggested search from Google.
Using these reports we can begin to understand the way people use the new search function and look for ways to optimise towards it.
The example above it has shown that Google Instant might helped to generate a higher volume of searches for more detailed search phrases than previously seen.
Perhaps the biggest impact we’re likely to see is a significant reduction in the number of unique phrases as people take advantage of the suggested results, effectively killing off the long tail of search. This might be a good thing for users as it may improve the quality of search but it will make SEO much harder as we all have to start fighting over few phrases which will drive more traffic.
This is probably going to be a win for the big players who already do well outside of the long tail and bad news for new sites trying to get a foothold.
on September 9, 2010 at 12:50 am
· Permalink
Awesome, already set it up!
on September 9, 2010 at 2:36 am
· Permalink
I can think of a lot of uses for this already. We need to figure out how to analyze the top suggested queries and target those. (easier said than done, and easier to target than to actually rank for…)
It would also be great to see a report that rolls up all of the short variations for the full keywords.
ie…
strictly come dancing 2010
– strictly
– stric
– strict
next keyword
– nex
– next key
and so on…
Something for me to work on…
on September 9, 2010 at 4:12 am
· Permalink
[...] Tips on Tracking Google Instant with Analytics [...]
on September 9, 2010 at 10:45 am
· Permalink
Thank you so much for posting such detailed instructions so quickly.
I will certainly be taking your advice and encouraging my clients to do so.
I think it will be extremely interesting to see the implications of this for the longer ‘brand-navigation’ searches I have seen in the past. (Site name or brand + ‘what visitor is looking for within the site’)
As ever: I’m going to need to get stuck into some real data before I can begin to understand the implications.
on September 9, 2010 at 11:03 am
· Permalink
hi there,
I think your filter also captures Google Suggest data. See this post from 2009 talking about the ‘oq’ parameter:
http://www.google.com/support/forum/p/Webmasters/thread?tid=43bcac607bade235&hl=en
I think perhaps ‘cp’ is the parameter you should filter on, which seems to capture the number of characters in the original search.
Hope that helps,
dan
on September 9, 2010 at 11:46 am
· Permalink
Just wondering – would it not make more sense to store the OQ parameter as a ‘user defined value’ rather than appending to the query? That would allow you to drill down into one ‘final’ query to see what the makeup of typed OQs are, and also it wouldnt split out each term into lots of different variants?
I’m guessing the way to tweak this to the above would be:
Field A -> Extract A
Referral: [?|&]oq=([^&]*)
Field B -> [leave blank]
Output To -> User-Defined Value
Campaign Term: $A1
Field A Required: Yes
Field B Required: No
Override Output Field: Yes
Case Sensitive: No
Correct? Would be interested in your thoughts…
on September 9, 2010 at 11:55 am
· Permalink
@ jaamit – That’s a great idea.
I’ve tested it and it works really well. The only problem is that the User Defined field is limited to 50,000 unique values per day but this shouldn’t be a problem for most people.
on September 9, 2010 at 12:41 pm
· Permalink
Thanks to Chris & Jammit for the tweaking.
We will certainly see changes in the future, regarding how Instant will (or will not) change the users searching behavior.
on September 9, 2010 at 12:42 pm
· Permalink
Chris: I think you should update this post. It doesn’t do what it says on the tin.
I’m seeing people applying this filter in & have had a couple of emails about it.
Those people will be trusting the results, which aren’t as advertised.
dan
ps. sorry, tried to email but I can’t find your address anywhere on the site.
on September 9, 2010 at 12:51 pm
· Permalink
@jaamit
How do you change:
Output To -> Constructor to
Output To ->User Defined Value
on September 9, 2010 at 1:35 pm
· Permalink
[...] Tracking Instant in Google Analytics. Related Information: Crafting an AdWords Account… [...]
on September 9, 2010 at 5:38 pm
· Permalink
This is great information. Thanks!
on September 9, 2010 at 6:49 pm
· Permalink
Ok, I love this…but I have a question. If within Google Analytics, it returns…
antique cars|antique c
…what would it mean if it returned…
antique cars|
…without anything after the | …
Thanks in advance.
on September 9, 2010 at 10:56 pm
· Permalink
This is great. Thanks so much for putting it together. Looking forward to watching this in the coming weeks and months.
on September 9, 2010 at 11:14 pm
· Permalink
I keep getting the “Tracking not installed” warning icon even though I added it to an existing domain. I obviously don’t need to install anything since the code is already on my site, correct?
Can anyone help me?
on September 10, 2010 at 8:22 am
· Permalink
Hey, I just wanted to make a quick comment about the search results you showed. Your data is showing that people found your site by typing in “stric”. I wonder if those are people who have been to your site already and have a temp file or a cookie in there.
When I type in “stric” it suggests to me “strictly soccer”, which I can’t stand soccer. Once it finally starts suggesting anything about dancing, its all news.
Anyways, was just curious if there is more of a “hyper local” flavor to these serps now. Local as in keeping track of everywhere people go on the web and customizing those results just for them.
Kinda sick if you ask me.
on September 10, 2010 at 8:58 am
· Permalink
@Wes
Wait a few hours.
New profiles (even using the same tracking code) usually take a few hours to start receiving data. Until then it says tracking not installed.
I wouldn’t worry too much.
on September 10, 2010 at 9:00 am
· Permalink
this is such a good method to extract the exact keywords. thanks for the article.
on September 10, 2010 at 9:01 am
· Permalink
“New profiles (even using the same tracking code) usually take a few hours to start receiving data. Until then it says tracking not installed.”
It is not particularly true, I got listed in 20 minutes.
on September 10, 2010 at 11:07 am
· Permalink
interesting
on September 10, 2010 at 2:25 pm
· Permalink
Thanks for this sharing…. This gives me more ideas on not only improving the ranking on how to improve your content.
Cheers!
on September 10, 2010 at 10:17 pm
· Permalink
@ Jason Green Have you tried http://www.suggestedsearchtool.com for analyzing the predictive/suggested searches? It de-localizes, too
It’s new (obviously), so any suggestions on making it better would be welcome!
on September 13, 2010 at 5:40 pm
· Permalink
Hey, Thanks for clarifying that this isn’t working.
I did sat it up, and was looking forward to the results. But no problem!
Does it actually do something, this report?
Does it track Google Suggest, like Dan Barker is suggesting?
Thx
Stefaan
on September 13, 2010 at 11:39 pm
· Permalink
So I just noticed that you updated the post and that this does not work as prescribed. Were you able to find any other resources that are tracking this?
on September 14, 2010 at 10:16 am
· Permalink
It looks like it won’t be possible to track Google Instant at all. Google have decided to omit any useful data from their redirected referring URL which was probably deliberate as their response to include more detail has fallen on deaf ears. It’s also contrary to Google’s suggestion that they support legitimate SEO activities
The filters described above will actually measure the Google Suggest function for people who have instant turned off so it still gives some insight into how people use the suggest function. Google Instant’s suggest has less suggestions and the fact you can see the results updating will probably mean the data is slightly different to Google suggest but it gives you a clue as to how people are using it.
Our analysis of the effect of Google Instant is showing very little change in overall traffic and the distribution of that traffic across keywords. We’re collecting more data and will be publishing the results of our findings in the next few weeks.