Social Search Is Google’s Killer Feature
by Hessam on January 28, 2010
This is something which I’m really excited about and I believe is a real killer feature for Google Search. Unfortunately with the frantic hype around Apple’s announcement of the iPad, this news has not yet got the attention it really deserves. Back in October 2009 Google announced Social Search as an experiment on Google Labs. Users could then opt-in and test the feature which enabled them to find more relevant results from their broader social circle. The result is relevant information recommended by your contacts which is bound to be better than from strangers, and which results in a more personalized search experience. Yesterday Google rolled this feature out of experimental and into beta which means it’s now available for everyone.
The principles of the Social Search feature are very simple. As a user I can create a Google Profile and link it with my various public accounts on for instance Twitter, YouTube, and Flickr. Google then uses this information, plus other clues such as my Gmail contacts or blogs in my Google Reader, to generate a social graph of my direct connections, friends of friends and so on. Next time I perform a search, the search algorithms look through the public content published in my social graph to see if there is something relevant available.
Here is an example of how it can look like: when I for instance search for [301 redirect], I get two very relevant results from my social circle blended with other results. One is an excellent blogpost from my friend Jesper Åström about how to implement a .htaccess 301 redirect, and a second blogpost from my former colleague at Google John Muller about 301 redirection issues on Twitter.
So how did Google found these two results? That’s simply because I have linked my Twitter profile from my public Google Profile. Google’s algorithms can then see that I’m following both of these guys on Twitter, and that both have a link to their blogs in their Twitter profiles. The type of content which might pop up in Social Search are for instance blogposts, tweets and status updates on other microblogging platforms, reviews, public Picasa or Flickr images, and other activity from your friends on places which are linked to via their profiles.
To start using Google Social Search, you need to first create your own Google Profile and add links to your public profile on Twitter, FriendFeed, YouTube, LinkedIn, Flickr, etc. Google’s algorithms will then use these links to outline your relationship graph and to identify relevant content from your online neighbourhood next time you make a search.
If you are already using this feature, head over to this page to get an overview of your current social circle as Google sees it. More details about Social Search here.
(Photo credit: laurelpapworth.com, personalizemedia.com)
Digg.com’s SEO Mistake And The 1 Second Fix
by Hessam on January 21, 2010
Update February 4: Following this post and my email conversations with the folks at Digg, they corrected their robots.txt file on January 29.
I was recently looking through some of the major tech news sites around the web and studying their usage of the robots.txt protocol. I came across Digg.com and found a potential error in their robots file which is costing them a lot of lost search engine traffic, as well as revenue from advertising. Since it looks like an honest mistake, I thought I write about it so that you won’t make the same mistake. For those of you not familiar with robots.txt, it is a way for webmasters to give instructions to search engine crawlers (also called robots and spiders) about what they are allowed to crawl, and which areas of the website should not be accessed. You can read more about the robots.txt protocol here.
To come back to Digg.com, I found a mistake among the instructions which causes a large part of the website (the Gadgets directory) to become inaccessible to search engines. If you take a look at their robots.txt file, you can find the following instruction:
Disallow: /gadget
If you are a SEO geek (like me) you have probably noticed the open ending to this line, and hence the wild card effect it causes in practice. What this line basically says is to block access to all content which have ‘digg.com/gadget’ as part of their URL. Note that this is not the same as exclusively blocking the “gadget” directory, but is far more inclusive. In practice, this instruction also covers Digg’s Gadgets directory, ‘digg.com/gadgets‘ (note the ending ‘s’), and hence the problem! The correct implementation includes a forward slash ending the line in order to avoid the wild card effect:
Disallow: /gadget/
Digg has a large amount of pages in its index, for instance a Google site search shows close to 290 thousands pages in the ‘digg.com/apple‘ directory alone. We can therefore only speculate exactly how much search engine traffic Digg.com is shutting out by accidentally blocking the /gadgets/ directory (Yahoo Site Explorer shows 108 thousands pages in /gadgets/). This could be done intentionally, but it seems more likely to be an honest mistake (it’s one sinlge “/” missing after all). According to Compete.com, Digg has around 40mil monthly unique visitors, so I can only speculate around how much money such a small mistake has cost them in advertising revenues during this period.
Finally, the lessons we learn from this are to:
1) generate easy to read robots.txt files so it’s easier to spot such mistakes. There are online tools which help constructing and deciphering robots.txt files.
2) always use web analytics tools and segment your traffic so it’s clear to you what is going on across the various parts of your site.
3) be afraid of the wildcard effect, in fact just completely avoid anything with the word “wild” in it :)
What are other important things to think about when creating a robots.txt file?
Google Stops Censoring in China
by Hessam on January 13, 2010
In a blogpost on Google’s official blog tonight the company is delivering a direct attack on the Chinese government and states that Google has had enough and will effectively stop censuring its search results in China. The firm lists three points for taking this momentous decision:
1- Google and a number of other organizations have lately been a target for a sophisticated cyber attack on its corporate infrastructure originating in China
2- Evidence that the attackers’ main object have been to gain access to Gmail accounts of human rights activists inside China
3- Evidence that Gmail accounts of a large number of human rights activists in U.S., Europe and China has been regularly accessed by third parties. This has been achieved through using illegitimate methods such as phishing attacks and malware installed on user’s computer.
Google declares:
“We launched Google.cn in January 2006 in the belief that the benefits of increased access to information for people in China and a more open Internet outweighed our discomfort in agreeing to censor some results. At the time we made clear that “we will carefully monitor conditions in China, including new laws and other restrictions on our services. If we determine that we are unable to achieve the objectives outlined we will not hesitate to reconsider our approach to China.”
These attacks and the surveillance they have uncovered–combined with the attempts over the past year to further limit free speech on the web–have led us to conclude that we should review the feasibility of our business operations in China. We have decided we are no longer willing to continue censoring our results on Google.cn, and so over the next few weeks we will be discussing with the Chinese government the basis on which we could operate an unfiltered search engine within the law, if at all. We recognize that this may well mean having to shut down Google.cn, and potentially our offices in China.“
Now, stop for a second and think about which other public company would pull out of the most lucrative market in the world, for a good cause? To me, it looks like Google continues to live up to its “Do No Evil” mantra. I really hope more companies will follow suit.
The entire post can be found here.
The Swell Season – Strict Joy
by Hessam on January 11, 2010
How could I miss this!? I blame it on becoming a father and not having enough time to keep up with all the fantastic albums being released :)
Anyways, if you don’t know The Swell Season then you really should check them out! The duo consists of Glen Hansard, singer of the Irish rock band The Frames, and Markéta Irglová who is an actress and musician. The couple became internationally famous for their roles in the fantastic movie Once and subsequently won an Oscar for best original song.
Their new album Strict Joy was released back in October, and except for having a wicked album cover, it sounds really good. Here is a mix with two tracks from the new album, plus a collection of previous stuff they have done. I hope you will enjoy it.
Watch a trailer for the new album here.
Listen to Strict Joy on 
Does the German Government Hate Google?
by Hessam on January 9, 2010
Not a week goes by without new allegations and threats against Google from German officials. Recently there were reports of federal and state officials debating to make the usage of Google Analytics illegal because of fears that Google might create detailed user profiles based on users’ interests. This would consequently impose fines on websites using this tool to collect and analyze anonymous usage data from their visitors. In other bizarre news, local officials in a small city have voted for a plan to charge Google per kilometer for Street View footage.
The latest assault comes from Germany’s minister of justice Sabine Leutheusser-Schnarrenberger (FDP) who in the Spiegel magazine (poor translation here) threatens with legislative actions unless Google is rethinking its what she calls “gigantomania”. She criticizes Google for its general arrogance against user privacy, lack of transparency, and megalomania. Just as in any Google bashing article, she also mentions Google Book Search which has recently been harshly criticized by Angela Merkel’s government. In a speech before the opening of the last Frankfurt Book Fair Merkel said her government opposes Google’s endeavour to create an online library due to the “considerable dangers” for copyright protection online.
Germany has a dark history of breeches in citizens’ privacy and still to this day there are new revelations about the GDR’s repulsive treatment of its own citizens. Thus, this suspicious view of Google the Giant might not come as a surprise. One can only hope though that there are enough politicians and advisors in the current government who understand and can explain new technology to their older peers. The way I see it, Google with its huge worldwide reach and innovative products has undeservedly become a convenient target for people’s legitimate fears to lose their privacy in the vast world wide web.
To answer the question if the German government dislikes Google, my answer is clearly No! Instead, I believe the lawmakers’ attacks are caused by pressure from understandably concerned citizens, but even worse, from slow moving industries, mainly in publishing and news, who have held on to their old ways and refused to adapt to the digital era. They are now failing to deliver on their outdated business models and governments are sadly once again intervening to save failing industries to the detriment of the users.
My name is Hessam Lavi, I'm a SEO & Web Analytics consultant and a former member of Google's search quality/webspam team. I'm an Iranian-born Swedish national living in Berlin. 
