Contextualized Search

I’ve previously written about the merits of attributing value to the context of finding information, rather than on any particular piece of information. This makes sense in an environment which literally explodes with new information, and shows no signs it’s gonna stop in any foreseeable future.

Google seems to think so too. After all, this is what Google do, and do really well. But it’s true no less of a somewhat overlooked product of Google’s. I’m talking about Google’s Custom Search. This service allows anyone to composit their own search engine, and place it on their own website. More accurately, your custom search engine filters Google’s index of webpages. Say you want a search engine on your site about your niche subject only to return results which relates to your site. It’s simple : type in your site name, and allow Google to show results from your site as well as all the sites your site links to. Or you can be even more specific, or list a range of sites you want results to be taken from. Or you’d like Google to still show results from the web, but emphasize results from your own site – this is also easily doable.

The only problem so far with Google’s Custom Search has been on the one hand that Google’s crawlers don’t seem to index every website too tightly and too frequently, and on the other, that results are still based on PageRank. Say you want your users to find a great piece on your blog about a particular subject, when they search for that subject, but that piece isn’t greatly linked to by other sites or articles. Chances are, that Custom Search will show a largely irrelevant, but greatly linked to article from another site, or simply not show that post at all, if it hasn’t been properly indexed. Your built-in blog search, such as WordPress’ search, will find that article very fast, because it searches your database directly. For smaller sites, local search as we know it, is still much more effective.

However, as sites grow and we as internet users and bloggers spread our activities over many sites and platforms, platform-specific search is too limited. We begin to look for more tailormade solutions. Google’s Custom Search is one, but there are others who want a piece of the action.

New kid on the block

Lijit is an internet startup based in Boulder, Colorado, which offers a promising version of “local” or “contextualized search”, which searches one’s blog, “content” (on sites such as YouTube, Flickr and many others) and the network of sites and “friends” your online activities connect you to. We’ve already created a Kaplak search engine powered by Lijit, and the Lijit widget is featured in the outer right column on this blog. I think Lijit could potentially be a very useful addition to the Kaplak toolbox. I plan to expand this search engine with further feeds and sites as our network and activities grow.

When I first tried Lijit, I wasn’t satisfied with the search results. I searched for a direct title in one of our blog posts, and it didn’t come up. As the impatient web customer I am, not hesitant to make a fuss about my problems with a free online service – on another free online service, I posted my quibbles on Twitter. It turns out, Lijit is on Twitter too, and so is Micah Baldwin, who works for Lijit and took time out to answer my quibbles.

It turned out Lijit based their first version on Google’s Custom Search, while developing their own web crawler. Switching Kaplak’s search to Lijit’s own crawler was a huge improvement from Google’s occasional crawl, and made me look much more enthusiastically at what this small team of extremely talented people are doing. I take my hat off for a company which acts so swiftly in response to “customer” sentiments, and make it a priority to help their users along with such friendliness. There are a lot of companies who could learn so much from Lijit. Micah and Lijit gives the expression “listening to the groundswell” a whole new meaning.

I like the freshness of Lijit and I like the results after being switched to their own crawler. I have only a few quibbles with it now. It’s got what I’d call some weaknesses in the versatility department, because I can’t control and finetune texts, messages and included sites/webpages as much as I’d like to and was quickly getting accustomed to in my short period of experience using Google’s Custom Search. For instance, I found all of my del.icio.us network automatically included in the search engine, where I’d like the opportunity to handpick whose links got to be included. Lijit’s search engine also wants to categorize results very neatly into “my blog” (even though the Kaplak Blog is not precisely “mine” – it’s the company blog and maintained by me, but not “mine”), “my content” and “my network”. What if we (which we’re probably going to) put the widget on our wiki? – that’s not exactly “mine” either. Our Kaplak universe is not so neatly organized, and while I do like the “Lijit picks” category, I prefer being able to scrap all categorization schemes altogether, get our own adsense stuff on the search results and just get on with finetuning and putting in more sites and feeds to give our visitors the best possible experience.

Lijit can potentially be a great key to tying together the many different platforms we operate on in Kaplak – and one we’d even pay for, if they included premium options we needed. As a company, we still do need search, and if Lijit could potentially even crawl user and product profile pages on our later-upcoming Kaplak Marketplace, we’d have something here, which we’d probably like to pay good human money for.

Conversational search

You can find most of my conversation with Micah via Summize, an online service which has built a search engine on top of Twitter, searching conversations on Twitter in realtime.

Imagine a service which have taken upon itself the daunting task of searching all things on Twitter instantly and is capable of threading and translating posts to and from numerous languages – globally. Then you have Summize.

Using Twitter a lot these last few months, I’ve found Summize indispensible to keep track of tweets, users and subjects. I’ve also used it for market research, i.e. “listening” to what other users are twittering. I find this stuff utterly incredible. There’s a lot of things happening in the search business these days.

I’m sure this is only the beginning.

[EDIT : Twitter's acquisition of Summize has broken the above link to the Summize search with my conversation with Micah. Here's a similar search on the new http://search.twitter.com which supposedly replaces Summize...]

flattr this!

Tags : , , , , , , , , , ,  

Timeline


7 comments ↓

#1 MortenBlaabjerg on 06.30.08 at 10:14 pm

Also got the opportunity now to test our new Skribit widget. Not sure if I quite answered how custom search relates to Kaplak, if only vaguely.

Kaplak won't rely on search for people to find niche products they want. We'll rely almost completely on the circulation of our widgets and on social recommendations, which are rewarded with Kaplak when they lead to sales. However, as I said in the post, we do rely on other tools in tying together our different platforms. Such tools are hyperlinks, RSS feeds – and search. Contextualized search put to work could do good things for our online multiplatform strategy, which is introduced elsewhere :-)

#2 MortenBlaabjerg on 07.03.08 at 11:53 am

Moved the Lijit widget nearer the top right of the page, which will give me the opportunity to use it a lot more :-) Looking forward to putting it to a further test when we open up the wiki. The latest tweets were also moved to the left column, and the Google Ads pushed down a bit. For some reason, our Amazon widget have ceased working – hopefully we can fix this ASAP.

#3 micah on 07.03.08 at 5:41 pm

Morten, Thanks for the interesting post. I would like to continue to work with you to make some of the changes you suggest, and perhaps figure out which ones would work on a more global scale. The truth is that the widget was originally designed for blogs and the inherent trust relationships blogrolls and other social links included. We should explore Wiki's given that they often have similar trust relationships to blogs.

#4 MortenBlaabjerg on 07.03.08 at 8:11 pm

Micah, I think you have something really great on your hands, and I am only excited if I/we can help in any way. Just let me know, if you need anything I am capable of giving back.

As ressources are always sparse, I can see the business disposition in focusing on one type of market, and then, when you've mastered that market, move on to other markets. You've definitely got a good point focusing on blogs, where Lijit works very well.

But you don't really need to change that much in your product or user interface, in order to make Lijit just seem a bit more versatile to people like me (I know you also have lots of infrastructure and scalability concerns behind the scenes, probably). Just make it possible to edit/localize the “labels” in search results. Say, “my blog” could just as well be “Our products”, and “my network” could be “Other products”. It's all hyperlinks and webpages anyway, so that is all needed IMO – but it creates whole new markets and types of use, and the best thing is, you don't have to work out the categories for different users, they can do that themselves.

Just as Disqus has something great to do, if offered to companies and product pages, I think Lijit can do wonders for linking together the different ressources of companies, and there you may also have an opportunity to monetize (dislike this word – isn't there another word for it?) better. But business people as most commonly bred are conservative to all new things. Bloggers are a more receptible breed, so I do believe you've done great by targeting bloggers first.

#5 MortenBlaabjerg on 07.17.08 at 11:59 pm

If you didn't know, Twitter has now acquired Summize : http://blog.twitter.com/2008/07/finding-perfect

EDIT : Apparently, this has also broken the link to the search on Summize for my exchanges with Micah. Will try and fix the link.

I somewhat wish the Twitter guys would just concentrate on getting their own service right, instead of buying a perfectly well-functioning service and begin to ruin that too. If the Summize team is now to be working on Twitter, there'll be less time to focus on the search side of Twitter (i.e. Summize), which is too bad, because it was so promising.

#6 Kaplak Stream | Everybody is an Aggregator on 09.09.08 at 11:14 pm

[...] a power otherwise vested in the large search engine companies. I’ve previously discussed the merits of tools such as Lijit and Google Custom Search. Automated RSS posting is even more promising, as it can support almost any segment of interest. [...]

#7 Everybody is an Aggregator — Captain's log on 11.25.11 at 11:56 pm

[...] a power otherwise vested in the large search engine companies. I’ve previously discussed the merits of tools such as Lijit and Google Custom Search. Automated RSS posting is even more promising, as it can support almost any segment of interest. [...]

Leave a Comment