ClickAider
You are currently browsing the Bogle’s Blog weblog archives for the day Thursday, November 10th, 2005.

minisearch: search all sites linked to from a URL

I have been “playing”:http://www.thebogles.com/blog/2005/05/new-utility-add-your-editorial-voice/ with the idea of making search more targeted, by allowing users to limit their search to a set of sites. “Rollyo”:http://www.rollyo.com (”Roll you own search engine”) has made a full fledged web destination out of this idea, with Ajax, tagging, and other web 2.0 buzz compliant features.

However, as “Dan Gottlieb points out”:http://www.dan-gottlieb.com/blog/?p=5 points out, Rollyo’s biggest weakness is that it’s an island. There is a huge volume of captured expertise out there in the form of del.ico.us tags, blog rolls, wikis, and specialized web pages, and Rollyo taps into none of this.

Here’s my problem with the site – it’s an island. URLs are a very core part of my daily workflow. I bookmark interesting sites in del.icio.us and tag them by topic. I reference sets of del.icio.us sites tagged by subject matter experts. Together with co-workers I maintain assorted sets of useful links on our workgroup’s wiki. I have a blog roll on my blog loaded with links to people whose views I respect (or at least I had one until my blog blew up). To use all of that knowledge in Rollyo, I’d have to re-key it, and then maintain that same set of information in multiple places. Rollyo also wants me to adopt yet another tagging scheme, which I’d then have to keep somewhat in sync with my del.icio.us, flickr, and backpack tags so that my brain doesn’t explode.

Dan has created a lightweight approach to creating customized search engines that does a much better job of leveraging existing resources and communities.


Thinking about it, I realized that what I really want is the ability to search an arbitrary set of sites linked to from a single web page. Severe insomnia and some Python brought this to life and I’m pretty happy with the result.

“Here”:http://dan-gottlieb.com/projects/minisearch/search.fcgi?page=http%3A//danslinks.pbwiki.com/DigitalCameras&title=&mode=site for instance is a search form that Dan created quickly by pointing at the links already created in a digital camera wiki.

Dan’s closing points are interesting.


miniSearch sits squarely at the sweet spot of my view of Web 2.0 – a small component with an optional GUI that sucks in existing datafrom multiple sources, does some simple processing, and then spits data back out.

It seems like one potential issue is that typical pages have lots of different links of different categories, and that you might want to search only some subset of those links. Perhaps there should be a lightweight way to group or categorize links on the page– this would help in applications like mobile content reformatters which need to collapse groups of links to reduce real estate requirements.