Google’s announcement that they are discontinuing support for their Search API has added urgency to the development of open search platforms. 

Alexa and Nutch approach the problem from different directions, each of them interesting. 

Alexa provides a hosted, scalable search service that you pay for; you can access their full-text index of their web, distributing computations across their cloud, and creating new indices to capture the results of those computations. Costs are modest. 

Our goal is to give unparalleled and unlimited access to search. Just think of it… where else can you:

• Take the reins of a Web crawler and direct it to crawl specific pages on specific domains and collect specific document types
• Mine the documents in the crawl and generate custom indices
• Reorder search results and create custom verticals
• Use your own advertising solution

This is by no means a complete list. I just put it together to illustrate a point.

Where other search engines may give you access to their search results, they will tie your hands. You won’t be able to access the raw documents in their crawl, create your own index, reorder the results or even use your own advertising solution. In some extreme cases they will only provide results if you give over part of your page to their ajax script. Why would these search giants create search solutions that are obviously limited and of little use to inventors? Because they are not interested in helping to create their next competitor.

Alexa on the other hand… that’s exactly what we are here to do. We are here to build a platform for you. We are designing our services to be consumed and manipulated by developers and inventors. We fully expect that the next great search engine will be unimaginable to us and won’t be based on a plain vanilla search index from one of the big boys. It will be built and based on a new idea and it will require the kind of access that only Alexa can provide.

You can get started in the new and revamped Developer’s Corner.

Nutch, on the hand, is a fully open source web search engine; the work of creating a cloud to run Nutch on is up to you or a partner:

Web search is a basic requirement for internet navigation, yet the number of web search engines is decreasing. Today’s oligopoly could soon be a monopoly, with a single company controlling nearly all web search for its commercial gain. That would not be good for users of the internet.

Nutch provides a transparent alternative to commercial web search engines. Only open source search results can be fully trusted to be without bias. (Or at least their bias is public.) All existing major search engines have proprietary ranking formulas, and will not explain why a given page ranks as it does. Additionally, some search engines determine which sites to index based on payments, rather than on the merits of the sites themselves. Nutch, on the other hand, has nothing to hide and no motive to bias its results or its crawler in any way other than to try to give each user the best results possible.

(I confess to being distracted myself during meetings by the Blackberry, so I can’t be the one to cast the first stone.)

BerrySearch v1.0 is now available for free over-the-air download by pointing your phone at http://thebogles.com/berrysearch.jad

This version adds IMDb autocomplete and search to the Google, Wikipedia, News, Images, and Techorati Searches of previous versions. 

Other additions:

• Shortcut key sequences allow you to quickly change the search type: “g:” for Google, “i:” for IMDb, “d:” for dictionary, and so forths.
• The “Edit Settings” menu item allows you change the network connection type to BES (if your company has a BES) or Direct (for using your carrier’s APN, which you should configure as described here.) 

I would appreciate input from users on the following question: is it better for BerrySearch to exist as a separate application from Berry411, or should Berry411 be a swiss army knife that does everything?

(Much of the context specific autocomplete in BerrySearch is now supported in Berry411 as well, in a slightly more complex form. For example if you begin a search in Berry411 with “g:” for Google or “imdb:”, the appropriate style of autocomplete is used.)

Please vote in the poll on this question.

 Loading …

Movie geeks will like this one– Berry411 now autocompletes over 200,000 movie titles from IMDB and links to a fully mobile-optimized version of the IMDB site. 

No custom code had to be written for IMDB reformatting, thanks to a few new advanced options in Berry411 search plugin configuration to strip images and apply the reformatting to linked pages on the same site.

Evil 1.0 was Microsoft.  Evil 2.0 is Apple and Google. 

I know Evil 1.0 well. I worked for it. I recognize Evil 2.0 easily by its signs.  I know the bad– and the good– that Evil can accomplish.

 Evil 2.0 must be understood.

Evil 1.0 was geeky and asocial.
	Evil 2.0 is cool and charismatic.

Evil 2.0 seduces.  We lust to touch the IPhone, even at the cost of our ability to choose software, carriers, and media formats.  No Skype or Ogg here, unless Apple and Cingular want them.

Evil 2.0 is lock in at the grandest scale. 

Evil 2.0 is smarter than you are.

Evil 2.0 misleads through paradox.

It is the evil that says “Don’t be evil”.  It’s the crawler that can’t be crawled, the datamining blackhole.

It’s the privately owned judge of who gets noticed on the internet.

Evil 2.0 is the Google search API that goes straight from beta to oblivion.

Evil 2.0 is all-knowing. No one knows exactly how much Google knows about you– except Google.

Take the thesis of Evil 1.0, synthesize with Evil 2.0, and toss in a good dash of Homeland Security, and we have a grave threat to freedom of software, thought and privacy. And we will greet our new overlords with flowers and iPhones.

Chirag Mehta’s US Presidential speeches tag cloud displays a tag cloud of presidential address from 1776 to the present with a time slider. Interesting to watch how our concerns have shifted over time. (Via O’Reilly Radar.)

 
 The 2006 State of the Union Address  (George Bush)

 
The 1789 Inaugural Address (George Washington)

The Jobster team shipped a number of new features for professionals last night, as described by Jason.  

Personalized url.  Now, every jobster user can have their own personalized url. 

        [phil writes: mine is http://www.jobster.com/people/philbo]

Get your own by visiting your home page on jobster.  Having your own personalized url on jobster is an easy way to direct people to your career profile, whether you are looking for a job, hiring, or just networking for your career.  Act fast to get yours before someone else claims it.

Personalized home page.  Now, every registered jobster user gets their own personalized homepage with real-time suggestions of jobs that match their interests and experiences.  To see this feature, you will need to create a free jobster account.

Video profiles.  This is a feature that we’ve received many requests for and are now doing some in-production testing with. 

Video profiles are a hot topic of late, as noted here in the Wall Street Journal and here on NPR.  Many of us also remember the story of Aleksey Vayner and his misfire here as well.

In our first implementation of video profiles, any Jobster user can add a YouTube video to their Jobster profile.  For instance, you can see mine here. This is a potentially great way for candidates to stand out by posting a video resume. We’ve also had a lot of interest from employers in using this feature to post videos about what it’s like to work at their companies or in their particular groups, or to just better explain what they are looking for in a candidate. 

We’ll see how this one develops.  A recent poll i conducted on this blog found that 4 out of 10 people would be interested in adding video to their career profile (higher than expected), and that there was considerable interest in posting video resumes and/or videos about what it’s like to work at a company.  Again, we’ll see how this one goes…  

Yellow pages are great for finding businesses by name or category, but if you want to find a specific product locally, sites like Yokel are a much better way to go.  Yokel lets you know exactly which local stores have a product in stock and how much it costs.

Thanks to some integration help from the good folks at Yokel, Berry411 mobile search is now able to interactively suggest helpful Yokel searches.

Suppose you’re looking for a Wii remote, for instance.  As you type wii, Berry411 will interactively suggest searches for Wii related products on Yokel (such as wii console and wii remote).

Clicking on those links take you directly to Yokel search results, including availability and prices at local stores.

In addition to a comprehensive set of product links from Yokel, Berry411 also automatically suggests other relevant plugin searches based on past searches by users. 

If you search for Happy Feet, relevant searches will be suggested using the plugins for Google Movies, IMDB, and Yokel shopping, as well as yellow pages.

These new features are available automatically to users of recent Berry411 releases (3.7 and above), no upgrade is required.

Urbanspoon has added Chicago and New York to the set of cities they index for restaurant reviews. If you love food and live in those cities, give them a try; the site is a great resource. (Berry411 hits include links to Urbanspoon results where available; this now includes the new cities.)

An interesting new Urbanspoon feature is cities at night, an interactive visualization of restaurant distribution in a city:

Sometimes you just blunder into a new feature. When we were setting up New York and Chicago, we thought it would be useful to have a visualization of all the restaurants in a city, divided by neighborhood. This was just to spot errors in our data.

Funny thing is, it looked really cool. At least to us. Kind of like flying over a city on a dark night, except only the restaurants have their lights on. In New York, for example, we have over 23,000 restaurants and it’s amazing how well the shape of the landscape and population density can be revealed with just restaurants.

So we decided to put it out there for you all to play with. Here’s the “night-time” view of Chicago, New York and Seattle.

Adam tells me some very interesting stories about the challenges they faced in normalizing the indexed data; with any luck he’ll blog about some of the tricks they employed (without giving away too much secret sauce.)