Search engines on my mind

I have search engines on my mind. Johnny Gee put an interesting bug in my bonnet when he blogged that EMC will be coming out with an enterprise search engine that promises to challenge Google enterprise search. I don't want to misinterpret what he says, so judge for yourself; here are his words:

With FAST being acquired by Microsoft, there has been a lot of talk on what will happen with the FAST search engine that is currently OEM with Content Server. Well, it seems that Lucene, which FAST also uses, will be the core of the how content is indexed with Documentum. However, the core search engine is going to built by EMC. I'm guessing this is to compete with Google's entry into ECM space with its own adaptors to Documentum repositories. Since the core of any search engine is the search algorithm and/or technology it uses, I'm not going to reveal anything that is proprietary. I will say the architecture for this new search engine is quite REVOLUTIONARY.

I can't wait to hear more about it, but I'll stop short of calling Johnny to see if I can get him to spill; he's far too ethical, besides, he's probably signed something that says he can't. (As a complete aside, though, I wonder if open source vendors handle the "special people get to sneak a peak at our cool new toys, flaunt the fact, and reveal nothing " in the same way.) I'm also wondering how likely it is that anyone is going to build something that outperforms Google in the near term.

It's all about algorithms, I get that. I also understand that knowledge workers need to be able to access information according to user-determined relevance rather than popularity-based page rank. Google gets that too; I had a good talk with Avinash Kaushik about it at a Web 2.0 conference I covered for the mainstream press. I regret to report that he didn't tell me anything proprietary (nor did I ask.) But what he did say is that they were working on Google becoming more intuitive; I think it will be able to almost read my mind ten years from now. Imagine it, folks might end up feeling more understood by a search engine than by people they care about.

All this search engine chatter motivated me to check out the newest players on the market: there's Cuil (pronounced "cool") which ranks my personal content much differently than Google (no Linkedin hits on the first page at all- they're first in Google), powerset (which I think is short on hits), Viewzi (which is incredibly cool), and boogami.

I'm rooting for anyone who will relieve me of the need to create a smart search strings.