This is a continuation of my earlier review; What makes Google so special? Sure enough Google is the most popular and fast search engine that returns the relevant results fast.
Google also has lot of trust and has higher loyalty i.e. people are loyal to Google and wouldn’t try any other search engine. Is Google invincible? Are there any successors? Successor or no is not known yet but there is a contender. Now Google is being challenged by a new technology. Just check out their new search engine; WISEnut; that is just as fast or is it faster, just as smart or is it smarter and just as comprehensive or do they have a wider coverage.
THE COMPANY:
WISEnut, Inc., founded in August 1999 and based in Santa Clara, is an advanced search technology company founded by Yeogirl Yun, co-founder and former CTO of mySimon. (mySimon, an online comparative shopping service was purchased by CNET for $700M)
THE COVERAGE:
As on Aug 18, 2001, Google claims 1, 387, 529, 000 web pages. Whooping 1 billion plus! But quite unknown is the fact that on the same day WISEnut claims 1, 495, 332, 308 Web pages. Is the claim real or just some number?
THE SPEED:
I tried searching for WISEnut;. Google produced 1, 060 documents in 0.09 sec. Something which is one can be proud of, but WISEnut produced 1, 096 documents in the same time.
RELEVANCY:
I tried Google’s advanced search and in Search for words section gave the words ;White house” (no jokes implied) and in “do not include these words” section gave an list of words like “usa”, “U.S.”, “government”, “president”. Google gave me 699000 pages against 313000 pages returned by WISEnut. Why? Is it that the coverage is less? I don’t deny this but I feel that this is probably because WISEnut returns results that are context-sensitive too.
FEATURES:
Google has a simple interface with a big Google logo, a textbox, a ‘Search’ button, links like “Advanced Search” and “Preference” and few links at the bottom. WISEnut has taken cues from them and maintain a similar simplicity. They have “WISESearch” section that has only three textboxes. One to mention the words, other to mention the words should not be included and a third to mention an exact phrase. The preferences has same features as that of Google such as languages, no. of results displayed at a time etc. But Google gains on added features such as special searches such as universities, U.S. government etc and then Google has also added image search. Google has concept of Web Directories, which gives results classified under several headings or categories. WISEnut does this at runtime. It classifies the results being searched and then puts them under categories that are displayed at top of page as “WISEGuide”.
TECHNOLOGY:
Complex and requires diagram to understand but will try to put it for all readers. There are 6 important things for search engine: Web crawler, link or URL management system, indexer, ranking system, database and query server.
Web Crawler:
WISEnut’s crawler called Zyborg crawls 50 million web pages per day returning entire web pages for indexing and storing. Wow! It is polite and respects the settings and rules mentioned in a special file “\robots.txt” (present on all web sites. Google’s crawler often ignores the settings in \robots.txt).
URL management system (UMS): All the links (URLs) i.e the text and attributes in “anchor tags” are extracted from web pages stored in a database by the crawler. The UMS checks these URLs one by one. If it encounters a new URL, it instructs the crawler to retrieve the web page mentioned in the URL. An identification number is assigned to the URL and this new information is stored in hash table.
Indexer:
A powerful indexer processes more than 30 GB of text/hr. Process? Indexer indexes every single word from the web page, and obtains information of “anchor tags” from the UMS and generates an indexed database.
Ranking system:
If Google has its famous PageRank technology, it is likely to be outdated by the WISERank technology of WISEnut. PageRank had a flaw that it gives rank only based on link structure (see earlier review). The WISERank system considers intrinsic information coming from Web page itself such as title, meta tags and more importantly the body text. Since such a case is open to “spamming” the system also considers extrinsic information similar to Google. i.e. the importance of page is weighed based on the number of links pointing to it especially if the link is pointed by an important page. The difference is that here the link is combined with anchor text attached to the hyperlink, text surrounding the link (information stored by the indexer and UMS) and then combined with the intrinsic information. WISErank determines how much weight should be given to each of these perspective (based on query submitted) and then ranks it accordingly. Thus instead of relying only on link information as google, the system becomes more context-sensitive and gives better relevancy.
Database system:
It stores the web pages, indexed web pages, ranked web pages. Millions of web pages all across the world wide web being stored again at one location! WISEnut claims a proprietary storage system that optimizes disk storage space requirements without sacrificing access speed. It claims that its system can store approximately 4 terabytes of data across 50 machines. Query server:Given the huge size of database, the pages are stored across multiple instances of search nodes. When the query server receives a query string from a user, it collects the relevant pages from each node and then sends it to the browser requesting a particular result.
Other Features:
It gives a text snippet like Google does.
The links are blue and the webpage mention below each result is in orange.
Imagine yourself searching for your Id name. For a site like MS, it will be present on ‘n’ no. of pages such as all the reviews that you have rated and commented so far (remember you name appears as link on left column of the review) besides your private and public profile page. WISEnut will produce only one such result and give a link below saying “100 more pages like this”. Neat isn’t it! 4. WISEnut has been accomplished with a crew of 15 developers and using 100 Windows 2000 servers and in just over a years time.
Any Flaw:
- The WISEguide mentioned earlier lists the categories based on search query but when I clicked on the link, nothing happened. No results were displayed and it gave me error 404 (file not found). Not once, not twice but many times. So I still do not know what it is all about. 2. There is no “cache” like Google. Though this makes Web designers happy, for all others it means that if a web site is not accessible, then you don’t even get a glimpse of the page. So WISEnut is a probable challenge to Google. But then Google is so popular and set in minds of people. They have a Google toolbar, some have it as homepage, some have included it in their “channels; and favourite; lists. Can WISEnut handle it? So technology is there. All it needs is intelligent marketing.