The main search engines
Search engines are automated tools that send out 'crawlers' or 'spiders' to scan the web and index every website that they find by following hyperlinks from one site to another. The submission process for a website is simply to provide the URL for a page and the search engine will visit the site and index the page content and HTML tags.
In contrast to directories which are compiled manually by humans, search engines are automated and so tend to have a much greater coverage of the web. They also provide better opportunities for a website to achieve a ranking across more search terms - although optimising a website for each search engine is crucial in order to get these results.
The most popular search engines
The following is a summary of the most popular search engines used globally and within Australia and New Zealand. They all operate differently and will use a variety of 'algorithms' to retrieve websites once a search has been made on them. More information is provided by links to pages on the history of each of the main search engines and information about how to get a site listed and to achieve a high ranking on these tools.
A late entrant in search engine terms, Google only launched in 1998 but has rapidly become the most popular search engine favoured by savvy web users due to the speed and accuracy of results served. It is now the undisputed leader in terms of search traffic and the most important search tool to achieve a top placement on, with Google results also now being used by AOL and Netscape Search, as well as portal sites that licence the data.
Google is also one of the main search engine to use 'link popularity' as a key element impacting search rankings, whilst Google's PageRank measure is a core indicator of how a site may perform in the search results, based on both internal and external factors on a website.
A brief history of Google.
Google optimisation - getting a site indexed and ranked.
Microsoft's own web portal MSN has always provided a search service to users - MSN Search - which was also set as the default search option from the Internet Explorer browser, so it tended to attract a reasonably high proportion of search traffic. However, Microsoft have been quite slow to recognise the potential of developing a search service themselves and therefore the MSN search function was provided by third parties, most notably Inktomi.
In the Spring of 2005 however, MSN finally launched its own search engine which was supported by a large advertising campaign. This new search tool is based on the Google model and provides some strong results, with a frequently updated database and a selection of advanced search functions. MSN Search is one of the most popular search tools in Australia and benefits from its association with the main MSN portal and other online tools provided by Microsoft.
A brief history of MSN Search.
MSN Search optimisation - getting a site indexed and ranked.
Yahoo! was traditionally the largest and most important directory on the Internet and it developed into a leading online portal offering a range of services to users. Over the last few years, the Yahoo! directory has dropped its profile and as the size of the web has grown dramatically, the provision of a spidered search engine service became more important, with the introduction of Yahoo! Search.
The search engine service on Yahoo! was initially provided as a supplement to the directory and results were sourced from third parties - initially from Inktomi until this was replaced by Google as the primary search supplier in 2003. However, in 2004 Yahoo! went through a significant acquisition process, buying Overture (along with AltaVista and AlltheWeb which had previously just been taken over by Overture) and then Inktomi. Yahoo! Search has now established a stronger search offering by using the Inktomi search engine and developing this with technology from its other search companies, so that the reach of Yahoo! has grown, to hold the third largest share of the Australian search market.
A brief history of Yahoo! Search.
Yahoo! Search optimisation - getting a site indexed and ranked.
A list of Australian search engines and New Zealand search engines and directories can be seen here.
Some of the other main search engines - present and past
Developed in Norway and launched in Spring 1999, AlltheWeb/FAST remained a relatively unknown search engine despite it providing results to the Lycos network. It was a well respected search index and retrieval system, offering a powerful range of search options and an accuracy that was considered second only to Google. AlltheWeb was acquired by Overture in 2003 and soon after by Yahoo!, so that this search engine is simply a satellite of Yahoo!, providing the same search engine results.
A brief history of AlltheWeb / FAST.
Since its launch at the end of 1995, AltaVista became one of the largest and most popular search engines, providing a good collection of search tools and advanced search functions. However, it suffered financial and operational setbacks during 2001, consequently losing a large number of users to the ascendant Google. It relaunched its interface - again - and provided new search functionality at the end of 2002 but was acquired by Overture and then Yahoo! - suffering the same fate as AlltheWeb. Once the leading search engine on the web, AltaVista is now simply a replica of the Yahoo! search engine.
Ask was formerly the Ask Jeeves search engine, which existed in an Australian version until 2001. Following a rebrand of the main US business in February 2006, Jeeves 'the butler' was finally dropped and the name of this search engine was changed as the focus was revised to try to gain more market share. The original Ask Jeeves always tried to set itself apart as a different kind of search engine - the search tool for the masses, or those that aren't very confident on using web search tools. Initially Ask Jeeves promoted itself as providing results to questions posed in plain English, so that users could type in a general question and receive relevant results. The information presented came from numerous sources - part search engine results, part directory, part paid links.
Direct Hit was a unique search tool that measured 'click-throughs' and the time spent on websites to improve the rankings on its search results. It worked closely with key partners, such as Ask Jeeves and HotBot, to refine and develop these results. Owned by Ask Jeeves, this search tool was superceded by Teoma towards the end of 2002.
A brief history of Direct Hit.
One of the original search engines, Excite became a popular general search tool and portal-based service. It acquired two competitors - Magellan and Webcrawler - in 1996, but sadly succumbed to financial losses at the end of 2001 by closing its own search database and many regional search sites. Excite.com currently provides metasearch results, including from Overture and Inktomi.
Launched in Spring 1996, HotBot is now owned by Lycos and is a large index with a good range of advanced searching features. Having taken its search results from the Inktomi and Direct Hit databases for some years, it was another search tool to relaunch at the end of 2002 with a new, cleaner look. It now offers a more powerful search functionality with multiple search options allowing users to search through the databases from Ask or MSN Search.
Inktomi did not provide a search engine interface in its own right, but it spidered the web and provided an index to many key partners such as HotBot and MSN. During 2002 it lost a significant share of the search market by key partners such as AOL and Netscape moving to Google results. At the end of the year Inktomi was acquired by Yahoo! and the search technology has now been absorbed and rebranded as Yahoo! Search.
One of the original search engines, Lycos still spiders the web but now places more emphasis on search results from FAST and on its directory format, taking data from the Open Directory Project (see the top Directories). Investment in this search tool has reduced its usage over recent years and more prominence is now given to its portal services.
Teoma is a relatively new search engine, founded in 2000 by a team from Rutgers University in the US. Using a highly regarded combination of criteria to rank relevant websites, this was seen as a potential new competitor to the dominance established by Google. During 2002 Teoma was purchased by Ask Jeeves and used as the main search results on this tool. When Ask Jeeves relaunched in 2006 as just Ask, the Teoma search engine was fully absorbed into this new search engine and no longer exists as a standalone search tool.
Other types of specialised search engines
As the number of websites and pages of content continues to grow substantially, even the main search engines are finding it difficult to index everything and provide the most relevant or accurate results. We have provided a forecast of how search engines of the future may develop, but one area that is becoming more important in a search engine marketing campaign is to target more specialised search engines, such as:
- Shopping search engines - there are a number of price comparison search tools that can provide ecommerce websites with another channel to present their product details and attract traffic through to their site.
- Local search engines - regional search traffic is vital for many businesses that just want to attract traffic from a local area. The growth in the number and quality of local search tools now provide an excellent resource to reach this market.
- Image search engines - searching for pictures has been a function provided by the main search engines for some time and some companies can easily take advantage of this search option by optimising their sites to rank well within the image results.
To find out how your website can achieve a top ranking on these search engines contact us for more information or request a FREE website assessment.
Web Search Workshop - Search engine optimisation and website marketing specialists, Australia
We also provide