Scalable Computing and Communications: Theory and Practice

23

–––––––––––––––––––––––

A Survey of Techniques for Improving Search Engine Scalability through Profiling, Prediction, and Prefetching of Query Results

C. Shaun Wagner, Sahra Sedigh, Ali R. Hurson, and Behrooz Shirazi

23.1   INTRODUCTION

Search engines are vital tools for handling large repositories of electronic data. As the availability of electronic data continues to grow, search engine scalability gains importance [1]. As noted in the literature, researchers have measured search engine performance in terms of response time and relevance [2‒5], both of which suffer as the number of potential search results increases. Response (or search) time is the time elapsed between initiation of a query and when the desired data item is available to the user. Relevance quantifies the relevance of the results returned. Search engines typically exhibit a trade-off between response time and relevance [6]. However, irrelevant search results (even when delivered quickly) require that the query be refined and repeated, increasing the overall search time [2]. This chapter presents a survey of techniques that can be used to increase the relevance and to reduce the response time of search engines by predicting user queries and prefetching the results.

FIGURE 23.1. Operational flow of a standard query‒response search engine.

When a user is querying for a specific item, such as a book title, ...