Mining web search behaviors: Strategies and techniques for data modeling and analysis

There is a growing interest in modeling Web searching behaviors using query log data. In this project, we identified some gaps in current research. We propose to model Web search behaviors along three dimensions: interactions, linguistic and cognitive behaviors. We propound Web search session as a vital important concept to study interactive behaviors using query logs. A highly granular, comprehensive relational model is presented for data extraction and transformation along with strategies and methods for session identification. To facilitate analysis, we developed an interactive Web tool for exploring different session thresholds. We demonstrate statistically that the 80‐20 empirical rule shows promise for setting session boundaries. In addition, we recommend that decisions for session boundary thresholds should be determined based on specific query corpus characteristics such as type and size of the database searched, and type of searchers who submit the queries. Our approach is based on the fact that data mining researchers do not always know all the hypotheses that the data can answer at the outset and the log data are diverse across environments due to the lack of standardization. This model maximizes transactional data inclusion, is flexible in handling data content, and can be extended easily to incorporate new hypotheses and new data elements as mining progresses.