Sideffective - system to mine patient reviews

of the Thesis Sideffective System to Mine Patient Reviews: Sentiment Analysis by Deepak Yalamanchi Thesis Director: Prof. Tomasz Imielinski Sideffective is the system to crawl, rank and analyze patient testimonials about side effects from common medications. Since the wealth of any mining model is the Data corpus, the data collection phase involved extensive crawling of massive medical websites comprised of user forums from the internet. Subsequently, the raw files were subjected to certain site-specific parsing routines, yielding outputs conforming to a well-defined data model. Currently, the system holds close to 400,000 user testimonials pertaining to more than 2500 drugs/medicines. Sideffective aims at gathering and aggregating this wealth of information, build useful associations and present interesting observations and numeric validations, all in a user-friendly interface. The important issues that we have tried to tackle are: Extracting side effects without relying on pre-built lists, aggregating distribution of different side effect for a give drug, site-specific search, ranking and determining the negativity of reviews. The system has been jointly built by Deepak Yalamanchi and Sangeetha Rajagopalan under the guidance of Prof. Tomasz Imielinski. This thesis focuses mainly on Sentiment Analysis of patient reviews. While most existing sentiment analysis systems are predicated by POS (parts of speech) tagging or Bayesian sentiment analysis methods, the same cannot be applied to medical reviews as they generally carry a negative flavor in them. We thereby approached the problem by identifying the features in the sentence and calibrating the sentiment on a Negativity Meter based on their relation to sentiment words. A