A domain-based feature generation and convolution neural network approach for extracting adverse drug reactions from social media posts

Abstract

The recent popularity of the social media networks including forums, blogs, and micro-blogging networks changed the way patients share their health experiences and treatment options. Such forums offer valuable, unsolicited, uncensored information on drug safety and side effects directly from patients. However, it is very challenging to extract useful information from such forums due to several factors such as grammatical and spelling errors, colloquial language, and post length limitation. Furthermore, due to the sensitivity of the domain for adverse drug reactions (ADR) detection, it is more critical to identify correct ADRs (i.e., achieve higher classification precision) than identifying non-precise ones. The aims of this thesis are: (i) to develop a new approach for ADR classification in twitter posts called Semantic Vector(SemVec); (ii) to explore natural language processing (NLP) approaches for generating domain features from text, and utilizing them for ADRs detection; and (iii) to improve convolution neural network (CNN) ADR classifi- cation precision by incorporating domain features. This thesis proposes a dynamic and pluggable model, named SemVec, for representing words as a vector of both domain and morphological features. Based on the problem domain, domain features can be added or removed to generate an enriched word representation with domain knowledge. SemVec represents each post as a matrix of word vectors, which is fed into CNN. SemVec is scalable, can be applied to other domains by employing relevant natural language processing methods and domain lexicons. The proposed method was evaluated on Twitter (ADR) dataset. Results show that SemVec improves the precision of ADR detection by 13.43% over other state-of-the-art deep learning methods with a comparable recall score.