Cleaning and sentiment tasks for news transcript data

Abstract

Today, vast amount of news in various forms is hosted on the web. They include news articles, digital newspapers, news clips, podcasts, and other sources. Traditionally, news articles and writings have been used to carry out sentiment analysis for topics. However, news channels and their transcripts represent vast data that have not been examined for business aspects. In this light, we have charted out a methodology to gather transcripts and process them for sentiment tasks by building a system to crawl Webpages for documents, index them, and aggregate them for topic analysis. Vector space model has been used for document indexing with predetermined set of topics and sentiment analysis carried out through the SentiWordNet data set, a lexical resource used for opinion mining. The areas of insight are mainly the polarity index (degree of polarity or subjectivity) of the news presented as well as their coverage. This research shows insights that can used by businesses to assess the content and quality of their content. � Springer Science+Business Media Singapore 2017.