In this proposal, we study the problem of understandinghuman sentiments from large scale collection ofInternet images based on both image features and contextualsocial network information (such as friend comments anduser description). Despite the great strides in analyzing usersentiment based on text information, the analysis of sentimentbehind the image content has largely been ignored. Thus, we extend the significant advances in text-based sentimentprediction tasks to the higherlevel challenge of predicting theunderlying sentiments behind the images. We show that neithervisual features nor the textual features are by themselvessufficient for accurate sentiment labeling. Thus, we provide away of using both of them, and formulate sentiment predictionproblem in two scenarios: supervised and unsupervised. Wedevelop an optimization algorithm for finding a local-optimasolution under the proposed framework. With experiments ontwo large-scale datasets, we show that the proposed methodimproves significantly over existing state-of-the-art methods. In the future, we are going to incorporating more informationon the social network and explore sentiment on signed social network.

Original languageEnglish (US)
Title of host publicationProceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages8
ISBN (Print)9781467384926
StatePublished - Jan 29 2016
Event15th IEEE International Conference on Data Mining Workshop, ICDMW 2015 - Atlantic City, United States
Duration: Nov 14 2015Nov 17 2015


Other15th IEEE International Conference on Data Mining Workshop, ICDMW 2015
Country/TerritoryUnited States
CityAtlantic City


  • Computer vision
  • sentiment analysis

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Computer Science Applications


Dive into the research topics of 'Sentiment Analysis for Social Media Images'. Together they form a unique fingerprint.

Cite this