Generating Semantically Precise Scene Graphs from Textual Descriptions for Improved Image Retrieval

Authors Sebastian Schuster, Ranjay Krishna, Angel Chang, Li Fei-Fei and Christopher D. Manning Actual results using a popular image search engine (top row) and ideal results (bottom row) for the query a boy wearing a t-shirt with a plane on it. Abstract Semantically complex queries which include attributes of objects and relations between objects still

Read MoreRead More

Image Retrieval using Scene Graphs

Authors Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David Ayman Shamma, Michael Bernstein, Li Fei-Fei Image search using a complex query like “man holding fish and wearing hat on white boat” returns unsatisfactory results in (a). Ideal results (b) include correct objects (“man”, “boat”), attributes (“boat is white”) and relationships (“man on boat”). Abstract

Read MoreRead More

Editorial Algorithms: Using Social Mediato Discover and Report Local News

Authors Schwartz, R., Naaman M., Teodoro R. Screenshot of CityBeat interface showing the Detected Events List, Event Window and the Statistics Sidebar Abstract The role of algorithms in the detection, curation and broadcast of news is becoming increasingly prevalent. To better understand this role we developed CityBeat, a system that implements what we call “editorial algorithms”

Read MoreRead More

CityBeat: Real-time Social Media Visualization of Hyper-local City Data

Authors Xia C., Schwartz, R., Xie K., Krebs A., Langdon A., Ting J. and Naaman M. Abstract With the increasing volume of location-annotated content from various social media platforms like Twitter, Instagram and Foursquare, we now have real-time access to people’s daily documentation of local activities, interests and attention. In this demo paper, we presentCityBeat1,

Read MoreRead More

Robust detection of hyper-local events from geotagged social media data

Authors Xie K., Xia C., Grinberg N., Schwartz R., and Naaman M. Architecture of our local event detection system. Including data collector, time-series builder, Gaussian Process regression model, alert engine and classifier. Arrows indicate input and output flow of each module. Abstract An increasing number of location-annotated content available from social media channels like Twitter,

Read MoreRead More

Making Sense of Cities Using Social Media: Requirements for Hyper-Local Data Aggregation Tools

Authors Schwartz, R., Naaman M., Matni, Z. Examples of geo-tagged social media data visualizations mockups. Clockwise: geo-tagged topic groupings, keywords appearance graphs, volume graph and heat map Abstract As more people tweet, check-in and share pictures and videos of their daily experiences in the city, new opportunities arise to understand urban activity. When aggregated, these

Read MoreRead More