Generating Semantically Precise Scene Graphs from Textual Descriptions for Improved Image Retrieval

Actual results using a popular image search engine (top row) and ideal results (bottom row) for the query a boy wearing a t-shirt with a plane on it. Authors Sebastian Schuster, Ranjay Krishna, Angel Chang, Li Fei-Fei and Christopher D. Manning Abstract Semantically complex queries which include attributes of objects and relations between objects still

Read More

Image Retrieval using Scene Graphs

Image search using a complex query like “man holding fish and wearing hat on white boat” returns unsatisfactory results in (a). Ideal results (b) include correct objects (“man”, “boat”), attributes (“boat is white”) and relationships (“man on boat”). Authors Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David Ayman Shamma, Michael Bernstein, Li Fei-Fei Abstract

Read More

Editorial Algorithms: Using Social Media to Discover and Report Local News

Screenshot of CityBeat interface showing the Detected Events List, Event Window and the Statistics Sidebar Authors Schwartz, R., Naaman M., Teodoro R. Abstract The role of algorithms in the detection, curation and broadcast of news is becoming increasingly prevalent. To better understand this role we developed CityBeat, a system that implements what we call “editorial algorithms”

Read More

CityBeat: Real-time Social Media Visualization of Hyper-local City Data

Authors Xia C., Schwartz, R., Xie K., Krebs A., Langdon A., Ting J. and Naaman M. Abstract With the increasing volume of location-annotated content from various social media platforms like Twitter, Instagram and Foursquare, we now have real-time access to people’s daily documentation of local activities, interests and attention. In this demo paper, we presentCityBeat1,

Read MoreRead More

Robust detection of hyper-local events from geotagged social media data

Architecture of our local event detection system. Including data collector, time-series builder, Gaussian Process regression model, alert engine and classifier. Arrows indicate input and output flow of each module. Authors Xie K., Xia C., Grinberg N., Schwartz R., and Naaman M. Abstract An increasing number of location-annotated content available from social media channels like Twitter,

Read More

Making Sense of Cities Using Social Media: Requirements for Hyper-Local Data Aggregation Tools

Examples of geo-tagged social media data visualizations mockups. Clockwise: geo-tagged topic groupings, keywords appearance graphs, volume graph and heat map Authors Schwartz, R., Naaman M., Matni, Z. Abstract As more people tweet, check-in and share pictures and videos of their daily experiences in the city, new opportunities arise to understand urban activity. When aggregated, these

Read More