Even though all the images contain the same objects (a person and a bicycle), it is the relationship between the objects that determine the holistic interpretation of the image. Authors Cewu Lu*, Ranjay Krishna*, Michael Bernstein, Li Fei-Fei Abstract Visual relationships capture a wide variety of interactions between pairs of objects in images (e.g. “man
Authorresearch
Generating Semantically Precise Scene Graphs from Textual Descriptions for Improved Image Retrieval
Actual results using a popular image search engine (top row) and ideal results (bottom row) for the query a boy wearing a t-shirt with a plane on it. Authors Sebastian Schuster, Ranjay Krishna, Angel Chang, Li Fei-Fei and Christopher D. Manning Abstract Semantically complex queries which include attributes of objects and relations between objects still
Embracing Error to Enable Rapid Crowdsourcing
(a) Images are shown to workers at 100ms per image. Workers react whenever they see a dog. (b) The true labels are the ground truth dog images. (c) The workers’ keypresses are slow and occur several images after the dog images have already passed. We record these keypresses as the observed labels. (d) Our technique
Image Retrieval using Scene Graphs
Image search using a complex query like “man holding fish and wearing hat on white boat” returns unsatisfactory results in (a). Ideal results (b) include correct objects (“man”, “boat”), attributes (“boat is white”) and relationships (“man on boat”). Authors Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David Ayman Shamma, Michael Bernstein, Li Fei-Fei Abstract
Editorial Algorithms: Using Social Media to Discover and Report Local News
Screenshot of CityBeat interface showing the Detected Events List, Event Window and the Statistics Sidebar Authors Schwartz, R., Naaman M., Teodoro R. Abstract The role of algorithms in the detection, curation and broadcast of news is becoming increasingly prevalent. To better understand this role we developed CityBeat, a system that implements what we call “editorial algorithms”
CityBeat: Real-time Social Media Visualization of Hyper-local City Data
Authors Xia C., Schwartz, R., Xie K., Krebs A., Langdon A., Ting J. and Naaman M. Abstract With the increasing volume of location-annotated content from various social media platforms like Twitter, Instagram and Foursquare, we now have real-time access to people’s daily documentation of local activities, interests and attention. In this demo paper, we presentCityBeat1,
Ensemble: Exploring Complementary Strengths of Leaders and Crowds in Creative Collaboration
A story scene showing (a) the winning draft for a scene. (b) Tabs can be used to switch to different drafts, with the winning draft being shown by default. (c) The scene prompt helps focus contributors on specific writing goals. (d) Contributors can make comments discussing the scene at a high level. Authors Joy Kim,
Robust detection of hyper-local events from geotagged social media data
Architecture of our local event detection system. Including data collector, time-series builder, Gaussian Process regression model, alert engine and classifier. Arrows indicate input and output flow of each module. Authors Xie K., Xia C., Grinberg N., Schwartz R., and Naaman M. Abstract An increasing number of location-annotated content available from social media channels like Twitter,
Making Sense of Cities Using Social Media: Requirements for Hyper-Local Data Aggregation Tools
Examples of geo-tagged social media data visualizations mockups. Clockwise: geo-tagged topic groupings, keywords appearance graphs, volume graph and heat map Authors Schwartz, R., Naaman M., Matni, Z. Abstract As more people tweet, check-in and share pictures and videos of their daily experiences in the city, new opportunities arise to understand urban activity. When aggregated, these