Lecture abstract

With the proliferation of visual content on the Internet and in various domains, many exciting approaches are being pursued to improve search capabilities. However, most work follows conventional wisdom from text search: the user provides query descriptions (either keywords or example images) and the system returns a ranked list of results presented in a linear fashion. In this talk, we assert that effective visual search requires a novel paradigm with different emphasis on helping users to make sense of the results throughout the search process, starting from query formulation, to search result summarization, and finally to intuitive navigation of content and semantic spaces. As a proof of concept, we will present several new techniques and systems we have developed using this new paradigm - (1) a real-time assistive user interface to help users formulate effective queries and navigate through the image space, (2) Visual Islands, an organizational tool to help users rapidly comprehend image search results while non-linearly delving deeper into a result list, and (3) Visual Migration Map, a new tool for tracing the plausible manipulation relations among visual content. We term this new paradigm Sensible Visual Search, and will show how Sensible Search techniques may be incorporated along with existing research such as semantic concept detection and relevance feedback in developing the next generation of visual search solutions.

(Joint work with Lyndon Kennedy and Eric Zavesky)

Speaker info

Shih-Fu Chang is a Professor of Electrical Engineering and Director of Digital Video and Multimedia Lab at Columbia University. He has made significant contributions to image/video search, content recognition, image forensics, and mobile video transmission. His research has been frequently cited with several large demo systems deployed online (such as VisualSEEk, VideoQ, CUVid Search, and TrustPhoto) and more than 10 technologies transferred to industry. Recognitions awarded to him and his group include a Navy ONR Young Investigator Award, an IBM Faculty Development Award, an ACM Recognition of Service Award, an NSF CAREER Award, two Best Paper Awards, and seven Best Student Paper Awards from IEEE, ACM, and SPIE. He has served in different advising/consulting capacities for major media companies like IBM, Kodak, Microsoft, PictureTel, and several other research institutions. He is an IEEE Fellow of IEEE and currently Editor-in-Chief for IEEE Signal Processing Magazine.