<<Taikichiro Mori Memorial Research Fund>>

Graduate Student Researcher Development Grant Report


Research Project: An Adaptive Imagination-based Image and Video Retrieval System for Reminding Visual Information

Project Researcher: Nguyen Thi Ngoc Diep

Affiliation:  1st year Doctoral student, Graduate School of Media and Governance, Keio University, Japan


Research Abstract

This research focuses on a new search mechanism and implementation of an information retrieval system for large-scale video and video database. The search mechanism is based on users’ imaginations as queries and has response time controllable. It is also expected to work fast and effectively as a real-time search application in general cases. Further, in some special cases, users can expect to get better results when giving the system more time to run.


By manipulating users’ imagination as query, the system provides users a flexible tool to remind their visual memories from daily captured data. As a consequence, the system is also supposed to bring users new memorizing and recalling experiences for their daily activities.


Research Activities and Results


Three contextual aspects in every search

Even by an input image, a user under different circumstances or different users have different interests while searching for information.


Description: Macintosh HD:Users:chupi:Dropbox:Screenshots:Screenshot 2015-02-26 23.48.43.png



System Architecture


Description: Macintosh HD:Users:chupi:Dropbox:Screenshots:Screenshot 2015-02-26 23.57.34.png


Search algorithms

+ Max first search

+ Threshold-based iterative search

+ Accelerating iterative search

+ Combinatorial search

+ Perfect search




      *Journal paper:

[1] D.T.N Nguyen and Y. Kiyoki, “An Adaptive Search Path Traverse for Large-scale Video Frame Retrieval”, Information Modelling and Knowledge BasesXXVI, p324 – 342, 2014.

[2] D.T.N Nguyen and Y. Kiyoki, “An imagination-based query creation method for image retrieval,” Information Modelling and Knowledge Bases XXIV, p. 201 – 220, 2012.

      *International Conference Papers:

[3] Nguyen, D.T.N., Sasaki, S. and Kiyoki, Y.:"A Mashup System Using Social Networks For Collaborative Environmental Monitoring Education With Spatiotemporal Analysis And Visualization", the 10th IASTED International Conference on Web-based Education (WBE 2013), Innsbruck, Austria, p. 909 – 916, February 13-15, 2013

[3] Nguyen, D.T.N., Sasaki, S. and Kiyoki, Y.: “Imagination-based Travel Designing System with 5D World PicMap”, the First Indonesian-Japanese Conference on Knowledge Creation and Intelligent Computing (KCIC 2012), Surabaya, Indonesia, 8 pages, March 13-14, 2012.

[4] Nguyen, D.T.N., Sasaki, S. and Kiyoki, Y.: "5DPicMap: Imagination-based Image Search System with Spatiotemporal Analyzers", IADIS International Conference, e-Society 2011, Avila, Spain, p. 272 – 278, March 10-13, 2011.

(5) Nguyen, D.T.N, Sasaki, S. and Kiyoki,Y.: "Imagination-based Image Search System with Dynamic Query Creation and Its Application," The 13th IASTED International Conference on Software Engineering and Applications SEA 2010, Marina Del Ray, USA, p. 435 – 442, November 8 - 11, 2010.

      *Domestic Conference Papers and Posters

[6] Nguyen, D.T.N., Sasaki, S. and Kiyoki, Y.: “Visualization and Analysis of Global Environmental Change with Multimedia Retrieval”, International Forum for Sustainable Asia and the Pacific (IGES - ISAP 2012), Yokohama, Japan, poster,  July 24-25, 2012.

[7] Nguyen, D.T.N.,;Sasaki, S. and Kiyoki, Y.: “Spatiotemporal Analysis and Visualization for Environmental Research and Education", Japanese Society for Information and Systems in Education (JSiSE) Research Report, Hokkaido, Japan, 4 pages, July 15, 2012.


Future Challenges

-       Distributed indexing strategy

-       Investigate the user interface

-       Database system implementation



I would like to express my gratitude to Taikichiro Mori Memorial Research Fund, which supports great financial policies to researchers like me to make progress on our ways.