<<Taikichiro Mori Memorial Research Fund>>

Graduate Student Researcher Development Grant Report

 

Research Project: A Distributed Content-based Video-Frame Retrieval System with Time-constrained Search Algorithm for Large-scale Video Data

Project Researcher: Nguyen Thi Ngoc Diep

Affiliation:  2^nd year Master’s student, Graduate School of Media and Governance, Keio University, Japan

 

Research Abstract

Multimedia retrieval task is facing with increasingly large datasets and variously changing preferences of users in every query. We realize that the high dimensional representation of physical data which previously challenges search algorithms now brings chances to cope with dynamic contexts. In this research we reveal a fast search algorithm using utilization of inverted indexes for high dimen- sional metadata and build a large-scale video frame retrieval environment handling users’ dynamic contexts of querying by imagination, and controlling response time. The search algorithm quickly finds an initial candidate which has highest-match possibility, and then iteratively traverses along feature indexes to find other neigh- bor candidates until the input time bound is elapsed. The experimental studies based on video frame retrieval system show the feasibility and effectiveness of our proposed search algorithm which can return results in a fraction of a second with a high successful rate and small deviation to the expected ones. Moreover, its potential is clear that it can scale to large dataset while preserving its search performance.

Research Activities and Results

 

Framework of Video Frame Retrieval

The video frame retrieval system using inverted feature indexes and max first search algorithm has overview architecture. The system includes seven main modules: video crawler, frame extraction, feature extraction, feature indexing, database manager, feature selection and prioritization, and max first search.

 

 

Time-constrained max first search algorithm

 

Experimental results

 

Input, output interface and results using the proposed max first search algorithm with limited time 5s. “Color” and “shape” are selected as features of interest. (Using 50000-frame database)

 

The results when using the proposed max first search algorithm with different limited input time comparing to results using brute force searching in (b).

 

Publications

 

      *Journal paper:

[1] D.T.N Nguyen and Y. Kiyoki, “An imagination-based query creation method for image retrieval,” Information Modelling and Knowledge Bases XXIV, p. 201 – 220, 2013.

      *International Conference Papers:

[2] Nguyen, D.T.N., and Kiyoki, Y.: “An Adaptive Search Path Traverse for Large-scale Video Frame Retrieval”, 24^th International Conference on Information Modelling and Knowledge Bases, EJC 2014, June 3-6, 2014, Kiel Germany, (submitting)

[3] Nguyen, D.T.N., Sasaki, S. and Kiyoki, Y.:"A Mashup System Using Social Networks For Collaborative Environmental Monitoring Education With Spatiotemporal Analysis And Visualization", the 10^th IASTED International Conference on Web-based Education (WBE 2013), Innsbruck, Austria, p. 909 – 916, February 13-15, 2013

[3] Nguyen, D.T.N., Sasaki, S. and Kiyoki, Y.: “Imagination-based Travel Designing System with 5D World PicMap”, the First Indonesian-Japanese Conference on Knowledge Creation and Intelligent Computing (KCIC 2012), Surabaya, Indonesia, 8 pages, March 13-14, 2012.

[4] Nguyen, D.T.N., Sasaki, S. and Kiyoki, Y.: "5DPicMap: Imagination-based Image Search System with Spatiotemporal Analyzers", IADIS International Conference, e-Society 2011, Avila, Spain, p. 272 – 278, March 10-13, 2011.

(5) Nguyen, D.T.N, Sasaki, S. and Kiyoki,Y.: "Imagination-based Image Search System with Dynamic Query Creation and Its Application," The 13^th IASTED International Conference on Software Engineering and Applications SEA 2010, Marina Del Ray, USA, p. 435 – 442, November 8 - 11, 2010.

      *Domestic Conference Papers and Posters

[6] Nguyen, D.T.N., Sasaki, S. and Kiyoki, Y.: “Visualization and Analysis of Global Environmental Change with Multimedia Retrieval”, International Forum for Sustainable Asia and the Pacific (IGES - ISAP 2012), Yokohama, Japan, poster,  July 24-25, 2012.

[7] Nguyen, D.T.N.,;Sasaki, S. and Kiyoki, Y.: “Spatiotemporal Analysis and Visualization for Environmental Research and Education", Japanese Society for Information and Systems in Education (JSiSE) Research Report, Hokkaido, Japan, 4 pages, July 15, 2012.

 

Future Challenges

-       Distributed indexing strategy

-       A new NoSQL database management framework for multimedia data

-       Investigate the user interface

-       Improve the search performance to become more stable

 

Acknowledgement

I would like to express my gratitude to Taikichiro Mori Memorial Research Fund, which supports great financial policies to researchers like me to make progress on our ways.