INSITE: A realtime, interactive telepresence system for virtual fieldtrips, distance learning, and cross-cultural interaction

Scott S. Fisher

Graduate School of Media & Governance

Keio University at Shonan Fujisawa

1. Objective

The objective of this research is to extend the capabilities of current telepresence technology with recent developments in Internet transmission of realtime audio and video data, and to explore and evaluate the use of these technologies for educational applications. A long-term goal will be to further refine the technology platform and to determine the unique advantages and opportunities that technologies of presence can provide as learning technologies.

2. Background and Rationale

In the past decade, a number of educational programs and commercial products have been developed using traditional broadband videoconference technologies. These systems allow limited interaction between students in the classroom and students and teachers at other classroom sites, or enable students to observe activities at remotely located scientific research sites. More recently, a number of live camera sites (webcams) have become available through the internet that allow WWW users around the world to access low frame rate/low resolution images from a static camera capturing remote educational sites such as zoos, remote landscapes, and historic locations.

But, in general these systems provide only one viewpoint into a remote space and do not provide the ability to literally explore and interact with activities at the remote site from a first person point of view. As a result, they offer a very limited sense of presence and immersion in a remote environment. The development of a realtime telepresence webcam system could enable a more immediate and personalized experience for learning about other cultures, and for individually participating in ‘virtual fieldtrips. By mounting the camera system on another person at the remote site, users could direct camera positioning to experience an event as if they were there. Or alternatively, they could experience a remote location through the viewpoint of another person with different cultural interests, insights, and opinions.

3. Approach

This project will develop a realtime telepresence webcam system and evaluate its use in a number of educational application scenarios. To test this concept, the initial technology platform will consist of a very lightweight stereoscopic camera system with wireless audio/video transmission that can be mounted on a remote users head and body. Data transmitted from the remote site will be experienced through a range of display technologies including HMD and stereoscopic desktop monitor or projection systems. Server software will be developed or adapted as needed to provide the interface between user and remote camera operator, and to provide additional capability for non-interactive viewers through a standard WWW display.

This system will be evaluated in scenarios such as the following:

- The camera system will transmit the realtime point of view of an artist as they engage in their craft. Users of the system can interact with the subject by asking questions about their actions or requesting another viewpoint of a current activity. For example, students might experience the making of an artwork through the eyes and ears of an artist, composer, or performing artist as they develop their work.

- Students from different cultures will exchange first person “guided-tours of events in their daily life in remote locations. Culture differences can be pointed out and discussed interactively.

- Other domain specialists will give interactive POV tours of inaccessible locations or special events. For example, several units could be developed to allow remote users to “virtually attend a special event taking place on the Keio SFC campus and view the activities through the eyes of different guides.

4. Project Status for FY 2000

4.1 Hardware System

A first prototype of the head-mounted remote camera system has been developed and field-tested on the SFC campus using analog wireless video technology to transmit images from the user’s point of view to a video projector in a nearby office space (Figure 1). In addition, computer server hardware and digital video software has been procured, setup, and tested in preparation for system testing over the Internet to enable interaction over longer distances within Japan and over Internet2 connections to the US.

4.2 Installation Site Development

Discussions have been initiated with a number of institutions in the U.S. to install the server system between their locations and the SFC campus. The first installation is expected to be completed in April, 2001 on the campus of University of California Berkeley, and testing is anticipated to begin in May, 2001.