This research proposes a Brain Computer Interface as an interactive and intelligent Image Search and Retrieval tool that allows users, disabled or otherwise to browse and search for images using brain signals. The proposed BCI system implements decoding the brain state by using a non-invasive electroencephalography (EEG) signals, in combination with machine learning, artificial intelligence and automatic content and similarity analysis of images. The user can spell search queries using a mental typewriter (Hex-O-Speller), and the resulting images from the web search are shown to the user as a Rapid Serial Visual Presentations (RSVP). For each image shown, the EEG response is used by the system to recognize the user's interests and narrow down the search results. In addition, it also adds more descriptive terms to the search query, and retrieves more specific image search results and repeats the process. As a proof of concept, a prototype system was designed and implemented to test the navigation through the interface and the Hex-o-Speller using an event-related potential(ERP) detection and classification system.
A comparison of different feature extraction methods and classifiers is done to study the detection of event related potentials on a standard data set. The results and challenges faced were noted and analyzed. It elaborates the implementation of the data collection system for the Brain Computer Interface and discusses the recording of events during the visual stimulus and how they are used for epoching/segmenting the data collected. It also describes how the data is stored during training sessions for the BCI. Description of various visual stimuli used during training is also given. The preliminary results of the real time implementation of the prototype BCI system are measured by the number of times the user/subject was successful in navigating through the interface and spelling the search keyword 'FOX' using the mental-typewriter Hex-O-Speller. Out of ten tries the user/subject was successful six times.
Library of Congress Subject Headings
Brain-computer interfaces--Design; Information retrieval; Image processing--Digital techniques; Classification
Electrical Engineering (MS)
Department, Program, or Center
Electrical Engineering (KGCOE)
Sildomar T. Monteiro
Kumar, Shitij P., "A Brain Computer Interface for Interactive and Intelligent Image Search and Retrieval" (2014). Thesis. Rochester Institute of Technology. Accessed from
RIT – Main Campus