Intelligent Assistive Technology and Systems Lab - click to go to homepage
IATSL develops assistive technology that is adaptive, flexible, and intelligent, enabling users to participate fully in their daily lives. Learn more about our research

Visit us:

Room 438

500 University Ave.

Toronto, Canada

P 416.946.8573

F 416.946.8570


Send us mail:

160 - 500 University Ave.

Toronto, ON, M5G 1V7



email us!


Follow IATSL on Twitter


Automatic Video Analysis for Product Usability Assessment

Keywords: Video Analysis, Usability Assessment, Temporal Segmentation, Operation Difficulty, Human-Product Interaction, Computer Vision, Machine Learning

In collaboration with: The IDEA Center (University at Buffalo), Toronto Rehab Institute, and the Centre for Sight Enhancement (University of Waterloo).

Overview of Research

As the world struggles to cope with a growing elderly population, concerns over how to preserve independence are becoming increasingly acute. A major hurdle to independent living is the inability to use everyday household objects. A critical factor in preserving independence and enabling aging-in-place is therefore the ability of older adults to use everyday household objects and perform activities of daily living. Comparative usability analysis allows the selection of the most appropriate product designs for this target population. Unfortunately, usability studies are expensive and laborious to perform and the number of everyday objects is vast.

The goal of this work is to automate the process of analyzing the usability of a product through the application of state-of-the-art computer vision and machine learning techniques. Automatic assessment will replace, or ease, laborious manual analysis. Towards this end, novel video analysis technique are being developed that enable temporal segmentation of video containing human-product interaction and automatically identifies time segments in which the human has any difficulties in operating the product. The approach is applied to a case study of water faucet design for the older adult population with dementia. Water faucets are chosen as a proof of concept product, as they are used multiple times daily and the ability of a person to use them is crucial in several self-care activities. This project is related to another IATSL project, The Impact of Familiarity on the Usability of Products by Older Adults, and is part of the RERC for Universal Design.

Intermediate outcomes of the automatic video analysis for processing handwashing videos include detecting the flow of water in a bathroom sink based on image and audio signal processing, and also partitioning each video along the time axis (Figure 1). Temporal segmentation allows for further processing of the time segments in which the human subject interacts with the faucet, i.e. by turning the water on or off or by adjusting the flow. The developed algorithms are capable of identifying human-product interactions in which the subject has any difficulty in operating the faucet. Comparative analysis of the frequency such difficulties for various faucet types reveals the advantage (or disadvantage) of one faucet type over another for the target population.

Image of an example image segmentationFigure 1: Temporal Segmentation Results during a Sample Handwashing Video Sequence (click on image to enlarge)

Funding Sources

National Institute on Disability and Rehabilitation Research (NIDRR)

Related Publications

Taati, B., Snoek, J. and Mihailidis, A. (accepted, 2011). Towards Aging-In-Place: Automatic Assessment of Product Usability for Older Adults with Dementia. IEEE Conference on Healthcare Informatics, Imaging, and Systems Biology (IEEE-HISB), July, San Jose, CA.

Taati, B. and Mihailidis, A. (accepted, 2011). Automatic Assessment of Product Design Usability - A Design Methodology. Rehabilitation Engineering and Assistive Technology Society of North America (RESNA), Toronto, Canada, June 5-8.

Snoek, J., Taati, B. Eskin, Y., Mihailidis, A. (June 2010). Automatic segmentation of video to aid the study of faucet usability for older adults. IEEE International Workshop on Computer Vision and Pattern Recognition (CVPR) for Human Communicative Behaviour Analysis, San Francisco, CA.

Taati, B., Snoek, J., Giesbrecht, D., Mihailidis, A. (June 2010). Water flow detection in a handwashing task. Proceedings of the 7th Canadian Conference on Computer and Robot vision, Ottawa, ON.

Research Team

Babak Taati (University of Toronto)

Jasper Snoek (University of Toronto)

Yulia Eskin (University of Toronto)

Alex Mihailidis (University of Toronto)