University of Padova

Enel Meters Usability

Project Description

Smart meters are a powerful tool that could enable households in reducing and having an overall control over energy consumption. The aim of this study is to investigate the usability of the Italian Enel Smart Meters. A field research was conducted with consumers who were requested to obtain information from the meter trough some tasks, complete a satisfaction questionnaire and a structured interview. The specific objectives of the study are:

– to find out if the information that the meter presents is understood by the users
– what prevents the users from successfully completing the task
– the users opinion and the overall evaluation of the meter

Enel meter Participant with the Enel meter
Work Description

A convenience sampling was used to select 40 Italian native speakers, who were responsible in paying their bills and didn’t have any expertise in electronics or engineering. The age of the participants ranged in between 23 and 85 with an average of M=47.2 (SD=18.8). The participants were divided into two groups: one group had to read the Enel’s meter manual before the experiment, and the other couldn’t. The participants were interviewed in their house and where requested to complete seven tasks aimed to investigate the comprehension of the meters feedback. The order of presentation of the tasks was randomized with the Latin square procedure eliminating learning effects. After the completion of the tasks a satisfaction questionnaire adapted from the Microsoft Desiderability Toolkit was administered to the participants with the aim of investigating the user’s opinion on the meter. A post structured interview was conducted on the answers from the satisfaction questionnaire in order to explore more in depth the motivation and opinions of the participants on selecting their answers. The tasks and the interviews were video recorded and the participants behaviors and verbalizations were coded and analyzed with the software Noldus the Observer.

The research activities will consist of surveys of passing traffic and the use of appropriate test samples, specifically identified and corresponding to the research requirements. The speed surveys will be conducted in a continuous cycle using laser traffic-counting equipment, with the aim of acquiring cumulative information referring both to the trend of the surveyed parameters during the day (according to the time of day) and during the week (according to the day of the week).

Participant with the Enel meter Video Analysis Noldus The observer
Results

69% of the tasks assigned to the participants were failed or abandoned. The participants had the most difficulties in understanding if the system had a failure and obtaining the energy consumed from the last electricity bill till the moment of the interview. Reading the manual didn’t help to complete more tasks because there is no statistically significant difference in between the two groups for the overall completion of the tasks. Furthermore we found no correlation in between age and completion of the tasks, suggesting that young and older participants perform alike. The information the meter gives also presented some problems because the participants made on average one interpretation mistake per task (SD=7.2). The most common mistakes in this category ranged from confusing a value for another to not perceiving the feedback relevant for the completion of the task. The affordances of the meter were problematic too because on average 2.2 mistakes of interaction were made per task, mostly due to the participants clicking the only navigation button when it wasn’t necessary for the completion of the task. On average 27% of the participants had to stay in an uncomfortable position while performing the tasks; and 30% had to move their head in order to read better the display. Both values denote problems in the visibility of the information. The efficiency with which the participants completed the tasks is also compromised because it took on average 6.8 clicks in excess (SD=9.4) and 46 seconds (SD=32) to complete the tasks with success. More than half of the participants couldn’t remember and explain how to complete a task completed with success when asked by the experimenter. On average every participant asked for help on the execution of the task or requested to abandon the task with an occurrence of M=0.95 (SD=1.44). Regarding the users satisfaction 37.5% of the participants had a negative opinion of the meter.

Result's graph


© 2019 Università di Padova - Tutti i diritti riservati