User-Centered Development and Evaluation of an Energy Information System: Incremental System Instantiations using Design Science and Multi-Method Evaluation

Publikation: Beitrag in Fachzeitschrift › Forschungsartikel › Beigetragen › Begutachtung

Abstract

Smart meter rollout activities worldwide generate high-resolution energy consumption data that has so far been underutilized to support private households in measuring their individual energy consumption and improving efficiency. This paper presents the development and evaluation of a user-centered Energy Information System (EIS) for private customers, following the Design Science Research (DSR) paradigm. Based on 119 user-identified requirements, 5 Design Principles, and 17 Design Features, a web probing study (n=35) validated individual UI elements in a first formative step. Afterwards three incremental system variants were built and evaluated in a controlled between-subjects experiment (n = 46). Task-based metrics (Task Success Rate, Efficiency), the Technology Acceptance Model (TAM), and the System Usability Scale (SUS) were applied. Looking at the results, the objective performance metrics (TSR and Efficiency) show significant differences between the instantiation variants. Interestingly, TAM and SUS do not replicate these subjectively, their scores are clustered close together across the instantiations. All variants achieve acceptable usability scores, and Actual Use shows a large effect favoring the feature-richer variants (d=0.87). This two-stage approach is methodologically relevant and enables the precise identification of potential feature-specific understanding and acceptance barriers, before they are integrated into a full system prototype. The study highlights the need for multi-method evaluation and provides recommendations for stepwise implementation, market launch, or revision of existing Energy Information Systems.

Details

OriginalspracheEnglisch
Seiten (von - bis)94176 - 94192
Seitenumfang17
FachzeitschriftIEEE access
Jahrgang14
Frühes Online-Datum18 Juni 2026
PublikationsstatusVeröffentlicht - 2026
Peer-Review-StatusJa

Externe IDs

Scopus 105043149446

Schlagworte

Forschungsprofillinien der TU Dresden

  • Informationstechnologien und Mikroelektronik - Between-subjects experiment, Design methodology, Energy, Feedback, Licenses, Nuclear facility regulation, Smart meters, System Usability Scale, Technology Acceptance Model, Technology acceptance model, Timing, Usability, Weighted sum model, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, task success rate, usability evaluation, user acceptance, Between-subjects experiment, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, system usability scale, task success rate, technology acceptance model, usability evaluation, user acceptance

DFG-Fachsystematik nach Fachkollegium

  • Betriebswirtschaftslehre - Between-subjects experiment, Design methodology, Energy, Feedback, Licenses, Nuclear facility regulation, Smart meters, System Usability Scale, Technology Acceptance Model, Technology acceptance model, Timing, Usability, Weighted sum model, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, task success rate, usability evaluation, user acceptance, Between-subjects experiment, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, system usability scale, task success rate, technology acceptance model, usability evaluation, user acceptance

Fächergruppen, Lehr- und Forschungsbereiche, Fachgebiete nach Destatis

  • Wirtschaftsinformatik (für Wirtschaftsingenieure) - Between-subjects experiment, Design methodology, Energy, Feedback, Licenses, Nuclear facility regulation, Smart meters, System Usability Scale, Technology Acceptance Model, Technology acceptance model, Timing, Usability, Weighted sum model, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, task success rate, usability evaluation, user acceptance, Between-subjects experiment, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, system usability scale, task success rate, technology acceptance model, usability evaluation, user acceptance
  • Wirtschaftsinformatik (für Wirtschaftswissenschaften) - Between-subjects experiment, Design methodology, Energy, Feedback, Licenses, Nuclear facility regulation, Smart meters, System Usability Scale, Technology Acceptance Model, Technology acceptance model, Timing, Usability, Weighted sum model, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, task success rate, usability evaluation, user acceptance, Between-subjects experiment, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, system usability scale, task success rate, technology acceptance model, usability evaluation, user acceptance
  • Wirtschaftsinformatik (für Informatiker) - Between-subjects experiment, Design methodology, Energy, Feedback, Licenses, Nuclear facility regulation, Smart meters, System Usability Scale, Technology Acceptance Model, Technology acceptance model, Timing, Usability, Weighted sum model, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, task success rate, usability evaluation, user acceptance, Between-subjects experiment, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, system usability scale, task success rate, technology acceptance model, usability evaluation, user acceptance

Ziele für nachhaltige Entwicklung

  • SDG 8 – Anständige Arbeitsbedingungen und wirtschaftliches Wachstum - Between-subjects experiment, Design methodology, Energy, Feedback, Licenses, Nuclear facility regulation, Smart meters, System Usability Scale, Technology Acceptance Model, Technology acceptance model, Timing, Usability, Weighted sum model, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, task success rate, usability evaluation, user acceptance, Between-subjects experiment, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, system usability scale, task success rate, technology acceptance model, usability evaluation, user acceptance
  • SDG 9 – Industrie, Innovation und Infrastruktur - Between-subjects experiment, Design methodology, Energy, Feedback, Licenses, Nuclear facility regulation, Smart meters, System Usability Scale, Technology Acceptance Model, Technology acceptance model, Timing, Usability, Weighted sum model, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, task success rate, usability evaluation, user acceptance, Between-subjects experiment, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, system usability scale, task success rate, technology acceptance model, usability evaluation, user acceptance
  • SDG 7 – Erschwingliche und saubere Energie - Between-subjects experiment, Design methodology, Energy, Feedback, Licenses, Nuclear facility regulation, Smart meters, System Usability Scale, Technology Acceptance Model, Technology acceptance model, Timing, Usability, Weighted sum model, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, task success rate, usability evaluation, user acceptance, Between-subjects experiment, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, system usability scale, task success rate, technology acceptance model, usability evaluation, user acceptance

ASJC Scopus Sachgebiete

  • Allgemeine Computerwissenschaft - Between-subjects experiment, Design methodology, Energy, Feedback, Licenses, Nuclear facility regulation, Smart meters, System Usability Scale, Technology Acceptance Model, Technology acceptance model, Timing, Usability, Weighted sum model, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, task success rate, usability evaluation, user acceptance, Between-subjects experiment, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, system usability scale, task success rate, technology acceptance model, usability evaluation, user acceptance
  • Allgemeine Materialwissenschaften - Between-subjects experiment, Design methodology, Energy, Feedback, Licenses, Nuclear facility regulation, Smart meters, System Usability Scale, Technology Acceptance Model, Technology acceptance model, Timing, Usability, Weighted sum model, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, task success rate, usability evaluation, user acceptance, Between-subjects experiment, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, system usability scale, task success rate, technology acceptance model, usability evaluation, user acceptance
  • Allgemeiner Maschinenbau - Between-subjects experiment, Design methodology, Energy, Feedback, Licenses, Nuclear facility regulation, Smart meters, System Usability Scale, Technology Acceptance Model, Technology acceptance model, Timing, Usability, Weighted sum model, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, task success rate, usability evaluation, user acceptance, Between-subjects experiment, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, system usability scale, task success rate, technology acceptance model, usability evaluation, user acceptance

Schlagwörter

  • Between-subjects experiment, Design methodology, Energy, Feedback, Licenses, Nuclear facility regulation, Smart meters, System Usability Scale, Technology Acceptance Model, Technology acceptance model, Timing, Usability, Weighted sum model, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, task success rate, usability evaluation, user acceptance, Between-subjects experiment, decision support systems, design science research, energy conservation, energy feedback systems, energy information systems, human-computer interaction, information systems design, smart energy management, smart metering, system usability scale, task success rate, technology acceptance model, usability evaluation, user acceptance