Open User Interface Standards - Towards Coherent, Task-Oriented And Scalabe User Interfaces In Home Environments

Gottfried Zimmermann
Access Technologies Group
Wilhelm-Blos-Str. 8, 72793 Pfullingen, Germany

This article has been published as short paper in: Proceedings of 3rd IET International Conference on Intelligent Environments (IE07), Sep. 24-25, 2007, Ulm University, Germany, p. 36ff. The IET, 2007.

Table of Contents:


This paper looks at some existing products for remote user interfaces for home appliances, with regard to their coherence, task orientation and scalability. Although some positive trends can be observed, these systems are based on proprietary, closed technologies. The Universal Remote Console framework (ISO/IEC 24752) and the Task Model Description standard (CEA-2018) are described as possible solutions for providing full access to home appliances for everybody, in any modality and interaction style, in a coherent, task oriented and scalable fashion.


Smart environments, remote user interface, task-based user interface, Universal Remote Console (URC), Task Model Description

1 Introduction

Researchers have explored home control and smart environments and their potentials for years. Despite of the perceived success in the form of prototypical systems and show homes, there has been relatively little impact on the mainstream market and personal homes of the consumers.

In this paper, we will look at the current market situation on remote user interfaces for the home environment. Hereby we will focus on three requirements that such user interfaces and their systems should meet: coherence, task-orientation and scalability. We will explain these characteristics later.

We will introduce two open standardization efforts that are currently underway and that have the potential to deliver a new generation of remote user interfaces to a broad customer base, while facilitating the desired characteristics.

2 Requirements on Remote User Interfaces for Home Environments

2.1 Coherence

Remote user interfaces should be coherent, that means they should allow for seamless control across the many devices in the home environment. This includes consumer electronics devices, appliances in a household such as stoves and washing machines, and other systems that are part of a home such as lighting, heating, air conditioning and security.

For example, the user might want to watch a movie and dim the lights at the same time. They should be able to do so with as little overhead for switching between the functionality of the DVD player, TV, sound system and the lights. Similarly, if the user wants to record a TV program, programming of the DVR should be available from the EPG.

2.2 Task Orientation

Remote user interfaces should be task-oriented or task-based, in that they should expose what the user can do with an integrated set of devices rather than how this will be achieved.

For example, the task "watch a movie" might involve the following steps: Switch on the DVR, switch on the TV and select the DVR as input source, switch on the receiver and select the DVR as input source, select a movie on the DVR, and start "play" on the DVR. Most users are not interested in seeing and controlling the individual steps of this task (except for selecting the movie which is a variable in this task).

2.3 Scalability

Remote user interfaces should be scalable so that a user can have an initial, but functional, system with only a small number of devices. When subsequently adding devices to the home environment, the remote user interface should present additional tasks that are made possible by the new devices.

Thus a home environment with many devices could incrementally evolve over years [8]. The user should not need a new controller for controlling a new device, and should not have to buy a new set of devices when transitioning to a different controller.

3 Current Market Situation

A thorough market review is beyond the scope of this paper. Instead, we focus mainly on three types of systems for remote user interfaces that are currently available on the market for home control, and how these types relate to the three requirements of coherence, task orientation, and scalability.

3.1 Infra-red remote controls

Traditional remote user interfaces are dedicated to one particular target. Infrared-based remote controls shipped with and bound to specific target devices are prevalent in today's homes. This results in a large number of remote controls in the digital home, a real usability problem with a growing number of target devices in the average home.

Clearly, traditional infra-red controls do not provide for user interfaces that are reasonably consistent, task oriented and scalable.

"Universal remote controls" are an improvement because they allow for controlling a set of devices with one controller only. This makes for more coherent user interfaces, and enables task-oriented features that span multiple devices. However, universal remote controls typically come with an increase on complexity, especially if user programming is required. Scalability is not guaranteed unless the user interface for the new device can be downloaded from the database of the universal remote control manufacturer.

3.2 Custom installation homes

Recently, custom installation homes have been equipped with integrated and sometimes task-oriented user interfaces for consumer electronics products and appliances at home. The drawback of custom installation is that the design of custom-made user interface is a cumbersome process requiring special skills. It is not really scalable since every time a new device is added, the user interface needs to be revised manually.

Custom installation is becoming popular among people who can afford high-end home electronics, in particular in the United States. This is a niche market, offering products that are out of reach for most people because of the expensiveness of the manual customization process that must be repeated for every home installation.

3.3 Windows Media Center

Microsoft's approach for controlling the home environment is the PC with Media Center (MC) [4], a special user interface for a TV screen facilitating remote control of the integrated entertainment system. MC comes with an infra-red remote and software libraries for third parties to integrate their devices and user interfaces.

By default, MC integrates user interfaces for devices and functions of a multimedia computer, including a TV tuner, DVR, video on demand, radio, MP3 player, sound system and slide show. For example, one can instruct the DVR to record a program directly from the EPG listing. User interfaces for other home appliances such as lighting, heating and security, can be added by third parties. For all devices, the user is in control through a TV screen and one remote.

MC is an example of a coherent, mostly task-oriented and scalable remote user interface. However, MC is constrained to the PC as controller platform, rendering graphical user interfaces only. It does not directly support remote user interfaces running on other platforms, possibly involving other modalities such as voice or gesture.

4 New User Interface Standards

An open approach for remote user interfaces is needed that would facilitate a diversity of remote user interfaces, contributed by device manufacturers, controller manufacturers and third parties. This approach should support the characteristics of coherence, task orientation and scalability.

4.1 Universal Remote Console framework

The Universal Remote Console (URC) framework is currently being finalized as the international standard ISO/IEC 24752 [3]. As its core, it specifies a "User Interface Socket" (or short "Socket") that is exposed by any device that can be remotely controlled. Controllers (called "Universal Remote Consoles", short URCs) can connect to this Socket, using any kind of user interface that "plugs into the Socket". Controllers can look for and retrieve pluggable user interfaces (called "User Interface Implementation Descriptions", short "UIIDs") from "Resource Servers" on the Internet. Figure 1 in the appendix illustrates the conceptual model of the URC framework.

Pluggable user interfaces are based on one or multiple User Interface Sockets, but are free to use any modality and user interaction mechanism they want. They can be provided by the manufacturer of a device, by the manufacturer of a particular controller, by companies that specialize in user interface design, or even by user groups.

Coherence of user interfaces is supported by the URC framework in two ways: First, every device describes its "abstract user interface" in the same way, enabling the use of one controller for all devices. Second, pluggable user interfaces can span multiple Sockets (and thus multiple devices), and so one user interface can be built to include the functionality of a whole set of devices.

Task orientation is supported by the URC framework to some degree, since the User Interface Socket can be an abstraction over the more low-level control protocol that the device employs. However, we will see in the following subsection that the combination of URC and a Task Model Description can bring about the full power of task-oriented user interfaces.

Scalability is enabled by the URC framework since every device exposes a User Interface Socket that any third party can design a pluggable user interface for. The collection of available pluggable user interfaces is maintained by Resource Servers. Controllers can download suitable pluggable user interfaces from these Resource Servers.

The URC framework does not specify the wire protocol between a controller and a controlled device. Instead, it must be implemented on top of one or more existing networking platforms that provide the necessary functionality for device discovery, control and eventing. Thus the URC framework is independent of any particular networking platform.

For the home environment, the "Universal Control Hub" architecture [7][9] represents a solution for the URC framework to be implemented even if the controlled devices and controllers are not URC compatible.

An implementation of the Universal Control Hub architecture has been developed at the Trace R&D Center, University of Wisconsin-Madison, USA. A recent prototype demonstrates how a variety of mobile devices can be used to remotely control a UPnP-based entertainment system [6].

The Universal Control Hub is also being applied in the "i2home" project [2], a project funded by the European Research programme IST. The i2home consortium consists of organizations and companies from Germany, Sweden, the Czech Republic, Spain and Portugal.

Organizations interested in the promotion and implementation of the URC standards have formed the URC Consortium (URCC) that provides more information on current projects and tools [5].

4.2 CEA-2018: Task Model Description

An XML-based language for the description of task models is currently being developed by the Working Group 12 of the CEA committee on home networks (CEA R7 WG12 [1]), and will be released as standard CEA-2018.

In CEA-2018, tasks can be defined in terms of subtasks (steps), and the atomic tasks can be "grounded" to actual device functions. It is possible to map tasks to the elements of a User Interface Socket. This combination of task models and User Interface Sockets has several benefits:

  1. The task layer on top of the Socket layer provides an explicit model for defining tasks for a particular combination of devices. Task engines that use this description interact with their users in terms of application-specific tasks. For example, a user may be presented with a list of possible tasks at any time, or the system may assist the user in performing a specific task step by step.
  2. The User Interface Socket can be used as a common model for conventional user interfaces (e.g. graphical or voice user interfaces) and a task engine. Thus, the user may either use the task engine to operate the devices, or a graphical/voice user interface, or both in any sequence. At any time, the current status of the devices will be reflected in both user interfaces.
  3. The middle layer of User Interface Sockets represents a convenient grounding platform for task models. The grounding to the typically more complex operations of the device interface layer can be delegated to the User Interface Socket, which can use procedural code to map to the actual device operations.

5 Conclusion

In this paper we have described three types of currently available products for home control. Among them, custom installation homes, meet most of the requirements of coherence, task orientation and scalability. Windows Media Center based home control can only partially fulfil them, and infra-red remote controls fall completely short of them. Today, existing products are either very expensive, proprietary or constrained to a particular platform. There is a need for a more open approach of user interface design that involves third-party user interface designers and task modellers.

The Universal Remote Console framework (ISO/IEC 24752) represents an open framework, facilitating any controller controlling any device through User Interface Sockets and pluggable user interfaces. In tandem with an emerging Task Model Description standard (CEA-2018), this approach allows for coherent, task-oriented and scalable user interfaces for home environments.

In the near future, we hope, products based on these open standards will deliver full access to electronic devices and appliances at home for everybody, in any modality and interaction style.


This work was funded by the US Dept of Education, NIDRR, under Grant H133E030012 (RERC on IT Access); and by the EU 6th Framework Program under grant FP6-033502 (i2home). The opinions herein are those of the author and not necessarily those of the funding agencies.


[1] CEA R7 WG12, Task-Based User Interface.

[2] Intuitive Interaction for Everyone with Home Appliances based on Industry Standards (i2home).

[3] ISO/IEC FCD 24752. Information technology - User interfaces - Universal remote console - 5 parts. International Organization for Standardization (ISO), 2006.

[4] Microsoft XP Media Center Edition 2005 Home Page,

[5] URC Consortium.

[6] URC Consortium, Tools and Prototype Implementations for the URC Framework.

[7] Zimmermann, G.; Vanderheiden, G.; Rich, C. (2006). Universal Control Hub & Task-Based User Interfaces. URC Consortium, 2006.

[8] Vanderheiden, G., and Zimmermann, G. (2007, Jul). Non-homogenous Network, Control Hub and Smart controller (NCS) Approach to Incremental Smart Homes. Proceedings of HCI International 2007, July 22-27, 2007, Beijing, China.

[9] Zimmermann, G., and Vanderheiden, G. (2007, Jul). The Universal Control Hub - An Open Platform for User Interfaces in the Digital Home. Proceedings of HCI International 2007, July 22-27, 2007, Beijing, China.

Appendix: Components of the URC Framework

Figure 1: Conceptual structure and components of the URC framework

Figure 1: Conceptual structure and components of the URC framework. It illustrates both elements for which there are formats specified by the URC framework, and elements that can use these or other standard or proprietary formats. Technical details for each component are provided in [3], part 1.