Brief Description

In envisioned smart environments, enabled by future ubiquitous technologies, electronic objects will be able to interconnect and interoperate. How will users make sense of the connections that are made and the information that is exchanged? This Internet of Things could have a life of its own, exchanging digital concepts and values between its members, having an understanding of each other and communicating in their own language. Will it be possible to represent this digital world in the physical reality we live in, providing handles to control and clues to understand, build conceptual models of what is happening in this hidden reality?

Exchanging values between different realities can be viewed in the widest sense. Not only in computer games and other virtual communities we can cross boundaries, also in daily life we are often interfacing with another reality, like the digital reality in many electronic products. As the products and our environments become smarter and more complex, these connections between physical and digital reality are becoming increasingly complex and problematic. Often, we cannot make sense of what is happening in the digital world anymore. As industrial designers it is our job to make sense of this hidden digital world.

Things make sense to users in different ways, by use and functioning, by appearing in language, human communication and social use. If we look at meaning from an Internet of Things point-of-view, physical connections between artefacts and conceptual and metaphorical connections play an important role. Artefacts can be physically connected by wires or wireless communication, but people also tend to group artefacts that are not physically connected together by finding resemblances in their meaning. In smart environments with many interconnected and interoperable objects – hiding their physical connections – these conceptual and metaphorical connections become even more valuable, and maybe even crucial for the understanding of a smart environment.

Traditional design methods and theories might have their shortcomings to deal with these questions. Perhaps we could build on, or borrow concepts from theories and fields of knowledge like product semantics, dealing with meaning, User Conceptual Models (UCMís) and semiotics. We may be able to realize solutions using technologies like the Semantic Web and ontologies (representations of domain specific knowledge), making it possible to describe the relationships between the various entities in the physical and digital world.

Objectives / Specifics

(project objectives; learning goals)
The main question this proposed project addresses is: How can we represent the digital world in the physical world? How can we exchange concepts and values between these worlds?

During the course of the project, the following objectives should be addressed and explored:

– What current devices are currently common in the average home, what connections exist amongst these devices and what do they mean?
– What are the relationships between users and these devices and between the devices themselves?
– How do we currently interact with them, what will be possible/desired in the future.
– How can we use the mechanisms developed in the SOFIA project? (see Information Sources)

Deliverables

Report, demonstrator/working prototype.

Information Sources

SOFIA:Smart Objects For Intelligent Applications
For this project there is the opportunity to work in the context of a European project called SOFIA. The overall goal of this project is to connect the physical world with the information world by enabling and maintaining interoperability between electronic systems and devices. The ID department is involved in the project, contributing by developing smart applications for the smart home environment and developing novel ways of user interaction. In the project there is the possibility to work with two PhD candidates working on the project and with an external client, which has expertise in the field of multimodal interaction and has developed several interaction devices.
http://www.sofia-project.eu
http://www.sofia.id.tue.nl/wiki

Client Organisation: CONANTE
CONANTE is a German/Dutch organisation, founded by Dr. Stefan Rapp. He started CONANTE Advanced Interface Solutions to bring innovative user interface concepts from the research labs to everyday use of appliances at home and work. Prior to that, he initiated and led research projects for Sony as a Principal Scientist, among these Sony internal projects as well as collaborations on a national and European level. He established research in the area of multimodal interfaces and automatic speech recognition of European languages. His work contributed to the speech interface of entertainment robots and game consoles. Also, he was concerned with ontology-based dynamic configuration of user interfaces from heterogeneous infrastructures. He has a MSc. Degree in computer science and received his PhD from the Institute of Natural Language Processing (IMS), Stuttgart University, for work on prosody and speech recognition technology.
Interaction techniques like CONANTEís spotlight navigation can be interesting to look at during the project. It can allow users to interface with objects that normally do not have an interface of their own or to reveal normally invisible connections and resource flows.

Spotlight navigation
As the size of technological devices continues to decrease, problems arise with reading from small screens and interacting with small devices using small keys or other input means. The display and keyboard sizes dominate the current form factor of the majority of small devices. One possibility to overcome these restrictions is to use projection to an external surface. Recent advances in projection technology allow for projectors that can easily fit into a user’s hand. By combining mobile projection with natural pointing gesture, Spotlight Navigation allows efficient and easy access to large data spaces. Operation is as easy as using a flashlight: By moving the light cone over the data and zooming in to areas of interest, users manipulate information instantly and intuitively.
(source: http://www.conante.com)

Further Reading:
Feijs, L. (2009). Commutative product semantics. Design and semantics of form and movement, 12-19.
Feijs, L., & Kyffin, S. (2005). A taxonomy of semantic design knowledge. Design and semantics of form and movement, 70-83.
Jepsen, T. C. (2009). Just What Is an Ontology, Anyway? IT Professional, 11(5), 22-27.
Krippendorff, K. (2006). the semantic turn: a new foundation for design. Boca Raton: CRC Press.