Header And Logo

• Home
• Research
• People
• Publications
• Teaching
• Links
• Contact
• Imprint

Publications

2012	Back to Top

Sustainable Instrumentation of Everyday Commodities - Concepts and Tools
Boris Brandherm, Alexander Kröner, Jens Haupert, Michael Schmitz, Frank Lehmann and Ralf Gampfer
10th Annual IEEE International Conference on Pervasive Computing and Communications. IEEE International Conference on Pervasive Computing and Communications (PerCom-12), 10th, March 19-23, Lugano, Switzerland
[BibTeX]

Nachhaltiges Konzept zur Förderung der Therapietreue [Sustainable Concept for Increasing Compliance
Boris Brandherm, Alexander Kröner, Michael Schmitz, Jens Haupert, Frank Lehmann and Ralf Gampfer
Demographischer Wandel - Assistenzsysteme aus der Forschung in den Markt. Deutscher AAL-Kongress (AAL-12), 5. Deutscher AAL-Kongress mit Ausstellung, January 23-25, Berlin, Germany
[BibTeX]

2011	Back to Top

IUI 2011 Workshop on Location Awareness for Mixed and Dual Reality ( LAMDa )
Gerrit Kahl, Boris Brandherm, Tim Schwartz, Andreas Forsblom, Eyal Dim and Petteri Nurmi
2011 International Conference on Intelligent User Interfaces

"Geremin'': 2D Microgestures for Drivers Based on Electric Field Sensing
Christoph Endres and Tim Schwartz and Christian Müller
Proceedings of the International Conference on Intelligent User Interfaces (IUI 2011)

Patientenindividuelle Förderung der Therapietreue durch intelligente Medikamentenverpackungen
Boris Brandherm, Alexander Kröner, Michael Schneider, Jens Haupert and Michael Schmitz
Demographischer Wandel - Assistenzsysteme aus der Forschung in den Markt. Deutscher AAL-Kongress (AAL), 4. Deutscher AAL-Kongress mit Ausstellung, January 25-26, Berlin, Germany
[BibTeX]

Emergency Indoor and Outdoor Localization
Michael Kruppa
Proceedings of the 4. Deutscher AAL-Kongress 2011
[ DOWNLOAD NOTE ] show abstract
Within the context of the automatic detection of a medical emergency and the aim of providing medical care as fast as possible, it is of utmost importance to detect the current position of the patient. The more precisely the patient location can be determined and the more effectively the paramedics are guided towards the patients current location, the less time is wasted prior to the initial medical care provided to the patient after the emergency has occurred. This paper describes a first, prototypical realization of a combined indoor and outdoor user localization component developed within the Smart Senior project. Apart from a detailed description of the technical background of the realized component, an important aspect discussed in this paper is the distinction between the two different concepts of “locating a user” and “finding a user”.

Concepts for Life-Like Interactive Objects
Michael Schmitz
Proceedings of the fifth international conference on Tangible, Embedded, and Embodied Interaction (TEI '11). ACM, New York, NY, USA, 157-164
[BibTeX] [ACM ref]

Synchronized Realities
Christoph Stahl, Jochen Frey, Jan Alexandersson, Boris Brandherm
Journal of Ambient Intelligence and Smart Environments, Vol. 3, Nr. 1/2011, pp. 13-25, IOS Press.
show abstract
Ambient Intelligence (AmI) promises future environments that support the well-being of their inhabitants through comfort and assistive environments. For the elderly users, Ambient Assisted Living (AAL) scenarios are targeting safe and independent living at home. Especially the integration of home automation components, such as intelligent light, window, and climate controls seem promising for such scenarios. In this paper, we present a new development method that is based on a detailed three-dimensional model and show how the model can be used for designing and evaluating assistive environments. Such a system has been developed and deployed. We present an architectural foundation of how Universal Remote Console technology can be applied to synchronize the virtual environment model with the real world. Furthermore, we extend previous work on Dual Reality by introducing the concept of Synchronized Realities that includes the synchronization between remote AmI environments. We will discuss application scenarios and show two demonstrators that we have implemented based on the proposed development method.

2010	Back to Top

The DFKI Competence Center for Ambient Assisted Living
Jochen Frey, Christoph Stahl, Thomas Röfer, Bernd Krieg-Brückner, and Jan Alexandersson
Ambient Intelligence 2010
show abstract
The DFKI Competence Center for Ambient Assisted Living (CCAAL) is a cross-project and cross-department virtual organization within the German Research Center for Artificial Intelligence coordinating and conducting research and development in the area of Ambient Assisted Living (AAL). Our demonstrators range from multimodal speech dialog systems to fully instrumented environments allowing the development of intelligent assistant systems, for instance an autonomous wheelchair, or the recognition and processing of everyday activities in a smart home. These innovative technologies are then tested, evaluated, and demonstrated in DFKI’s living labs.

UbiSpot – A User Trained Always Best Positioned Engine for Mobile Phones
Tim Schwartz, Christoph Stahl, Christian Müller, Hao Ji, Valentin Dimitrov
Proceedings of Ubiquitous Positioning Indoor Navigation and Location Based Service (UPINLBS 2010), IEEE, ISBN 978-1-4244-7878-1
show abstract
We implemented a positioning engine for mobile phones that can be trained by the users to recognize places as personal landmarks by their wireless communication fingerprint. Our always-best-positioned approach integrates heterogeneous sensor data, such as Bluetooth (BT) device addresses, WLAN MACs, GSM cell ids and GPS coordinates, if available. As an alternative to measuring the signal strength of wireless access points, our positioning engine measures the relative frequency of their appearance and disappearance over time, which closely correlates to their distance. The user can add new places as symbolic names to a hierarchical location model at any time using their mobile phone. For each place, the wireless sensor fingerprint can be trained by the user to define a landmark. Once landmarks have been trained, the positioning engine continuously matches the current sensor profile against the database of learned fingerprints and chooses the most likely place. In case that no BT or WLAN APs are visible, the hierarchical data model can at least derive a higher-level description of the current region based on GSM or GPS as fallback strategy in the sense of being always best positioned. We evaluated the positioning accuracy in our university’s lab environment in terms of hits and misses and investigated the effect of various time window sizes for the frequency measurement of the fingerprint. The symbolic location model can be applied for example to adapt the mobile device to different contexts, e.g. automatically mute the ringtone in meeting rooms, trigger location-dependent rules and events, or disclose the current location to friends.

What if it Suddenly Fails? Behavioural Aspects of Advanced Driver Assistant Systems on the Example of Local Danger Alerts
Angela Mahr, Yujia Cao, Mariet Theune, Tim Schwartz and Christian Müller
19th European Conference on Artificial Intelligence (ECAI 2010)

Designing Tangible Interaction With Anthropomorphic Smart Objects
Michael Schmitz
Proceedings of the IADIS International Conference on Interfaces and Human Computer Interaction. Freiburg, Germany

A Framework for Auditory Displays in Intelligent Environments
Ralf Jung
Proceedings of International Conference on Intelligent Environments 2010, July 18- July 21, 2010, Kuala Lumpur, Malaysia
show abstract
In this paper, a novel framework for Auditory Displays is presented that provides methods to notify persons in instrumented environments individually. The used non-speech audio cues are unnoticeable for others and they have no disturbing effect on attendees. On the symbolic level, the notification signals represent events and can be seamlessly embedded in an ambient soundscape running in the background of the listener’s attention. The type of notification depends on the user’s preferences as well as on his current situation and location.

Modeling and Simulating Ambient Assisted Living Environments - A Case Study
Tim Laue and Christoph Stahl
In Proceedings: Ambient Intelligence and Future Trends-International Symposium on Ambient Intelligence (ISAmI 2010), Advances in Soft Computing, Vol.72, pp. 217-200. Berlin: Springer. Http://www.springerlink.com/content/e4247wl8g5377230/?p=06a9ebf68e744432a0d65189f6a013d5
[pdf] show abstract
Modeling the layout of an apartment -- including different furniture configurations and several moving elements -- is a non-trivial task, especially if accessibility has to be assured e.g. for wheelchairs. In this paper, we describe a two-staged example for such a task by connecting two different applications: In the first step, the Yamamoto toolkit is used to efficiently model the building structure and to plan the furnishing of the environment in 3D. Afterwards, the desired configuration becomes exported to SimRobot, a robot simulator based on rigid body dynamics. Thereby, a realistic evaluation of the physical configuration becomes possible by interactively driving an electrical wheelchair through the environment.

Konsum und Handel zwischen digitalem Marketing, gläsernem Kunden und elektronischer Produktinformation
Ralf Jung and Gesche Roy
Loccumer Protokolle: "Chancen nutzen, Risiken minimieren! Unser zukünftiger Umgang mit allgegenwärtigen Informations- und Kommunikationstechnologien" (to appear)
show abstract
Die Informations- und Kommunikationstechnologie macht rasante Fortschritte. Mittlerweile sind die Geräte in der Lage, Unmengen von Daten zu erfassen, zu speichern und zu verwenden. Wenn überall und jederzeit Daten zu unterschiedlichsten Aktivitäten erfasst werden können, wenn eine technisch vermittelte Kommunikation zu jeder Zeit und an jedem Ort möglich ist und weite Bereiche unserer Infrastruktur automatisch gesteuert werden, hat das weit reichende Konsequenzen für die Arbeits- und Lebenswelt, die auf der Tagung unter dem Stichwort „Anytime, Anywhere Communication and Computing (AACC)“ diskutiert werden.

Non-Intrusive Audio Notification in Emotion Classified Background Music
Ralf Jung
Proceedings of International Conference on Acoustics 2010, April 19- 23, 2010, Baltimore, USA

Roles and Rights Management Concept With Identification by Electronic Identity Card
Boris Brandherm, Jens Haupert, Alexander Kröner, Michael Schmitz, Frank Lehmann
8th Annual IEEE International Conference on Pervasive Computing and Communications. IEEE International Conference on Pervasive Computing and Communications (PerCom-2010), March 29 - April 2, Mannheim, Germany, Pages 768-771, ISBN 978-1-4244-5328-3, IEEE Computer Society, 2010.
[ DOWNLOAD NOTE ] [BibTeX] show abstract
Adherence to therapy, what is termed "compliance", is a very important topic in Medicare. It has considerable effects not only in terms of enhancing quality of life for patients, but also in reducing the financial burden on the health care system. The individualized weekly medicament blister for patients (7x4 Box) from 7x4 Pharma increases the compliance and helps to keep the health expenditures as low as possible. A potential extension of the 7x4 Box could be a digital product memory which makes the imprinted data on the box electronically accessible in order to enable new innovative services which helps to increase the compliance. If we do not secure the data from an unauthorized access there is a possibility that third parties can read out the data unperceived and on a grand scale. In this paper we show how we can realize an authorized access -- individually or role-based -- on the data via identification by the new German electronic identity card.

Demo: Authorized Access on and Interaction With Digital Product Memories
Boris Brandherm, Jens Haupert, Alexander Kröner, Michael Schmitz, Frank Lehmann
8th Annual IEEE International Conference on Pervasive Computing and Communications. IEEE International Conference on Pervasive Computing and Communications (PerCom-2010), March 29 - April 2, Mannheim, Germany, Pages 838-840, ISBN 978-1-4244-5328-3, IEEE Computer Society, 2010.
[ DOWNLOAD NOTE ] [BibTeX] show abstract
In the Project "Semantic Product Memory" we exploit the lessons learned so far for the general design of a Digital Product Memory (DPM). DPMs can be used along their product's life cycle to collect data by different instances. In case of the individualized weekly medicament blister for patients from 7x4 Pharma not only the collected data but even the product itself is private. If we equip such a product with a DPM the collected data should be accessible -individually or role-based- only by authorized persons. This authorized access can be accomplished via a roles and rights management system and a secure identification provided by the new German electronic identity card. With our demonstrator we will present a system, which allows an authorized access on a DPM. Depending on the rights and roles of a user different data and different views will be available and displayed accordingly.

Seamless Resource Adaptive Navigation
Tim Schwartz, Christoph Stahl, Jörg Baus, and Wolfgang Wahlster
In: Matthew Crocker and Joerg Siekmann (eds). Resource-Adaptive Cognitive Processes. Cognitive Technologies Series. Springer Verlag, Berlin, 2010, pp. 239-265

From Resource-adaptive Navigation Assistance to Augmented Cognition
Hubert D. Zimmer, Stefan Münzer, and Jörg Baus
In: Matthew Crocker and Joerg Siekmann (eds). Resource-Adaptive Cognitive Processes. Cognitive Technologies Series. Springer Verlag, Berlin, 2010, pp.35-53

Innovative Retail Laboratory - Investigating Future Shopping Technologies
Antonio Krüger, Lübomira Spassova and Ralf Jung
it - Information Technology, Volume 52, Issue 2, Oldenbourg Verlag, pp. 114-118
show abstract
The Innovative Retail Laboratory (IRL) is an application-oriented research laboratory of the German Research Center for Artificial Intelligence (DFKI) run in collaboration with the German retailer GLOBUS SB-Warenhaus Holding in St. Wendel. In this living lab, we conduct research in a wide range of different domains all connected to intelligent shopping consultants, which range from a virtual assistant responsible for matters of dieting and allergies, over a digital sommelier, to personalized cross and up selling, smart items with digital product memories as a further development of the RFID technology, indoor positioning and navigation as well as new logistics concepts, to see if they are suitable for everyday life and useful for customers. New ways of customer interaction are developed and tested for implementation. The range varies from personalized shopping assistants to “talking” products and intelligent shopping carts, which plan and show the way through the store according to your shopping list. Furthermore they can give advice on what to buy for the recipes you have in mind, they compare products, point out special offers in a personalized way and give additional information about the products. However, the concepts and technologies that regard the self-service store of the future as a place for shopping are not IRL´s only focus. The relation between the store and its customers begins way before the shopping trip itself takes place. It starts with an individual shopping preparation and a personalized presentation of offers at home and will be continued afterwards through advice that is given about purchased goods and information about their use.

The Shopping Experience of Tomorrow: Human-Centered and Resource-Adaptive
Wolfgang Wahlster, Michael Feld, Patrick Gebhard, Dominikus Heckmann, Ralf Jung, Michael Kruppa, Michael Schmitz, Lübomira Spassova and Rainer Wasinger
Matthew Crocker and Joerg Siekmann (eds). Resource-Adaptive Cognitive Processes. Cognitive Technologies Series. Springer Verlag Berlin, 2010, pp. 205-237

Local Danger Warnings for Drivers: The Effect of Modality and Level of Assistance on Driver Reaction
Yujia Cao, Angela Mahr, Sandro Castronovo, Mariet Theune, Christoph Stahl, and Christian Müller
IUI 2010 (to appear)
show abstract
Local danger warning is an important function of Advanced Driver Assistance Systems (ADAS) to improve the safety of driving. The user interface (the warning presentation) is particularly crucial to a successful danger avoidance. We present a user study investigating various warning presentations using a scenario of emergent road obstacles. Two presentation factors were selected: modality and level of assistance. The modality factor had 4 variants: speech warning, visual and speech warning, visual warning with blinking cue, and visual warning with sound cue. The level of assistance varied between with or without action suggestions (AS). In accordance with the ISO usability model, a total of 6 measurements were derived to assess the effectiveness and efficiency of the warnings and the drivers’ satisfaction. Results indicate that the combination of speech and visual modality leads to the best performance as well as the highest satisfaction. In contrast, purely auditory and purely visual modalities were both insufficient for presenting high-priority warnings. AS generally improved the usability of the warnings especially when they were accompanied by supporting information so that drivers could validate the suggestions.

Cinematic Analysis of Automotive Personalization
Christoph Endres, Michael Feld, Tim Schwartz and Christian Müller
Proceedings of the 2nd International Workshop on Multimodal Interfaces for Automotive Applications (MIAA 2010) in Conjunction with IUI 2010, ISBN 978-1-60558-997-8, ACM
show abstract
Personalization has become an important aspect in the design of cars and the human machine interface (HMI). Successful personalization bears the potential of increasing both safety as well as customer satisfaction. In order to reveal innovative concepts in automotive personalization, we present in this paper a cinematic study -- an analysis of what the movie industry "invented" and what might be worthwhile following up upon in research. Thereby, our notion of personalization is twofold: a) tailoring the car or parts of the car to a specific user (group) and b) the car or parts of the car becoming a persona -- the latter being relevant with respect to current automotive research activities on the "emotional" car. Based on the analysis of popular movies and TV series from the last four decades, we introduce a scheme that describes personalization concepts as imagined (and maybe even anticipated) by film-makers, who's creative drives have not been slowed down by the restraints of feasibility and costs.

Benutzer-Lokalisierung in unterschiedlichen Umgebungen
Michael Kruppa
Proceedings of the 3. Deutscher AAL-Kongress 2010
[ DOWNLOAD NOTE ] show abstract
Within the context of the automatic detection of a medical emergency and the aim of providing medical care as fast as possible, it is of utmost importance to detect the current position of the patient. The more precisely the patient location can be determined and the more effectively the paramedics are guided towards the patients current location, the less time is wasted prior to the initial medical care provided to the patient after the emergency has occurred. This paper describes the user location component as it will be realised within the context of the Smart-Senior project. What makes this concept unique is the fact that it supports a large variety of different application contexts (e.g. in the car, at home, as a pedestrian or in a public building) by integrating a large number of varying positioning technology in a single service.

2009	Back to Top

A Simulation Framework for Sensor-Based Systems in Second Life
Helmut Prendinger, and Boris Brandherm, and Sebastian Ullrich
Mel Slater; Janet Weisenberger (Hrsg.). Presence: Teleoperators and Virtual Environments, Vol. 18, No. 6, Pages 468-477, The MIT Press, 12/2009.
[ DOWNLOAD NOTE ] [BibTeX] show abstract
This paper describes a simulation framework for sensor-based systems based on Second Life, a popular virtual 3D multi-user online world. With this platform, the components of sensor-based systems can be mapped to, or represented by, virtual devices and objects. The intuitive user interface of Second Life (SL) and its comprehensive visualization capability support evaluation tasks of ubiquitous computing applications. Developers, as avatars, can directly control and manipulate virtual devices. In this way, different settings of sensor-based systems can be tested, and interactively improved. A bidirectional interface between sensor-based systems and Second Life has been implemented to demonstrate Second Life as a testbed for an RFID-based positioning system.

Multilingual Speaker Age Recognition: Regression Analyses on the Lwazi Corpus
Michael Feld, Etienne Barnard, Charl van Heerden, Christian Müller
Proceedings of the 2009 IEEE Workshop on Automatic Speech Recognition & Understanding (ASRU 2009)
[ DOWNLOAD NOTE ] [BibTeX] show abstract
Multilinguality represents an area of significant opportunities for automatic speech-processing systems: whereas multilingual societies are commonplace, the majority of speech-processing systems are developed with a single language in mind. As a step towards improved understanding of multilingual speech processing, the current contribution investigates how an important para-linguistic aspect of speech, namely speaker age, depends on the language spoken. In particular, we study how certain speech features affect the performance of an age recognition system for different South African languages in the Lwazi corpus. By optimizing our feature set and performing language-specific tuning, we are working towards true multi-lingual classifiers. As they are closely related, ASR and dialog systems are likely to benefit from an improved classification of the speaker. In a comprehensive corpus analysis on long-term features, we have identified features that exhibit characteristic behaviors for particular languages. In a follow-up regression experiment, we confirm the suitability of our feature selection for age recognition and present cross-language error rates. The mean absolute error ranges between 7.7 and 12.8 years for same-language predictors and rises to 14.5 years for cross-language predictors.

Spatial Modeling of Activity and User Assistance in Instrumented Environments
Christoph Stahl
Dissertation, Saarland University, 2009
show abstract
This dissertation presents a design method for proactive user assistance systems in instrumented environments. The method addresses typical design issues, such as the modeling of users’ needs and the choice and placement of sensors and actuators for human-environment interaction. The design process is supported through the combination of a geometric environment model and a situational semantic activity model. The geometric model is used to visualize the spatial context, in which the activities that are to be supported take place. The activity model is derived from Activity Theory and hierarchically represents tasks and activities in their situational context. Both models are linked by an ontology and form a hybrid location model. To support the method, we implemented a map modeling toolkit that allows to geometrically represent built environments in 3-D, and to model their furnishing and instrumentation with sensors and actuators. Of particular importance was the development and integration of an ontology-based activity editor. Furthermore, the toolkit facilitates the development of navigational aid through a route finding algorithm. The work concludes with five use cases that describe how the method and modeling toolkit have been applied for the design and development of intelligent environments and navigational aid for indoor and outdoor spaces. It also highlights how Dual Reality settings have contributed to the simulation of the developed assistance systems.

UbisLabel & UnderscoreEncoding: a new Approach for Label-Encoding in the Multilingual World Wide Web
Dominikus Heckmann, Matthias Loskyll
Internet/WWW 2009 (submitted)
show abstract
Internationalization of identification names, for cities for instance, bears two major problems: which language to choose for the label and which character set to choose for those characters that are not covered by the ASCII character set. UbisLabel is a new approach to combine ideas for internationalizing the labels for identifiers on the Semantic Web and the Web in general. We introduce an inline syntax to put several, possibly multilingual labels into one string. UderscoreEncoding is able to represent the full Unicode code points shorter than any existing UTF representation. It has been developed in order to represent special characters in an efficient manner without using any characters apart from alphanumerical letters plus the underscore only. The idea is that such encoded labels can be attached directly to the identifiers (like URIs) without the need to be further encoded anywhere else in the Internet.

Addressing Challenges of Ubiquitous User Modeling: Between Mediation and Semantic Integration
Shlomo Berkovsky, Dominikus Heckmann, Tsvi Kuflik
Ubiquitous User Modeling, Spring 2009
show abstract
Ubiquitous User Modeling aims at providing personalized services to inhabitants of smart environments. Current research in ubiquitous user modeling focuses on two directions. The first is a practical approach that tries to resolve current problems of sparseness of data and heterogeneity of user modeling techniques and representations by mediation of user models or building hybrid systems. The second approach is based on semantic standardization of user modeling enabling user modeling data exchange and sharing by using a common user modeling ontology and language. Although both approaches have their limitations, their integration has the potential to leverage their advantages and overcome the limitations. This paper discusses initial work done in this direction, suggests a path for such integration, and points out research directions aimed at bridging the gap between these approaches.

DiLiA - A DIGITAL LIBRARY ASSISTANT - A new approach to information discovery through information extraction and Visualization
Inessa Seifert and Kathrin Eichler and Holmer Hemsen and Sven Schmeier and Michael Kruppa and Günter Neumann and Norbert Reithinger
Joaquim Filipe; Kecheng Liu (Hrsg.). International Conference on Knowledge Management and Information Sharing. International Conference on Knowledge Management and Information Sharing (KMIS-09), located at The International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, October 6-8, Madiera, Portugal, INSTICC Press, 2009
show abstract
This paper presents preliminary results of our current research project DiLiA (Digital Library Assistant). The goals of the project are are twofold. One goal of the project is the development of domain-independent information extraction methods. The other goal is the development of information visualization methods that interactively support researchers at time consuming information discovery tasks. We first describe issues that contribute to high cognitive load during exploration of unfamiliar research domains. Then we present a domain-independent approach to technical term extraction from paper abstracts, describe the architecture of the DiLiA, and illustrate an example co-author network visualization.

A Pool of Topics: Interactive Relational Topic Visualization for Information Discovery.
Inessa Seifert and Michael Kruppa
Maulin Huang et al (Hrsg.). Visual Information Communication. VINCI-09, September 24-25, Sydney, Australia, Springer, 2009.
show abstract
In this paper, we present a novel relational visualization that supports people at information discovery tasks in digital libraries. This visualization displays search query results structured into topics and highlights the intersections between them. The proposed visual repre- sentation introduces interactive drag-and-drop operations for manipula- tion of the generated topics. These operations mirror the human online searching strategies that involve boolean AND, OR, and NOT operators. In doing so, the information seeker can rene (or relax) a search query in an interactive way during a focusing or a defocusing phase. The inter- sections of topics are made explicitly visible to enable the information seeker to avoid frustrating "no hits" situations.

Mashing up user data in the Grapple User Modeling Framework
Fabian Abel, Dominikus Heckmann, Eelco Herder, Jan Hidders, Geert-Jan Houben, Daniel Krause, Erwin Leonardi, Kees van der Slujis
ABIS 2009
show abstract
In this paper we demonstrate the Grapple User Modeling Framework (GUMF), which exploits Semantic Web technologies and Web 2.0 paradigms to model users across different applications and domains. It introduces novel features such as dataspaces, which logically bundle user data, and user pipes, which allow to mash up user data from different sources.

Visualization of Spatial Knowledge with Ontology Trees and Adaptable Search Result Grids in the Era of Web 3.0
Matthias Loskyll, Dominikus Heckmann, Christian Glahn
9th International Conference on Knowledge Management and Knowledge Technologies
show abstract
With the emerging trend ofWeb 3.0 and the resulting huge amount of usergenerated semantically-enriched data, improved ways of knowledge visualization and human computer interaction are needed. We present several techniques of visualizing particularly spatial knowledge in largely scalable, clear structured ontology trees on the web. In addition, we describe the representation of search results with a combined approach consisting of Ajax-based grids and Google Maps.

The Augmented Dorfladen – Interacting with Real and Virtual Products in Instrumented Shelves
Michael Schmitz, Elena Minina and Homeira Quraischy
Proceedings of 5th International Conference on Intelligent Environments, Barcelona, Spain, July 20 - 21, 2009
[BibTeX] show abstract
This work describes a multimodal product information and comparison system designed for a rural corner store (the so called Dorfladen). Such a Dorfladen only provides a limited assortment, which we extend with virtual items on embedded displays, allowing customers to inspect and select unavailable products along their real world shopping activities for later ordering. We present the design and implementation of the system prototype and discuss the main results of a formal user study evaluating the user interface, which combines tangible interaction and natural language dialogues to interact with real and virtual products in one instrumented space.

UbisEditor 3.0: Collaborative Ontology Development on the Web
Matthias Loskyll, Dominikus Heckmann, Ichiro Kobayashi
Web 3.0: Merging Semantic Web and Social Web, Workshop at Hypertext 2009
show abstract
Ontologies continuously become larger and more complex, and therefore more and more dicult to maintain, to edit and to develop by one single person or a small group of experts. The basic principle of Web 2.0, on the other hand, is to use the willingness and knowledge of a huge community of users to create rich user-generated content. The obvious idea that comes to mind is to combine the technologies of the Semantic Web with the trend of the Web 2.0. We present UbisEditor 3.0, an easy-to-use web tool for the creation and manipulation of structured collective knowledge represented as ontologies. This web ontology editor is realized as part of the UbisWorld project and already supports ontology editing techniques like adding new concepts, renaming and deleting, but also the creation of personalized ontology views.

Ambiente Audionotifikation - Ein System zur kontext-sensitiven Integration von nicht-intrusiven Notifikationssignalen in emotionsklassifizierten Soundscapes
Ralf Jung
PhD Thesis, FR Informatik, Saarbrücken, Germany: Saarland University, 2009
[BibTeX] show abstract
Besides the visual information presentation, the auditory modality is the most common used modality in the area of Human-Computer Interaction. Audio signals are generally used for warnings and acoustic feedback. Especially in environments with many people, such signals often affect intrusively and distractive because of their differentness compared to the natural sound environment. In this thesis, the novel approach of Ambient Audionotification is presented that provides methods to notify persons individually. The used audio cues are unnoticeable for others and they have no disturbing effect on attendees. On the symbolic level, the novel Notification Instruments represent events and can be seamlessly embedded in an ambient soundscape. A novel notification hierarchy with three signal classes is introduced to realize a personalized and a context-aware notification method. An analysis of musicpsychological models of emotion and an investigation of music parameters that are well-known from musicology lead to a classification model for soundscapes based on their emotional expression. The effectiveness of the novel notification signals and the plausibility of the soundscape classification methods are confirmed by empirical studies.

Ideas for Collaborative Ontology Development on the Upcoming Web 3.0 Era
Matthias Loskyll, Dominikus Heckmann
11th Intl. Protégé Conference
show abstract
Ontologies continuously become larger and more complex, and therefore more and more dicult to maintain, to edit and to develop by one single person or a small group of experts. The basic principle of Web 2.0, on the other hand, is to use the willingness and knowledge of a huge community of users to create rich user-generated content. The obvious idea that comes to mind is to combine the technologies of the Semantic Web with the trend of the Web 2.0. In this paper we present UbisEditor, an easy-to-use web tool for the creation and manipulation of structured collective knowledge represented as ontologies. This web ontology editor is realized as part of the UbisWorld project (http://www.ubisworld.org) and already supports ontology editing techniques like adding new concepts, renaming and deleting, but also the creation of personalized ontology views.

Towards Collaborative Ontology Development in the Upcoming Web 3.0 Era with UbisEditor
Matthias Loskyll, Dominikus Heckmann
11th Intl. Protégé Conference
show abstract
Collaborative ontology development combines the community approach of Web 2.0 with one of the key technologies of the Semantic Web; therefore, this idea perfectly ts to the denition of Web 3.0 by Wahlster and Dengel [4]. In this demo description we present UbisEditor, an easy-to-use web tool for the creation and manipulation of structured collective knowledge represented as ontologies, similar to the idea of WebProtege [3]. This web ontology editor is realized as part of the UbisWorld project (http://www.ubisworld.org) and already supports ontology editing techniques like adding new concepts, renaming, deleting, multilingual label editing, but also the creation of personalized ontology views.

An Integrated Development Environment for Speech-Based Classification
Michael Feld, Christian Müller
Proceedings of the 13th International Conference ``Speech and Computer'' SPECOM 2009
[ DOWNLOAD NOTE ] [BibTeX] show abstract
This paper presents a new machine learning framework for speech-based classification tasks that was developed in conjunction with the Agender project (age and gender recognition for telephone applications). The main goal of this framework is to provide a completely integrated development environment supporting all processes from design over evaluation to deployment of classification systems. It is intended for both researchers as well as application developers and specializes in audio signals as the resource to be classified. We show that the proposed framework outperforms other tools in several aspects.

Evaluating Tangible Interaction with Real and Virtual Products – Qualitative User Feedback Results
Elena Minina, Michael Schmitz and Homeira Quraischy
IADIS Interfaces and Human Computer Interaction (IHCI), Algarve, Portugal, 20 - 22 June
show abstract
In this paper, we introduce a user study of symmetric multimodal product information and comparison system related to tangible interaction with real and virtual products. The system aims to extend the limited assortment of a small store with virtual items on touch-sensitive display embedded into shelves. It blends tangible interaction of both real products in a shelf and virtual products represented on a nearby screen. The user interface is complemented with a conversational interface that allows users to carry out a natural language dialog to accomplish the comparison and inspection of multiple products of the real and virtual world. The user study examines whether the multimodal interaction with the prototype is perceived as intended. In particular, we describe the key issues of the qualitative user feedback and discuss their implications for the future research steps.

Burgomaster and Pedro - A Pervasive Multi-Player Game for Rural Tourism
Michael Schmitz and Mehdi M. Moniri
IEEE International Conference in Games and Virtual Worlds for Serious Applications, Coventry, UK, March 23 - 24, 2009
[BibTeX] show abstract
Pervasive Games have the potential to add new qualities to otherwise possibly tedious or uninspiring areas and let users interact with their environment in novel and enjoyable ways. The spreading of wireless communication infrastructures and low cost programmable mobile devices equipped with positioning systems and cameras span a large and growing open playground for pervasive gaming applications. In this work we describe the game Burgomaster and Pedro, the result of an interdisciplinary project for the state Ministry of Environment to promote development and tourism of rural areas. It is a multiplayer game that requires its players to explore villages in order to find real and virtual objects, which have to be delivered to given destinations. It also allows to hide from or seek other players, since objects that are being delivered can be stolen by competitors. GPS positioning, wireless communication and visual marker recognition are the key technologies in this game that is designed for playful engagement with rural environments.

Tangible Interaction with Real and Virtual Products - Designing a Shopping Assistant for Rural Communities
Michael Schmitz and Homeira Quraischy
Tangible and Embedded Interaction, Cambridge, UK, February 16-18, 2009
show abstract
This work describes the design and development process of an interactive ordering system for rural corner stores, extending the limited assortment of a small store with virtual items on touch-sensitive displays embedded into shelves. The interface blends tangible interaction of both real products in a shelf and virtual products iconically represented on nearby screens. The tangible interaction component is complemented by a natural language interface, supporting comparison and inspection of multiple products of the real and virtual world.

Global Lab: an interaction, simulation, and experimentation platform based on Second Life and OpenSimulator
Anette von Kapri, Sebastian Ullrich, Boris Brandherm, Helmut Prendinger
Proceedings Pacific-Rim Symposium on Image and Video Technology (PSIVT'09), Tokyo, Japan
[ DOWNLOAD NOTE ] show abstract
In this paper we describe a novel platform for virtual worlds called Global Lab. Through example applications, we demonstrate the capabilities of this platform for ubiquitous simulation and experimentation for sensor-based systems. Furthermore, with multimedia information representation and support for bi-directional communication real and virtual devices are connected in the example of a participatory ecosystem scenario. The Global Lab contains multimodal presentation and interaction techniques which enhance universal communication. The overall motivation is to provide a framework with strong support for collaboration in virtual worlds for research purposes.

2008	Back to Top

Automated Usability Evaluation during Model-based Interactive System Development
Sebastian Feuerstack, Marco Blumendorf, Maximilian Kern, Michael Kruppa, Michael Quade and Mathias Runge
Proceedings of TAMODIA 2008 in conjunction with Engineering Interactive Systems 2008
show abstract
In this paper we describe an approach to efficiently evaluate the usability of an interactive application that has been realized to support various platforms and modalities. Therefore we combine our Multi-Access Service Platform (MASP), a model-based runtime environment to offer multimodal user interfaces with the MeMo workbench which is a tool supporting an automated usability analysis. Instead of deriving a system model by reverse-engineering or annotating screenshots for the automated usability analysis, we use the semantics of the runtime models of the MASP. This allows us to reduce the evaluation effort by automating parts of the testing process for various combinations of platforms and user groups that should be addressed by the application. Furthermore, by testing the application at runtime, the usability evaluation can also consider system dynamics and information that are unavailable at design time.

MeMo Workbench for Semi-Automated Usability Testing
Klaus-Peter Engelbrecht, Michael Kruppa, Sebastian Möller and Michael Quade
Proceedings of Interspeech 2008 Conference
show abstract
In this paper we present a new approach to the automation of usability evaluation for interactive systems. Design ideas or complete systems are modeled as a conditional state machine. Then, user interactions with the system are simulated on the basis of tasks, by first searching for possible solution paths and then generating deviations from these paths under consideration of user groups and system attributes. The approach has been implemented into a workbench which supports the modeling of the system and the evaluation of the simulations. We present first results for the reliability of the approach in modeling interactions with a spoken dialog system.

Simulation Framework with Testbed for Sensor-based Systems in Second Life
Sebastian Ullrich, Boris Brandherm, and Helmut Prendinger
Demo Session of the 10th International Conference on Ubiquitous Computing (UBICOMP’08), 2008
show abstract
This paper introduces a simulation framework and testbed for sensor-based systems utilizing “Second Life”, a popular virtual three-dimensional multi-user online world. The main contribution of our work consists of a bi-directional interface to Second Life, which can greatly benefit developers of sensor-based systems for ubiquitous computing applications. While simulators for individual sensors exist, testing a complete system is still a major challenge. Our approach aims to solve this problem by proposing a framework that embeds complex sensor-based systems into Second Life, and utilize its versatile and intuitive three-dimensional user interface.

Speaker Classification for Mobile Devices
Michael Feld, Christian Müller
Proceedings of the 2nd IEEE International Interdisciplinary Conference on Portable Information Devices (Portable 2008)
[pdf] [BibTeX] show abstract
User adaptivity is a key topic in the context of mobile devices and applications, and speech is one the sources of information which has more recently been discovered for this purpose. While considerable work has already been done in both finding algorithms and designing well-performing implementations for this speaker classification task on the desktop platform as part of the AGENDER approach, efforts to bring the results to portable platforms in a working framework have been rather scarce so far. This work seeks to state the major aspects that make mobile speaker classification different from its desktop counterpart, and proposes a number of changes and enhancements to the existing infrastructure to fulfill the requirements emerging from it.

Embedded Modules for Speaker Classification
Michael Feld
Proceedings of the Second IEEE International Conference on Semantic Computing 2008 (ICSC 2008)
[ DOWNLOAD NOTE ] [BibTeX] show abstract
Classifying speakers and their context is a research topic that increasingly finds its way into market-ready products. This paper describes how a speech-based classification problem can be split into components that are then combined in a classification module, which can be compiled for a specific platform and scenario with its respective technical requirements and limitations. We are focusing on the AGENDER Speaker Classification approach to show how a theoretic model can be transformed into a finished Embedded Module and present a tool that facilitates this in a fully automated build process.

A New Approach to Design and Evaluate Ambient Systems for Instrumented Environments
Ralf Jung and Tim Schwartz
Proceedings of International Conference on Intelligent Environments 2008, July 21- July 22, 2008, Seattle, USA
[BibTeX] show abstract
In this paper we raise the question whether we can measure the ambience of a system that is integrated in an existing environment. This paper describes a new approach for de- signing and developing ambient systems. Other than the traditional software development concepts, we concentrate on the challenges of ambient designs. First, we give a mo- tivation why it would be interesting to measure ambience followed by some ideas how it could look like. By trying to find a formula to rate the “level of ambience”, we identify a few factors that seem to be important for ambient systems. We think that having such a formula would be of great help for developing, designing and comparing different concepts of systems. In the second part, we introduce the Ambience Design Cy- cle which consists of four designing stages that could assist designers in the developing process. Both parts should be seen as food for thought of how to design ambient systems.

Information Transfer Efficiency of Peripheral Audio Cues
Ralf Jung
Proceedings of International Conference on Intelligent Environments 2008, July 21- July 22, 2008, Seattle, USA
[BibTeX] show abstract
Transmitting basic information with non-speech audio cues is often afflicted with some disadvantages because of the differentness of the signals compared to the environmental noises. In this paper we present an evaluation of our ambient au- dio notification approach that uses instruments included in specially arranged aesthetical background music. First, we give a short introduction to non-speech audio cues and audi- tive perception. The study includes findings of the notifica- tion effectiveness and the recognition delay of notification instruments and compares it to a traditional audio cue.

Take your Smart Music with you and be up to date
Ralf Jung
Proceedings of International Conference on Intelligent Environments 2008, July 21- July 22, 2008, Seattle, USA
[BibTeX] show abstract
We present a user adaptive ambient audio notification ser- vice for multi-user environments connected to a user mod- eling service. The location adaptive system works unobtru- sively by embedding non-speech audio cues, in the form of natural instruments, in aesthetical self composed music. First we introduce the ambient notification system for intel- ligent environments followed by the identification of exten- sion properties for the implemented user model ontology to get the personalized ambient audio notification service.

Cognitive Approach to the Design of a User-adaptive Interface for an Intelligent Product Consulting System
Elena Minina
IADIS International Conference Intelligent Systems and Agents 2008, Amsterdam, Netherlands

Integrated Speaker Classification for Mobile Shopping Applications
Michael Feld and Gerrit Kahl
Proceedings of the 5th International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH 2008), July 28 - August 1, 2008, Hannover, Germany
[ DOWNLOAD NOTE ] [BibTeX] show abstract
This paper presents an approach to how speaker classification can be used to enable new ways for recommender systems in a mobile shopping environment to bootstrap user models and avoid common problems such as the "early rater". In a concrete shopping scenario, we introduce the speech-controlled Mobile ShopAssist demonstrator that allows a new customer to more quickly find a product that fulfills his or her demographic group’s specific requirements by exploiting features extracted from speech using the AGENDER speaker classification system. We propose a method for computing preference scores based on the user's profile and demonstrate how the application’s GUI can be adapted to deliver the recommendations to the user.

Smart Sound Environments: Merging Intentional Soundscapes, Non-Speech Audio Cues and Ambient Intelligence
Ralf Jung
Proceedings of International Conference on Acoustics 2008, June 29- July 04, 2008, Paris, France
[BibTeX] show abstract
We introduce an intelligent audio notification system for multi-user environments that provides users with information about events (e.g. important emails) in a more discreet and non-distracting way. The peripheral awareness of individual-related events is done by using non-speech audio cues which can be seamlessly integrated into artificial background soundscapes. These ambient soundscapes are self-composed with respect to well-known perceptual constraints like auditive Gestalt laws as well as music psychological findings. To follow a hierarchical approach for the notification sounds we use notification instruments, ambient noises and traditional alert signals that are grouped by their level of intrusiveness. Since the notification system also follows a human-centered approach it takes parameters like user preferences, his/her current position in the environment and the type of event into consideration to decide which notification is the appropriate at this time. In the paper, we will describe the architecture of the personalized ambient audio notification service, compositional constraints as well as some findings of a user study in which we tested successfully the efficiency of our system with 25 subjects.

Ambience for Auditory Displays: Embedded Musical Instruments as Peripheral Audio Cues
Ralf Jung
Proceedings of International Conference on Auditory Display (ICAD 2008), June 24 - June 27, 2008, Paris, France
[BibTeX] show abstract
From alarm signals and data sonification to multimodal interfaces, auditory displays are omnipresent in our everyday life and they become more and more popular. But there are some challenges we have to meet because of the differentness of the auditory sense compared to the visual sense. Usually, audio notification signals are limited to simple warning cues and system feedback that are in most cases intrusive be- cause they differ from the environmental noise. That has the effect that people present in the room could be distracted from their current tasks because they cannot “close their ears.” To prevent the disturbing effect of traditional notification signals we developed the novel concept of non-speech audio notification embedded in ambient soundscapes to provide multi-user notification in a more discreet and non-disturbing way. Instead of using well-known non- speech cues like auditory icons and earcons, we decided to compose and record peripheral soundscapes and notification instruments by ourselves towards a more aesthetic approach. In this paper, we give an overview of our location-aware system with two applications (PAAN, AeMN) and sketch a real life scenario in a wine department of a supermarket. We will also present findings from a user study and provide a small collection of notification instruments and soundscapes as audio samples.

Beam-Its – Virtual Sticky Notes in the Real World
Lübomira Spassova and Andreas Butz
Adjunct Proceedings of Pervasive Computing 2008
[pdf] [BibTeX] show abstract
We present the concept, design, and implementation of Beam-Its, a virtual version of sticky notes, which can be placed in the physical environment. Beam-Its are created on a PDA and placed on objects or surfaces in the environment, where they are currently displayed by a steerable projector. Beam-Its can contain handwritten text and sketches, just as physical Post-Its, but also photos and sound recordings made on the PDA, thus extending the functionality of physical Post-Its. They can also appear or disappear depending on context, which enables additional usage scenarios.

Dynamic Bayesian Network Based Interest Estimation for Visual Attentive Presentation Agents
Boris Brandherm, Helmut Prendinger, Mitsuru Ishizuka
Proceedings of 7th International Conference on Autonomous Agents and Multiagent Systems (AAMAS’08), ACM Press, 2008, 191-198
[ DOWNLOAD NOTE ] show abstract
In this paper, we report on an interactive system and the results of a formal user study that was carried out with the aim of comparing two approaches to estimating users’ interest in a multi-modal presentation based on their eye gaze. The scenario consists of a virtual showroom where two 3D agents present product items in an entertaining way, and adapt their performance according to users’ (in)attentiveness. In order to infer users’ attention and visual interest with regard to interface objects, our system analyzes eye movements in real-time. Interest detection algorithms used in previous research determine an object of interest based on the time that eye gaze dwells on that object. However, this kind of algorithm does not seem to be well suited for dynamic presentations where the goal is to assess the user’s focus of attention with regard to a dynamically changing presentation. Here, the current context of the object of interest has to be considered, i. e., whether the visual object is part of (or contributes to) the current presentation content or not. Therefore, we propose to estimate the interest (or non-interest) of a user by means of dynamic Bayesian networks that may take into account the current context of the attention receiving object. In this way, the presentation agents can provide timely and appropriate response. The benefits of our approach will be demonstrated both theoretically and empirically.

Simulation of Sensor-based Tracking in Second Life (Demo Paper)
Boris Brandherm, Sebastian Ullrich, Helmut Prendinger
Proceedings of 7th International Conference on Autonomous Agents and Multiagent Systems (AAMAS’08), ACM Press, 2008, 1689-1690
[ DOWNLOAD NOTE ] show abstract
This paper describes “Second Life” as a novel type of testbed and simulation environment for sensor-based applications. Second Life is a popular virtual online world that provides a free networked multi-user three-dimensional (3D) environment. The overall goal of our work is to support the development, testing, and deployment of sensor-based applications. In particular, pervasive systems like smart environments make heavy use of wireless sensor networks. However, the development of such systems requires much effort and the success of a system relies heavily on good planning and testing. Many different factors have to be taken into consideration and the environment has to be modeled carefully to foresee potential problems or to be able to perform changes before actual implementation. Until now, only custom-made solutions exist whereby technical limitations restrict adequate testing. By contrast, our approach introduces a flexible architecture for an extensible testbed for sensor-based applications. It employs Second Life to model an easily customizable three-dimensional environment with various interaction possibilities.

Three Output Planning Strategies for Use in Context-aware Computing Scenarios
Gerrit Kahl, Rainer Wasinger, Tim Schwartz, and Lübomira Spassova
Proceedings of the AISB 2008 Symposium on Multimodal Output Generation (MOG 2008), April 3 - April 4, 2008, Aberdeen, Scotland, UK
[pdf] [BibTeX] show abstract
In everyday life, it is useful for mobile devices like cell phones and PDAs to have an understanding of their user's surrounding context. Presentation output planning is one area where such context can be used to optimally adapt information to a user's current situational context. This paper outlines the architecture of a context-aware output planning module, as well as the design and implementation of three output generation strategies: user-define, symmetric multimodal, and context-based output planning. These strategies are responsible for selecting the best suited modalities (e.g. speech, gesture, text), for presenting information to a user situated in a public environment such as a shopping mall. A central point of this paper is the identification of context with finite resources to obtain a private and/or public output. We show via a wording demonstrator the extent to which such factors can, with readily available technology, be incorporated into a system. The paper also outlines the set of reactions that a system might take when given context information on the user and the environment.

The Digital Sommelier: Interacting with Intelligent Products
M. Schmitz, J. Baus and R. Dörr
Internet of Things 2008, March 26-28, 2008, Zurich, Switzerland LNCS, Springer-Verlag Berlin Heidelberg
show abstract
We present the Digital Sommelier, an interactive wine shopping assistant that provides an intuitive multi-modal interface to general product information as well as to particular attributes of a certain product, such as its current temperature. Wine bottles sense their state via attached wireless sensors and detect user interaction over RFID and acceleration sensors. Visitors can inquire information either through physical interaction with products or a natural language interface. We describe a framework and toolkit for efficient prototyping of sensor based applications as the foundation for the integration of different sensor networks utilized by the sommelier. We further introduce our concept of talking products, an anthropomorphic interaction pattern that allows customers to directly talk to products with personalities.

Context Integration for Ubiquitous User Modeling: Solving Semantic Conflicts with WordNet and GUMO
Dominikus Heckmann and Christian Blass
5th International Workshop on Ubiquitous User Modeling, UbiqUM'2008, Gran Canaria, Spain, 2008

A Survey of Human-Computer Interaction Design in Science Fiction Movies
M. Schmitz, Ch. Endres and A. Butz
Second International Conference on Intelligent Technologies for Interactive Entertainment (ICST INTETAIN ’08). January 8-10, 2008, Cancun, Mexico. Copyright 2008 ICST. ISBN 978-963-9799-13-4.

Sharing Memories of Smart Products and their Consumers in Instrumented Environments
Wolfgang Wahlster, Alexander Kröner, Michael Schneider and Jörg Baus
it - Information Technology 50(1), Special Issue on Ambient Intelligence, pp. 45-50, Oldenbourg.
show abstract
Intelligent assistants need precise knowledge about activity sequences and the habits of their users so as to support them in an adequate manner. The following contribution addresses an approach to user support, which takes advantage of the object centred nature of many day-to-day activities. By means of application examples from the everyday, we illustrate how a combination of smart items and digital memories allows for realizing innovative support mechanisms, which take into account static knowledge about objects as well as situational observations and historical data. Here, we devote special attention to applications originating from the sharing of data gathered this way between users and systems.

2007	Back to Top

Looking for Unexpected Consequences of Interface Design Decisions: The MeMo Workbench
Anthony Jameson, Angela Mahr, Michael Kruppa, Andreas Rieger, Robert Schleicher
6th International workshop on TAsk MOdels and DIAgrams (TAMODIA 2007)
show abstract
This paper discusses and illustrates work in progress on the MEMO workbench for early model-based usability evaluation of interface designs. Characteristic features of the workbench include (a) the prediction of errors via rules that refer to user attributes; and (b) the automatic generation of methods for performing specific tasks and for recovering from errors.

Big MaMUT is Watching You – Interaction-Tracking for Instrumented Environments
Michael Schmitz and Henning Zimmer
Proceedings of the 3rd IET International Conference on Intelligent Environments (IE’07), Ulm, Germany
[ DOWNLOAD NOTE ] show abstract
A major challenge in ubiquitous computing applications in intelligent environments is the simultaneous support of multiple users, especially when users interact with and move freely within the environment - without being equipped with sensors or personal devices. In such a situation, it is difficult to provide personalized services, which - for example - rely on user profiles or histories, if the user does not explicitly identify her- or himself with the system, whenever an interaction takes place. We chose a computer vision based approach to tackle this problem and divided the problem space into two concerns: Macro-tracking takes care of people tracking and consistent labelling of users in order to determine which user is in vicinity of a specific interactive device (e.g. a touchscreen); Micro-tracking observes the adjacency of such an interactive device to resolve ambiguities, in case that more than one person are potential interactors. The results of these two trackers are fused to conclude and assign persons to interactions, detected by the environment. We describe our implementation that can theoretically integrate arbitrary numbers of cameras, utilizing off-the-shelf-hardware, i.e. personal computers and cameras - which was one of the main requirements of MaMUT, the Macro- and Micro User Tracker.

Interest Estimation Based on Dynamic Bayesian Networks for Visual Attentive Presentation Agents
Boris Brandherm, Helmut Prendinger, Mitsuru Ishizuka
ICMI ’07: Proceedings of the 9th international conference on Multimodal interfaces, ACM, 2007, 346-349
[ DOWNLOAD NOTE ] show abstract
In this paper, we describe an interface consisting of a virtual showroom where a team of two highly realistic 3D agents presents product items in an entertaining and attractive way. The presentation flow adapts to users’ attentiveness, or lack thereof, and may thus provide a more personalized and user-attractive experience of the presentation. In order to infer users’ attention and visual interest regarding interface objects, our system analyzes eye movements in real-time. Interest detection algorithms used in previous research determine an object of interest based on the time that eye gaze dwells on that object. However, this kind of algorithm is not well suited for dynamic presentations where the goal is to assess the user’s focus of attention regarding a dynamically changing presentation. Here, the current context of the object of attention has to be considered, i. e., whether the visual object is part of (or contributes to) the current presentation content or not. Therefore, we propose a new approach that estimates the interest (or non-interest) of a user by means of dynamic Bayesian networks. Each of a predefined set of visual objects has a dynamic Bayesian network assigned to it, which calculates the current interest of the user in this object. The estimation takes into account (1) each new gaze point, (2) the current context of the object, and (3) preceding estimations of the object itself. Based on these estimations the presentation agents can provide timely and appropriate response.

Providing Individual Route Instructions for Indoor Wayfinding in Complex, Multi-Level Buildings
Stefan Münzer, Christoph Stahl
GI-Days 2007 Young Researchers Forum, Sept. 10-12, Münster, ifgi Prints series - to appear
show abstract
The present paper describes an interdisciplinary approach to personalized indoor wayfinding in complex multi-level buildings. Users are presented with individual visual route instructions on ambient displays in the environment. These instructions are provided by a modelling software which calculates individual paths through a virtual model of the building. Since users should be able to comprehend and learn such route instructions easily and quickly, the question arises which instructional format will foster human wayfinding in this scenario most effectively. An empirical study was therefore designed. Egocentric, view-based visual instruction formats are compared to a map-based, allocentric format. Currently, data on route learn-ing and wayfinding success after being presented with route instructions in different formats are collected in a complex, multi-level building. The results of this study will contribute to the development of design guidelines for cognitively adaequate indoor route instructions, in particular for egocentric and animated visual formats.

The Roaring Navigator: A Group Guide for the Zoo with Shared Auditory Landmark Display
Christoph Stahl
In: Proceedings of the 9th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '07). ACM (SIGCHI), New Nork. pp. 219-222
show abstract
In this paper, we introduce a shared auditory landmark display which conveys spatial survey knowledge and navigational aid to multiple users. Our guide is situated in a zoo environment, so we use recordings of animal voices to indicate the location of the animal enclosures. Spatial audio manipulates the volume and stereo balance of the sound clips, so that the listener can identify their distance and direction. The system also proactively presents audio clips with detailed information about each animal. To avoid the typical effect of social isolation through audio guides, we use shared audio so that the same sounds will be presented to each user at the same time. We have conducted an initial user study of paired visitors in the zoo to evaluate the usability of the system with positive results. The participants reported that the system is easy to use and has a stimulating influence on the communication between the visitors. As a further result, the study indicates that ‘lightweight’ navigational aid can be sufficient for wayfinding tasks in certain environments, which provides only the linear distance and direction of the destination.

The User Model and Context Ontology GUMO revisited for future Web 2.0 Extensions
Dominik Heckmann, Eric Schwarzkopf, Junichiro Mori, Dietmar Dengler, Alexander Kröner
Contexts and Ontologies: Representation and Reasoning, pp.37-46
show abstract
We revisit the top-level ontology Gumo for the uniform man- agement of user and context models in a semantic web environment. We discuss design decisions, while putting the focus on ontological is- sues. The structural integration into user model servers, especially into the U2M-UserModel&ContextService, is also presented. We show ubiq- uitous applications using the user model ontology Gumo together with the user model markup language UserML. Finally, we ask how data from Web 2.0 and especially from a social tagging application like del.icio.us as a basis for user adaptation and context-awareness could in°uence the ontology.

A Location-Adaptive Human-Centered Audio Email Notification Service for Multi-User Environments
Ralf Jung, Tim Schwartz
Proceedings of HCI International 2007, July 22-27, 2007, Beijing, P.R. China
[ DOWNLOAD NOTE ] [BibTeX] show abstract
In this paper, we introduce an application for a discreet notification of mobile persons in a multi-user environment. In particular we use the current user position to provide a personalized email notification with non-speech audio cues embedded in aesthetic background music. The notification is done in a peripheral way to avoid distraction of other people in the surrounding.

Peripheral Notification With Customized Embedded Audio Cues
Ralf Jung, Tim Schwartz
Proceedings of the 13th International Conference on Auditory Display (ICAD), June 26-29, 2007, Montréal, Canada
[ DOWNLOAD NOTE ] [BibTeX] show abstract
Peripheral notification services allow users to monitor information with less distraction of attendees in their surrounding. In the majority of cases, the information is provided by visual displays that often have several disadvantages, e.g. the lack of privacy or the user is locally bounded to the surrounding of the display. In this paper, we introduce an approach for a discreet notification of persons in multi-user environments. In particular, we use the current user position to provide a personalized and locationaware notification service with non-speech audio cues embedded in aesthetic background music. Thereby we enriched the music, and especially the notification audio cues, with functionality to in- fluence the perception and to control the attention of the listeners. These functional pieces of music should stay in the peripheral background to avoid too much attention. The used ambient soundscapes and the set of corresponding notification instruments were composed and recorded by ourselves. The development process including compositional constraints with respect to auditive perception and emotional effects raised by music will also be introduced in this paper as well as the nomadic event notification service which includes our indoor positioning system for mobile devices.

Providing Context-Aware Personalization through Cross-Context Reasoning of User Modeling Data
Shlomo Berkovsky, Lora Aroyo, Dominik Heckmann, Geert-Jan Houben, Alexander Kröner, Tsvi Kuflik, Francesco Ricci
International Workshop on Ubiquitous and Decentralized User Modeling (UbiDeUM'2007), pp. 2-7
[ DOWNLOAD NOTE ] show abstract
Existing personalization systems base their services on user models that typically disregard the issue of context-awareness. This work focuses on developing mechanisms for cross-context reasoning, i.e., inferences linking user model data in two different contexts. That reasoning process augments the typically sparse user models, by inferring the missing information from other contextual conditions, and better supports context-aware personalization. Thus, the proposed approach upgrades the existing personalization systems and facilitates provision of more accurate context-aware personalized services.

Mining the Structure of Tag Spaces for User Modeling
Eric Schwarzkopf, Dominik Heckmann, Dietmar Dengler, Alexander Kröner
Proceedings of the Workshop on Data Mining for User Modeling at the 11th International Conference on User Modeling (K-DUUM'2007), pp. 63-75
[ DOWNLOAD NOTE ] show abstract
We propose an approach for using data from a social tagging application like del.icio.us as a basis for user adaptation. We discuss several algorithms for learning taxonomies of tags from tag spaces. The learned taxonomy can be used to define adaptation rules that determine how to change a system given the tags a user is most interested in. We describe how the latter can be determined by mapping a user's personal tag space into the taxonomy. The contributions of this work are a description of an application scenario for taxonomymining algorithms, a discussion and comparison of algorithms by Mika[3], Heymann et al.[2], and Schmitz et al.[4], and the proposal of an extension to the algorithms that takes the contexts of tags into account when building a taxonomy.

Here and Now: A User-Adaptive and Location-Aware Task Planner
Christoph Stahl, Dominik Heckmann, Tim Schwartz, Oliver Fickert
International Workshop on Ubiquitous and Decentralized User Modeling (UbiDeUM'2007), pp. 52-63
[pdf] show abstract
In this paper, we present a ubiquitous location-based task planner that integrates a to-do list and a schedule/calendar with route knowledge and adapts both its view and alarms to the user’s current situation. The task planner is hosted on a web server and can be accessed from everywhere via a mobile Web terminal, such as a mobile phone or notebook, or a public display infrastructure which recognizes the user by their Bluetooth-device. Tasks can be localized by specifying a certain location where the task can be accomplished, such as an office or a store. Alternatively, a category can be chosen from an ontology that includes activities like shopping, sports or traveling by airplane. Since the task planner is likely to include dozens of tasks for the near future, it is too large to be browsed on the go. Therefore the planner implements a ‘here-and-now’ view, which adapts to the current time/date and location of the user. Based on knowledge about the purpose, address and opening hours of locations and routes, the task planner is able to filter for tasks that can be accomplished nearby, considering the time to reach the location and other deadlines. A second feature is an adaptive reminder, which considers the time that is needed to travel to the specified location of a task.

Learning Ubiquitous User Models based on Users’ Location History
Junichiro Mori, Dominik Heckmann, Yutaka Matsuo, Anthony Jameson
Proceedings of the Workshop on Data Mining for User Modeling at the 11th International Conference on User Modeling (K-DUUM'2007), pp. 40-49
[ DOWNLOAD NOTE ] show abstract
Recent development of location technologies enables us to obtain the location history of users. This paper proposes a new method to infer users’ long-term properties from their respective location histories. Counting the instances of sensor detection for every user, we can obtain a sensor-user matrix. After generating features from the matrix, a machine learning approach is taken to automatically classify users into different categories for each user property. Inspired by information retrieval research, the problem to infer user properties is reduced to a text categorization problem. We compare weightings of several features and also propose sensor weighting. Our algorithms are evaluated using experimental location data in an office environment. Our algorithm will bootstrap creating ubiquitous user models to enable context-aware information services.

Adaptive, Intelligent Presentation of Information for the Museum Visitor in PEACH
Oliviero Stock, Massimo Zancanaro, Paulo Busetta, Charles Callaway, Antonio Krüger, Michael Kruppa, Tsvi Kuflik, Elena Not and Cesare Rocchi
UMUAI (5/2007) - User Modeling and User-Adapted Interaction (Winner of the James Chen Best Paper Award 2007)
show abstract
The study of intelligent user interfaces and user modeling and adaptation is well suited for augmenting educational visits to museums. We have defined a novel integrated framework for museum visits and claim that such a framework is essential in such a vast domain that inherently implies complex interactivity. We found that it requires a significant investment in software and hardware infrastructure, design and implementation of intelligent interfaces, and a systematic and iterative evaluation of the design and functionality of user interfaces, involving actual visitors at every stage. We defined and built a suite of interactive and user-adaptive technologies for museum visitors, which was then evaluated at the Buonconsiglio Castle in Trento, Italy: (1) animated agents that help motivate visitors and focus their attention when necessary, (2) automatically generated, adaptive video documentaries on mobile devices, and (3) automatically generated post-visit summaries that reflect the individual interests of visitors as determined by their behavior and choices during their visit. These components are supported by underlying user modeling and inference mechanisms that allow for adaptivity and personalization. Novel software infrastructure allows for agent connectivity and fusion of multiple positioning data streams in the museum space. We conducted several experiments, focusing on various aspects of PEACH. In one, conducted with 110 visitors, we found evidence that even older users are comfortable interacting with a major component of the system.

Integration between mobile and stationary devices
Antonio Krüger, Michael Kruppa
O. Stock, M. Zancanaro (Eds.) : PEACH - Intelligent Interfaces for Museum Visits. Intelligent Interfaces for Museum Visits (Cognitive Technologies), 04/2007, Springer Verlag Berlin, Heidelberg, ISBN 978-3540687542

Interaction Patterns for Smart Products
Michael Schmitz, Robert Dörr, Jörg Baus
Proceedings of the 1st Workshop on the Design of Smart Products (DoSP 2007), 23rd March 2007, Furtwangen University, Germany

Interacting with Projected Displays by Means of a 3D Interface and an Instrumented Bracelet
Lübomira Spassova
IEEE Symposium on 3D User Interfaces
[pdf] show abstract
This paper introduces a 3D user interface and a concept for an interaction device that we refer to as an instrumented bracelet. Both interfaces are purpose-designed to facilitate user interaction with the AR projection system Fluid Beam. The 3D interface is a desktop application that enables the creation and manipulation of projected displays (virtual displays) in the environment. With the instrumented bracelet, the user will be able to interact more freely and more intuitively with the projected displays using arm movements and speech.

The Role of Visuo-Spatial Working Memory in Incidental Environmental Learning
Stefan Münzer, Hubert Zimmer, Jörg Baus
Submitted to: Spatial Cognition and Computation

Adaptive Mobile Guides
Antonio Krüger, Jörg Baus, Dominik Heckmann, Michael Kruppa, Rainer Wasinger
P. Brusilovsky, A. Kobsa, and W. Nejdl (Eds.): The Adaptive Web, LNCS 4321, pp. 521–549, 2007. Springer-Verlag Berlin Heidelberg
show abstract
In this chapter we discuss various aspects of adaptive mobile guide applications. After having motivated the need for web based mobile applications, we will discuss technologies that are needed to enable adaptive mobile web applications, including not only positioning technologies but also sensor technologies needed to determine additional information on the context and situation of usage. We will also address issues of modeling context and situations before giving an overview on existing systems coming from three important classes of mobile guides: museum guides, navigation systems and shopping assistants. The chapter closes with an extensive discussion of relevant attributes of web based mobile guides.

Modelling Personality in Voices of Talking Products Through Prosodic Parameters
Michael Schmitz, Antonio Krüger, Sarah Schmidt
Proceedings of IUI2007: International Conference on Intelligent User Interfaces, January 28-31, 2007, Honolulu, Hawaii, USA, ACM Press, New York, 313-316
[ DOWNLOAD NOTE ] [BibTeX] show abstract
In this paper we report preliminary findings from two user studies that on the one hand investigate how prosodic parameters of synthetic speech can influence the perceived impression of the speakers personality and on the other hand explores if and how people attribute personality to objects such as typical products of daily shopping. The results show that a) prosodic parameters have a strong influence on the perceived personality and can be partially used to achieve a desired impression and b) that subjects clearly attribute personalities to products. Both findings encourage us to continue our work on a dialogue shell for talking products.

Auditory Perceptible Landmarks in Mobile Navigation
Jörg Baus, Rainer Wasinger, Ilhan Aslan, Antonio Krüger, Andreas Maier, Tim Schwartz
Proceedings of IUI 07 (to appear)
[ DOWNLOAD NOTE ] show abstract
Normally, mobile pedestrian navigation systems use visually perceptible landmarks to guide their users through the environment. In this article we introduce concepts for the use of auditory perceptible landmarks in route descriptions. Such auditory perceptible landmarks complement their visual counterparts and also stand to be beneficial for certain groups like the visually impaired and the elderly.

Computer Assisted Navigation and the Acquisition of Route and Survey Knowledge
Stefan Münzer, Hubert D. Zimmer, Maximilian Schwalm, Jörg Baus, Ilhan Aslan
Journal of Environmental Psychology, Volume 26, S. 300-308.
[ DOWNLOAD NOTE ] show abstract
It describes an empirical study on spatial learning conducted in a real environment. In the study, incidental spatial learning during wayfinding using navigation assistance on a PDA computer is compared with learning during wayfinding using maps. Computer assisted navigation is varied with respect to additional spatial context information and modality of presentation.

2006	Back to Top

Migrating Characters: Effective User Guidance in Instrumented Environments
Michael Kruppa
Akademische Verlagsgesellschaft Aka GmbH, Berlin, ISBN 3-89838-301-6 and ISBN 1-58603-667-X
[BibTeX] show abstract
The book deals with the conceptual design as well as with the realization of virtual characters, which, unlike previous works in this research area, are not limited to a use in virtual worlds. The presented Migrating Character approach on the contrary allows virtual characters to act and interact with the physical world. Different technical solutions allowing a Migrating Character to move throughout physical space, either completely autonomously or in conjunction with a user, are introduced and discussed as well as resulting implications for the characters behavior. While traditional virtual characters are acting in a well defined virtual world, Migrating Characters need to adapt to changing environmental setups in a very flexible way. A Migrating Character must be capable of determining these environmental changes by means of sensors. Furthermore, based on this data, an adequate adaptation of the characters behavior has to be realized. Apart from a theoretical discussion of the necessary enhancements of a virtual character when taking the step from virtual to real worlds, different exemplary Migrating Character implementations are introduced in the course of the book.

Eingebettete dynamische Bayessche Netze n-ter Ordnung
Boris Brandherm
[ DOWNLOAD NOTE ] show abstract
Das Ziel dieser Arbeit war die Konzeption, die Realisation und die Anwendung eines Systems, das eingebettete Systeme mit geringer Rechenleistung und wenig Arbeitsspeicher mit der Fähigkeit ausstattet, probabilistische Prozesse verarbeiten zu können. Die Grundlage für dieses System bildet der differentielle Ansatz von Darwiche zur Lösung Bayesscher Netze, der ein Bayessches Netz in ein multivariates Polynom umwandelt und dann auswertet. Diesen Ansatz von Darwiche haben wir so erweitert, dass nun auch die speziellen Bedürfnisse dynamischer Bayesscher Netze berücksichtigt werden. Wir können im dynamischen Bayesschen Netz eine Vorwärts- und Rückwärtspropagierung und auch eine Kombination der beiden Verfahren durchführen. Alte Zeitscheiben und andere überflüssige Netzstrukturen können ohne Informationsverlust aufgerollt werden, wobei das Polynom bei konstantem Speicherverbrauch ausgewertet wird. Aufbauend auf dieser theoretischen Ausarbeitung wurde eine Anwendung mit dem Namen JavaDBN entwickelt, die dynamische Bayessche Netze in spezielle Polynome umwandelt und für diese Quellcode (wahlweise Java oder C++) generiert. Dieser Quellcode führt die Berechnungen für die Auswertung des Polynoms (die Inferenz im dynamischen Bayesschen Netz) und das Anhängen neuer Zeitscheiben mit dem gleichzeitigen Rollup (dem Abschneiden der vorhergehenden Zeitscheibe) durch. Der Quellcode wurde so realisiert, dass in der Initialisierungsphase alle notwendigen Variablen eingeführt werden, und danach während der Laufzeit kein Speicher mehr allokiert oder deallokiert werden muss. Somit wird auch keine automatische Speicherbereinigung benötigt, die gerade in Echtzeit-Systemen wie z.B. eingebetteten Systemen nicht toleriert werden kann, da der Zeitpunkt ihrer Durchführung oft nicht vorhergesehen werden kann, und dadurch die Programmausführung zu nicht voraussehbaren Zeitpunkten unterbrochen werden würde. Zusätzlich lässt sich dadurch der Speicherbedarf und die Laufzeit des Programmes nach oben abschätzen. Ein weiterer großer Vorteil des Quellcodes liegt darin, dass er leicht lesbar ist, so dass das Verfahren der Inferenz und des Rollups für den Anwender transparent bleibt. Dadurch lassen sich Veränderungen im Quellcode vornehmen, so dass sich Zusatzfunktionen wie beispielsweise Datenbankzugriffe oder Methodenaufrufe realisieren lassen, die nicht oder nur sehr schwer möglich wären, wenn der Inferenzalgorithmus vorkompiliert ist und das dynamische Bayessche Netz zur Verarbeitung als Eingabe erhält. Dazu haben wir gezeigt, wie sich konstante Tabelleneinträge zu variablen Tabelleneinträgen umwandeln lassen, indem Arrays durch Methodenaufrufe ersetzt werden. Dies ist z.B. dann sinnvoll, wenn Tabelleneinträge nicht im Voraus bekannt sind oder sich zur Laufzeit verändern können. Ein Methodenaufruf könnte beispielsweise einen Datenbankzugriff realisieren (um beispielsweise die Tabelleneinträge aus einem Benutzerprofil einzulesen) oder von der abgelaufenen Zeit seit der letzten Instantiierung einer Zeitscheiben abhängig sein. Weiterhin erzeugt JavaDBN Quellcode für die Sensitivitätsanalyse und unterstützt das Werkzeug MatLab. Von den Anwendungen, die mit JavaDBN erzeugt wurden, möchte wir an dieser Stelle den Alarm Manager und den multisensoriellen Positionierungsservice Loriot erwähnen. Anhand der prototypischen Anwendung Alarm Manager demonstrierten wir erstmals, wie physiologische Daten eines Benutzers durch physiologische Sensoren erfasst und durch ein dynamisches Bayessches Netz in Echtzeit auf einem persönlichen eingebetteten System interpretiert werden können, um dem Anwender Nachrichten (in diesem Fall eine wichtige Benachrichtigung) an seinen aktuellen Zustand angepasst zu präsentieren. Als Sensoren verwendeten wir einen Beschleunigungssensor, einen Muskeltätigkeitssensor und einen Augenbrauensensor. In der Anwendung Loriot führen wir als Innovation die sogenannten georeferenzierten dynamischen Bayesschen Netze (geoDBNs) ein, die es ermöglichen, dass die Benutzerposition auf dem persönlichen Handheld-Gerät berechnet werden kann, ohne dass eine externe Verbindung zu einem Server besteht, auf dem dann die Positionsberechnung durchgeführt würde. Diese neue Methode hat sowohl an Rechenzeit als auch Arbeitsspeicher nur einen geringen Ressourcenbedarf und ist hoch-skalierbar für die In- und Outdoor-Positionierung. Momentan werden IR- und RFID-Sensoren verwendet, aber ebenso lassen sich leicht andere Sensoren einbauen, indem das dynamische Bayessche Netz um den entsprechenden Sensorknoten erweitert wird. Für die Modellierung dynamischer Bayesscher Netze spezifizieren wir neue Modellierungsstrukturen im Zusammenhang mit der Benutzermodellierung und der Sensorverarbeitung und führen damit den Begriff der dynamischen Bayesschen Netze n-ter Ordnung ein. Wir zeigen insbesondere im Zusammenhang mit der Sensorverarbeitung wie sich die Probleme lösen lassen, die durch die Latenz der Sensordaten auftreten können.

Ubiquitous User Modeling
Dominik Heckmann
Akademische Verlagsgesellschaft Aka GmbH, Berlin, ISBN 3-89838-297-4 and ISBN 1-58603-608-4
[ DOWNLOAD NOTE ] [BibTeX] show abstract
More and more interactions take place between humans and mobile or connected IT-systems in daily life. This offers a great opportunity, especially to user modelling, to reach better adaptation with ongoing evaluation of user behaviour. This work develops a complete framework to realize the newly defined concept of ubiquitous user modelling. The developed tools cover methods for the uniform exchange and the semantic integration of partial user models. They also account for the extended needs for privacy and the right of every human for introspection and control of their collected data. The SITUATIONAL STATEMENTS and the exchange language USERML have been developed on the syntactical level, while the general user model ontology GUMO and the UBISWORLD ontology have been developed on the semantic level. A multilevel conflict resolution method, which handles the problem of contradictory statements, has been implemented together with a web-based user model service, such that the road capability and the scalability can be proven with this approach.

Sharing Sensor Data in Intelligent Environments
Tim Schwartz, Dominik Heckmann, and Jörg Baus
Proceedings of the Workshop on Ubiquitous Knowledge Discovery for Users (UKDU 2006), pages 81 - 88.
[ DOWNLOAD NOTE ] [BibTeX] show abstract
Instrumented environments are enriched with sensors,senders and computing devices in order to support intelligent applications for the human-computer interaction. Because the sensors and senders in the environment can deliver a large amount of data, these so-called “intelligent environments” form ideal playgrounds to test the novel idea of ubiquitous knowledge discovery. In this paper we will describe the sensor architecture and the management software that is installed in the SUPIE (Saarland University Pervasive Intelligent Environment) for sharing basic sensor data. As an example application for ubiquitous knowledge discovery we describe our positioning system that collects data from different types of senders and that derives the user’s position by fusing all data that can be helpful for this task.

A Development Cycle for Ambient System Design
Tim Schwartz, Ralf Jung
International Workshop on Modelling and Designing User Assistence in Intelligent Environments (MODIE'06), Espoo, Finland, 2006
[ DOWNLOAD NOTE ] [BibTeX] show abstract
This paper describes a design development cycle for ambient systems. Other than the traditional software development concepts, we concentrate on the challenges of ambient designs. First we introduce the Ambient Development Cycle which consists of four designing stages. By trying to find a formula to rate the ``level of ambience", we identify a few factors that seem to be important for ambient systems. We think that having such a formula would be of great help for developing, designing and comparing different concepts of systems.

Sharing Control of Dispersed Situated Displays between Nomadic and Residential Users
Christian Kray, Keith Cheverst, Daniel Fitton, Mark Rouncefield, Corina Sas, Christoph Stahl
In Proceedings of the 8th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI 2006), ACM Press, 2006, pp. 61-68
show abstract
As the number of public displays in the environment increases, new opportunities open up to improve situated interaction and to enable new kinds of applications. In order to make distributed display resources available to nomadic users, a key issue to address is how control can be dynamically shared between display users. It is important to study how control over a shared display can be acquired, released or shared by nomadic and residential users given their competing demands for display resources.
In this paper, we present a system and a user study investigating these issues in the context of two applications both competing for display resources provided by a deployment of interactive office doorplates. The first application (Hermes II) provides situated note leaving and messaging services whereas the second one (GAUDI) supports user navigating a university department. Office occupants (i. e. residential users) can control whether the navigation application may (temporarily) use their doorplate display (thus giving priority to the navigation needs of nomadic users to the department). We report on findings from a user study, and discuss interface design implications for specifying display control.

Simulating and Evaluating Public Situated Displays in Virtual Environment Models
Christoph Stahl, Jens Haupert
International Workshop on Modelling and Designing User Assistance in Intelligent Environments (MODIE 2006). In: SFB 378 - Resource-Adaptive Cognitive Processes, Memo Nr. 86: Thomas Pederson, Helder Pinto, Michael Schmitz, Christoph Stahl, Lucia Terrenghi (Eds.): MODIE 2006, ISSN 0944-7822, pp. 32-35, 2006. Saarland University
[pdf] [BibTeX] show abstract
In this position paper, we address the problems that arise during the design phase of intelligent environments that make use of public situated displays for user assistance. We propose a new design process, which is based on an architectural model of the environment. It employs a toolkit that allows the designer of an intelligent environment to verify the visibility of displays from various virtual viewpoints before they are actually acquired and deployed. Our approach furthermore allows the early evaluation of application prototypes. We mirror the screens of real machines to the virtual displays and use an avatar as a positioning loopback device in order to test the behavior of context-aware presentations.

Modelling and Designing User Assistance in Intelligent Environments (MODIE 2006)
Thomas Pederson, Helder Pinto, Michael Schmitz, Christoph Stahl, Lucia Terrenghi (Eds.)
SFB 378 - Resource-Adaptive Cognitive Processes, Memo Nr. 86: Modelling and Designing User Assistance in Intelligent Environments (MODIE 2006) Workshop in Conjunction with the 8th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI 2006), ISSN 0944-7822, Saarland University
[pdf] [BibTeX] show abstract
Ubicomp research continually develops novel interaction techniques, sensing technologies, and new ways of presenting personalized information to the user. Gradually, companies operating in environments such as airports, museums or even shopping malls are becoming aware of the potential benefits in letting such technologies assist their users and customers. Intelligent environments are predicted to aid their users in pursuing their activities, such as wayfinding or shopping, through the situated presentation of personalized information. However, due to the large design space that ranges from wearable computing to public displays, the conceptual and technological choices pose new challenges to the designer of such user-assistance systems. The MODIE workshop aims towards models, principles and methodologies, which guide the designer of an intelligent environment in the early stages of the development process, such as task and requirements analysis and conceptual design.

Predicting User Experiences through Cross-Context Reasoning
Shlomo Berkovsky, Lora Aroyo, Dominik Heckmann, Geert-Jan Houben, Alexander Kröner, Tsvi Kuflik, Francesco Ricci
submitted to ABIS'06
show abstract
The existing personalization systems typically base their services on general user models that ignore the issue of context-awareness. This work focuses on developing mechanisms for cross-context reasoning of the user models, which can be applied for the context-aware personalization. The reasoning augments the sparse user models by inferring the missing information from other contextual conditions. Thus, it upgrades the ex-isting personalization systems and facilitates provision of accurate context-aware services.

SharedLife: Towards Selective Sharing of Augmented Personal Memories
Wolfgang Wahlster, Alexander Kröner, and Dominik Heckmann
In: O. Stock, M. Schaerf (eds.): Reasoning, Action and Interaction in AI Theories and Systems. Essays Dedicated to Luigia Carlucci Aiello, LNAI 4155, Berlin, Heidelberg: Springer, 2006, pp. 327-342
[ DOWNLOAD NOTE ] show abstract
The rapid deployment of low-cost ubiquitous sensing devices – including RFID tags and readers, global positioning systems, wireless audio, video, and bio sensors – makes it possible to create instrumented environments and to capture the physical and communicative interaction of an individual with these environments in a digital register. One of the grand challenges of current AI research is to process this multimodal and massive data stream, to recognize, classify, and represent its digital content in a context-sensitive way, and finally to integrate behavior understanding with reasoning and learning about the individual’s day by day experiences. This augmented personal memory is always accessible to its owner through an Internet-enabled smartphone using high-speed wireless communication technologies. In this contribution, we discuss how such an augmented personal memory can be built and applied for providing the user with context-related reminders and recommendations in a shopping scenario. With the ultimate goal of supporting communication between individuals and learning from the experiences of others, we apply this novel methods as the basis for a specific way of exploiting memories — the sharing of augmented personal memories in a way that doesn’t conflict with privacy constraints.

A Framework for Multi-User Support in Instrumented Public Spaces
Michael Schmitz
Proceedings of Humans & Computers - Sixth multidisciplinary Conference September 3-6, 2006, University of Applied Sciences Gelsenkirchen, ISBN 978-3-486-58129-4, p. 469-472
[ DOWNLOAD NOTE ] show abstract
The emerging trend towards complex technology that supports multiple users in public spaces is evidenced by the presence of shared displays in shopping malls, museums, and airports. As sensing and interaction begin to play a greater role in these environments, application and interaction concepts are evolving to take the distribution and dynamics of users and devices into account. In this paper we describe a layered approach to support multiple in environments that allows concurrent interaction with multiple devices and displays. We further identify issues that arise in public, interactive spaces with multiple users and discuss how we address them in our proposed framework.

Ambient Audio Notification with Personalized Music
Ralf Jung, Dominik Heckmann
Workshop on Ubiquitous User Modeling (UbiqUM'06), Riva del Garda, Italy, 2006, pp. 16-18
[ DOWNLOAD NOTE ] [BibTeX] show abstract
In this paper we present a user adaptive approach for an ambient audio notification application for multi-user environments. We provide a user centralized notification system working unobtrusively by embedding audio cues in an ambient soundscape by connecting the u2m.org user modeling service. First we introduce the ambient notification system for intelligent environments followed by the identification of extension properties for the user model on\-to\-logy to get an ambient user adaptive notification service. This short paper should be seen as work in progress that addresses the current research field combining ubiquitous computing and user modeling.

Exploiting the Link Between Personal, Augmented Memories and Ubiquitous User Modeling
Alexander Kröner, Dominik Heckmann and Michael Schneider
Workshop on Ubiquitous User Modeling (UbiqUM'06), Riva del Garda, Italy, 2006, pp. 25-26
[ DOWNLOAD NOTE ] show abstract
Dense logging of a user’s interactions with an intelligent environment enables the creation of artificial memory structures, which augment the user’s natural memory. In this article we discuss the interrelationship between such augmented memories and ubiquitous user modeling, and the particular benefits of linking both concepts. On the basis of a prototype implementation, we illustrate how ubiquitous user modeling contributes to the building of personal, augmented memories, and discuss how the latter one may enrich ubiquitous user models and thus grant other applications access to memory content. We conclude with a discussion of how users may exploit such hybrid systems to share their personal memories with other users.

SAFIR: Low-Cost Spatial Audio for Instrumented Environments
Michael Schmitz and Andreas Butz
Proceedings of the 2nd International Conference on Intelligent Environments, 5-6 July, 2006, Athens, Greece, ISSN 0537-9989, p. 427-430
[ DOWNLOAD NOTE ] [BibTeX] show abstract
We present a high-level Java API that allows developers of pervasive computing environments to integrate 2D or 3D spatial sound over loudspeakers into systems without any DSP knowledge. The system is platform independent and allows most arbitrary speaker configurations, thereby providing a scalable tool for real-time spatialization with low-cost off-the-shelf hardware.

Acquisition of Spatial Knowledge in Location Aware Mobile Pedestrian Navigation Systems
Ilhan Aslan, Maximilian Schwalm, Jörg Baus, Antonio Krüger, Tim Schwartz
Proceedings of the 8th international Conference on Human Computer Interaction with Mobile Devices and Services (Mobile HCI 2006), pages 105 - 108, ACM Press
[ DOWNLOAD NOTE ] [BibTeX] show abstract
In this paper we regard the navigation aid provided by mobile navigation systems in a real environment and the effects of these mobile assistants to the development of spatial knowledge. Therefore, we report on a user study concerning the acquisition of spatial knowledge. This study sets up on a former study described by Krüger and colleagues and sheds light on problems concerning the acquisition of survey knowledge while being navigated by a mobile handheld PC.

Towards a Multilingual Approach on Speaker Classification
Christian Müller, Michael Feld
Proceedings of the 11th International Conference "Speech and Computer" SPECOM 2006, 25-29 June 2006, St. Petersburg, Russia.
[pdf] [BibTeX] show abstract
This paper outlines a framework for a multilingual speaker classification system which is based on an underlying language identification module. First, the Agender speaker classification technology is introduced, a two-layered approach which primarily recognizes the speakers' age and gender but also incorporates novel domain-independent aspects that can be applied to other speaker characteristics like emotions or cognitive load. Then, it is pointed out that one of its major drawbacks consists of the fact that it has not been verified that the chosen set of speech features also works for other languages, especially for those with different phonological aspects. To overcome this drawback, it is suggested to extend Agender with a language identification module. The module presented here is designed to meet the requirements of a specific telephone-based application (which itself is not within the focus of this paper): The languages German, English and Turkish shall be discriminated on the basis of the initial utterance of the speaker; for each of the possible languages, hypotheses about the nature of the initial utterance are available; the domain encompasses a list of English product names. Although the suggested method is as yet only partly implemented, the first evaluation results are very promising: Turkish could be identified with an accuracy of 71.75%, German with an accuracy of 78.39%, and English with an accuracy of 79.89%. Besides this, the paper outlines the use of the language identification module within a multilingual version of Agender.

Taking Location Modelling to new Levels: A Map Modelling Toolkit for Intelligent Environments
Christoph Stahl and Jens Haupert
2nd International Workshop on Location- and Context-Awareness. In: M. Hazas, J. Krumm, and T. Strang (Eds.): LoCA 2006, LNCS 3987, pp. 74 – 85, 2006. Springer-Verlag Berlin Heidelberg.
[ DOWNLOAD NOTE ] [BibTeX] show abstract
We present a map modelling toolkit that meets the special requirements of pedestrian navigation in intelligent environments. Its central component is a graphical editor, which supports geometric modelling of architectural ground plans through polygon meshes. Multiple levels and their interconnections, such as ramps and staircases, can be represented through the aid of layers. In order to support a full range of activities, from travelling to interacting with pervasive user interfaces, coarse models on an outdoor scale can be hierarchically refined by submodels on building and room scales. The XML-encoded models can be useful for positioning systems, referencing spatial context and for route finding through multi-story buildings. Besides the editor, the toolkit provides a routing module for pedestrian navigation.

Towards a Notation for the Modeling of User Activities and Interactions Within Intelligent Environments
Christoph Stahl
3rd International Workshop on the Tangible Space Initiative (TSI 2006). In: Thomas Strang, Vinny Cahill, Aaron Quigley (Editors). Pervasive 2006 Workshop Proceedings, Deutsches Zentrum für Luft- und Raumfahrt, pages 441-452. ISBN 978-3-00-018411-6
[pdf] [BibTeX] show abstract
The design of ubiquitous computing systems for pedestrian assistance in large and complex environments, such as airports, museums or conferences, poses new challenges to existing development processes. The designer of such a system has to specify typical activities within the environment and how the system supports them, and decide which sensors and devices will be used for interaction. We propose a new design process, which extends the scenario-based design approach through the use of a more structured model of user activities and human-computer interaction in intelligent environments.

Anthropomorphized Objects: A Novel Interaction Metaphor for Instrumented Spaces
Michael Schmitz, Jörg Baus, Sarah Schmidt
Thomas Strang, Vinny Cahill, Aaron Quigley (Editors). Pervasive 2006 Workshop Proceedings. Deutsches Zentrum für Luft- und Raumfahrt. ISBN 978-3-00-018411-6
[ DOWNLOAD NOTE ] [BibTeX] show abstract
In this position paper we discuss the new interaction paradigm of Anthropomorphized Objects, which involves multimodal interaction with instrumented objects, spanning the modalities of speech, gestures, sound and haptics. This work aims at assisting non-expert users to access complex environments by offering an intuitive interface to services that support users e.g. in shopping malls to accomplish their shopping tasks. We further argue that the notion of affect will not only enrich the user experience but also has the potential to increase the efficiency of interactions and problem solving abilities of the user.

Situation Modeling and Smart Context Retrieval with Semantic Web Technology and Conflict Resolution
Dominik Heckmann
T.R. Roth-Berghofer, S. Schulz, and D.B. Leake (Eds.): MRC 2005, LNAI 3946, pp. 34–47, Springer-Verlag Berlin Heidelberg, 2006
[ DOWNLOAD NOTE ] [BibTeX] show abstract
We present a service to model situations and retrieve contextual information in mobile and ubiquitous computing environments. We introduce the general user model and context ontology GUMO for the uniform interpretation of distributed situational information in intelligent semantic web enriched environments. Furthermore, we present the relation to the user model and context markup language USERML, that is used to exchange partial models between different adaptive applications. Our modeling and retrieval approach bases on semantic web technology and complex conflict resolution concepts.

SPECTER: Building, Exploiting and Sharing Augmented Memories
Alexander Kröner, Dominik Heckmann, Wolfgang Wahlster
Workshop on Knowledge Sharing for Everyday Life, KSEL 2006, Kyoto, Japan, ISBN 4-902401-03-7, pages 9-16
[ DOWNLOAD NOTE ] [BibTeX] show abstract
Knowledge about the history of a given situation is of special interest for adaptive systems; it enables them to learn about users, to identify their habits, and thus to improve the quality of user support. In addition such knowledge can be applied by users to learn about themselves— and from others. We describe in this article how these issues can be addressed by the means of augmented memories created from sensor data automatically captured in an intelligent environment. After a discussion of how to represent and to process such memories, we show how they can be exploited by adaptive systems and users as well. One promising way of exploiting memories is to share them with others; this is reflected by the last part of our contribution where we address varying ways of sharing augmented memories.

The Anthropomorphized Product Shelf: Symmetric Multimodal Interaction with Instrumented Environments
Rainer Wasinger, Wolfgang Wahlster
Chapter in: Emile Aarts and José Luis Encarnação (eds.): True Visions: The Emergence of Ambient Intelligence. Heidelberg: Springer, 2006, in press (due out Feb.2006), ISBN: 3-540-28972-0.
[BibTeX] show abstract
Ambient intelligence environments require robust and intuitive interfaces for accessing their embodied functionality. This chapter describes a new paradigm for tangible multimodal interfaces, in which humans can manipulate, and converse with physical objects in their surrounding environment via coordinated speech, handwriting and gesture. We describe the symmetric nature of human environment communication, and extend the scenario by providing our objects with human-like characteristics. This is followed by the results of a usability field study on user acceptance for anthropomorphized objects, conducted within a shopping context.

Assessment of a User's Time Pressure and Cognitive Load on the Basis of Features of Speech
Anthony Jameson, Jürgen Kiefer, Christian Müller, Barbara Großmann-Hutter, Frank Wittig, Ralf Rummer
Journal of Computer Science and Technology
[BibTeX]

2005	Back to Top

Fusion and Coordination for Multimodal Interactive Information Presentation
Wolfgang Wahlster, H. Bunt, M. Kipp, M. Maybury
Stock, O., Zancanaro, M. (eds.): Multimodal Intelligent Information Presentation. Series Text, Speech and Language Technology, Vol 27. Kluwer Academic. pp. 325-340
[ DOWNLOAD NOTE ] show abstract
Users require more effective and efficient means of interaction with increasingly complex information and new interactive devices. This document summarizes the results of the international Dagstuhl Seminar on Coordination and Fusion in Multimodal Interaction that took place at Schloss Dagstuhl in Germany October 27 through November 2, 20011. We first outline a research roadmap in the near and long term. Next we describe requirements and an abstract architecture for this class of systems. We then detail requirements for semantic representations and languages necessary to enable these systems. Finally, we describe data, annotation methodologies and tools necessary to further advance the field. We conclude with a recommended action plan for forward progress in the community.

COMPASS2008: Multimodal, multilingual and crosslingual interaction for mobile tourist guide applications. Long paper accepted for presentation at INTETAIN 2005 conference.
Ilhan Aslan, Feiyu Xu, Hans Uszkoreit , Antonio Krüger and Jörg Steffen
INTETAIN 2005
[ DOWNLOAD NOTE ] show abstract
The Compass2008 project is a sino-german cooperation, aiming at integrating advanced information technologies to create a hightech information system that helps visitors to access location-sensitive information services during the 2008 Olympic Games in their preferred language, offering a variety of service-adaptive modalities available on the mobile devices. In this paper, we demonstrate one of the COMPASS2008 services, the Smart Dining Service, to showcase the new interaction concepts between multimodality, multilingual and location-sensitive information search.

The COMPASS2008 Smart Dining Service
Ilhan Aslan, Feiyu Xu, Hans Uszkoreit , Antonio Krüger and Jörg Steffen
Demo paper at INTETAIN 2005
[ DOWNLOAD NOTE ] show abstract
The Compass2008 project is a sino-german cooperation, aiming at integrating advanced information technologies to create a hightech information system that helps visitors to access location-sensitive information services during the 2008 Olympic Games in their preferred language, offering a variety of service-adaptive modalities available on the mobile devices. In this paper, we demonstrate one of the COMPASS2008 services, the Smart Dining Service, to showcase the new interaction concepts between multimodality, multilingual and location-sensitive information search.

The Virtual Room Inhabitant – Intuitive Interaction With Intelligent Environments
Michael Kruppa, Lübomira Spassova, Michael Schmitz
Proceedings of the 18th Australian Joint Conference on Artificial Intelligence (AI05)
[pdf] [BibTeX] show abstract
Abstract. In this paper we describe a new way to improve the usability of complex hardware setups in Intelligent Environments. By introducing a virtual character, we facilitate intuitive interaction with our Intelligent Environment. The character is capable of freely moving along the wall of the room. The character is aware of the users position and orientation within the room. In this way, it may offer situated assistance as well as unambiguous references to physical objects by means of combined gestures, speech and physical locomotion. We make use of a steerable projector and a spatial audio system, in order to position the character within the environment.

Performing Physical Object References with Migrating Virtual Characters
Michael Kruppa, Antonio Krüger
Proceedings of the first conference on INtelligent TEchnologies for interactive enterTAINment (INTETAIN2005)
[ DOWNLOAD NOTE ] [BibTeX] show abstract
In this paper we address the problem of performing references to physical objects in instrumented spaces. The concept behind our solution is based on virtual characters. These characters are capable of performing reasonable combinations of motion, gestures and speech in order to disambiguate references to real world objects. The new idea of our work is to allow characters to migrate between displays to find an optimal position for the reference task. Based on a user study, we have developed a rule-based system that, depending on the individual situation in which the reference is performed, determines the most appropriate reference method and technology from a number of different alternatives.

Parallel Presentations for Heterogenous User Groups - An Initial User Study
Michael Kruppa and Ilhan Aslan
Proceedings of the first conference on INtelligent TEchnologies for interactive enterTAINment (INTETAIN2005)
[ DOWNLOAD NOTE ] [BibTeX] show abstract
Presentations on public information systems, like a large screen in a museum, usually cannot support heterogeneous user groups appropriately, since they offer just a single channel of information. In order to support these groups with mixed interests, a more complex presentation method needs to be used. The method proposed in this paper combines a large stationary presentation system with several Personal Digital Assistants (PDAs), one for each user. In this way, it is possible to maximize the benefit the large system provides for all users, while the individual PDAs may also adapt to each user's preferences. The basic idea is to "overwrite" presentation parts on the large screen, which are of little interest to a particular user, with a personalized presentation on the PDA. We performed an empirical study with adult participants to examine the overall performance of such a system (i.e. How well is the information delivered to the users and how high is the impact of the cognitive load?). The results show, that after an initial phase of getting used to the new presentation method, subjects' performance during parallel presentations was on par with performance during standard presentations. A crucial moment within these presentations is whenever the user needs to switch his attentional focus from one device to another. We compared two different methods to warn the user of an upcoming device switch (a virtual character "jumping" from one device to another and an animated symbol) with a version, where we did not warn the users at all. Objective measures did not favour either method. However, subjective measures show a clear preference for the character version.

Ubiquitous User Modeling
Dominik Heckmann
PhD thesis, Computer Science Department, Saarland University, Germany
[ DOWNLOAD NOTE ] [BibTeX] show abstract
More and more interactions take place between humans and mobile or connected IT-systems in daily life. This offers a great opportunity, especially to user modeling, to reach better adaptation with ongoing evaluation of user behavior. This work develops a complete framework to realize the newly defined concept of ubiquitous user modeling. The developed tools cover methods for the uniform exchange and the semantic integration of partial user models. They also account for the extended needs for privacy and the right of every human for introspection and control of their collected data. The SITUATIONALSTATEMENTS and the exchange language USERML have been developed on the syntactical level, while the general user model ontology GUMO and the UBISWORLD ontology have been developed on the semantical level. A multilevel conflict resolution method, which handles the problem of contradictory statements, has been implemented together with a web-based user model service, such that the road capability and the scalability can be proven with this approach.

Product Associated Displays in a Shopping Scenario
Lübomira Spassova, Rainer Wasinger, Jörg Baus, Antonio Krüger
4th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 05), 2005, pp. 210-211.
[ DOWNLOAD NOTE ] [BibTeX] show abstract
In this paper, we introduce the concept of Product Associated Displays – PADs – as a way of providing visual feedback to users interacting with physical objects in an instrumented environment. PADs are projected public displays created at locations that can be intuitively associated with the objects they show information about. The concept is illustrated in a shopping scenario.

Design and Implementation of a Widget Set for Steerable Projector-Camera Units
Dennis Reiter and Andreas Butz
Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR) 2005
[ DOWNLOAD NOTE ] show abstract
We describe the design and implementation of graphical interaction widgets for use with a steerable projectorcamera unit. The design of our widgets is adapted to provide the right visual cues when projected and they are controlled by the user’s hand. The widgets’ input regions are arranged in an ergonomic way and they use a simple but robust computer vision technique for interaction.

Design and Applications of a Beer Mat for Pub Interaction
Andreas Butz and Michael Schmitz
Poster at The Seventh International Conference on Ubiquitous Computing (Ubicomp 2005), September 11—14, 2005, Tokyo, Japan
[ DOWNLOAD NOTE ] [BibTeX] show abstract
We describe the design of an interactive beer mat for the support of entertainment activities in pubs. The mat uses a gravity sensor to sense motion and orientation in space and a pressure sensor to sense the weight resting on it. Care was also taken to preserve its original functions, such as absorbing superfluous liquids and providing advertising space. We present a number of activities supported by the mat and show how it can improve the profit of the pub and the mood of the crowd simultaneously.

Geo Referenced Dynamic Bayesian Networks for User Positioning on Mobile Systems
Boris Brandherm, Tim Schwartz
Proceedings of the International Workshop on Location- and Context-Awareness (LoCA 2005), Munich, Germany, 2005
[ DOWNLOAD NOTE ] [BibTeX] show abstract
The knowledge of the position of a user is valuable for a broad range of applications in the field of pervasive computing. Different techniques have been developed to cope with the problem of uncertainty, noisy sensors, and sensor fusion. In this paper we present a method, which is efficient in time- and space-complexity, and that provides a high scalability for in- and outdoor-positioning. The so-called geo referenced dynamic Bayesian networks enable the calculation of a user’s position on his own small hand-held device (e.g., Pocket PC) without a connection to an external server. Thus, privacy issues are considered and completely in the hand of the user.

GUMO - the General User Model Ontology
Dominik Heckmann, Tim Schwartz, Boris Brandherm, Michael Schmitz and Margeritta von Wilamowitz-Moellendorff
Proceedings of the 10th International Conference on User Modeling (UM'2005), Edinburgh, UK, 2005, Springer-Verlag Berlin Heidelberg, LNAI 3538, pp. 428-432
[ DOWNLOAD NOTE ] [BibTeX] show abstract
We introduce the general user model ontology GUMO for the uniform interpretation of distributed user models in intelligent semantic web enriched environments. We discuss design decisions, show the relation to the user model markup language UserML and present the integration of ubiquitous applications with the user model service U2M.

Towards explicit physical object referencing
Michael Kruppa
Doctoral Consortium at User Modeling 2005, Edinburgh
[ DOWNLOAD NOTE ] [BibTeX] show abstract
The main goal of the work presented in this paper is the realization of virtual characters capable of performing judicious combinations of speech, gesture and motion in order to disambiguate references to objects in the physical environment.

Using Physiological Signals in a User-Adaptive Personal Assistant
Boris Brandherm, Holger Schultheis, Margeritta von Wilamowitz-Moellendorff, Tim Schwartz, Michael Schmitz
Proceedings of the 11th International Conference on Human-Computer Interaction (HCII-2005), Las Vegas, Nevada, USA, 2005
[BibTeX] show abstract
Since psychophysiological signals are continuously available and usually quickly reflect changes of the user’s state, they constitute an important source of information for adaptive assistance systems. Despite their potential benefits however, physiological information is frequently neglected in current adaptive systems. This may—at least partly—be due to the fact that physiological measures cannot be easily used for adaptation. Instead several steps have to be taken to be able to draw on the advantages of physiology. First, each measure has to be evaluated regarding its suitability to distinguish between user states. Second, the exact relationships between physiological measures and states need to be identified. Finally, psychophysiological information has to be integrated over time and with other sources of information. In the scope of the BAIR project all three sub problems have been tackled and the respective solutions have been combined to give a systematic approach for the utilization of physiological information in user-adaptive personal assistance systems.

Integrating Intra and Extra Gestures into a Mobile and Multimodal Shopping Assistant
Rainer Wasinger, Antonio Krüger, Oliver Jacobs
Proc. of the 3rd International Conference on Pervasive Computing (Pervasive), 2005, pp. 297-314.
[ DOWNLOAD NOTE ] [BibTeX] show abstract
Accompanying the rise of mobile and pervasive computing, applications now need to adapt to their surrounding environments and provide users with information in the environment in an easy and natural manner. In this paper we describe a user interface that integrates multimodal input on a handheld device with external gestures performed with real world artifacts. The described approach extends reference resolution based on speech, handwriting and gesture to that of real world objects that users may hold in their hands. We discuss the varied interaction channels available to users that arise from mixing and matching input modalities on the mobile device with actions performed in the environment. We also discuss the underlying components required in handling these extended multimodal interactions and present an implementation of our ideas in a demonstrator called the Mobile ShopAssist. This demonstrator is then used as the basis for a recent usability study that we describe on user interaction within mobile contexts.

Tangible UIs for Media Control - Probes Into the Design Space
Andreas Butz, Michael Schmitz, Antonio Krüger and Harald Hullmann
in extended Proceedings of CHI 2005, April 2 - 7, 2005, Portland, Oregon, USA
[ DOWNLOAD NOTE ] [BibTeX] show abstract
In a student project over the summer of 2004 teams of computer science and product design students worked together to develop new forms of interfaces for media control in living room contexts. In this paper we des-cribe the design process from collecting first ideas of design choices and iteratively evolving (low-fidelity) prototypes to fully functional products, partially even meeting mass production requirements. We discuss how the interdisciplinary collaboration influenced the creative process in such a way, that the solutions were more realistic than purely design-informed solutions and more inspired than purely technology-informed ones. We experienced that the combination of skills lead to a much more focused design process, which produced fully functional prototypes in a short time. The resulting designs include one interface installed in the room, two autonomous interaction objects which can be freely moved around, and a two-handed inter-face. While these are only small spotlights into a large design space, they nicely show the possible diversity. We also learned that fully functional and aesthetically pleasing prototypes can be developed with techno-logically relatively simple means.

Seamless User Notification in Ambient Soundscapes (IUI 2005)
Andreas Butz , Ralf Jung
Proceedings of IUI 2005, January 10-13, 2005, San Diego, California, USA
[ DOWNLOAD NOTE ] show abstract
We describe a method for notifying users through auditory cues embedded in an ambient soundscape in the environment. It uses pieces of music which are composed in such a way, that particular instruments or motifs can be added or omitted without losing the aesthetic quality of the overall composition. This allows for very subtle modifications in the soundscape which are only noticed by those users who have chosen this particular instrument or motif as their notification instrument before. As a side effect, the soundscape itself can be used to subtly influence the mood of users. The method has been implemented in a prototype, which we briefly discuss. The prototype is implemented using a spatial audio framework and can hence notify users from particular directions.

The Virtual Room Inhabitant
Michael Kruppa, Lübomira Spassova, Michael Schmitz
2nd Workshop on Multi-User and Ubiquitous User Interfaces (MU3I)
[pdf] [BibTeX] show abstract
In this paper we describe a new way to improve the usability of complex hardware setups in Instrumented Environments (IEs). By introducing a virtual character, we facilitate intuitive interaction with our IE. The character is capable of freely moving along the walls of the room. In this way, it may offer situated assistance to users within the environment. We make use of a steerable projector and a spatial audio system, in order to position the character within the environment. Our concept of a virtual character “living” within the IE, and thus playing the role of an assistant, allows both novice and advanced users to efficiently interact with the different devices integrated within the IE. The character is capable of welcoming a first time visitor and its main purpose is to explain the setup of the environment and to help users while interacting with it.

Adaptive multimodal presentation of multimedia content in museum scenarios
Michael Kruppa, Dominik Heckmann, Antonio Krüger
KI Journal, volume 01/05, pp 56-59
[ DOWNLOAD NOTE ] [BibTeX] show abstract
In this article we will present the concepts of a user-adaptive and context-sensitive multimedia presentation guide for museums. The focus of our approach lies on an architecture that facilitates the adaptive presentation of multimedia content on mobile and stationary devices. Life-like characters are used to maintain the coherence of the presentation, even if the presentation medium is changing. Different types of characters convey different kinds of information. The content is selected according to the actual situation of users, which not only includes the position and orientation in the museum, but also the history of their visit and their interests. A very general user modelling framework is used to request and update information on users. This framework on the one hand allows users to control and modify personal information, and on the other hand facilitates to share the user model with other applications, e.g. with other museums. Finally, we will share our experiences that we have gained in two different museum settings.

A Survey of Map-based Mobile Guides
J. Baus, K. Cheverst, and C. Kray
Map-based mobile services - Theories, Methods and Implementations Meng/Zipf (Hrsg.), Springer, S. 197-213
[ DOWNLOAD NOTE ]

Resource-Adaptive Personal Navigation
Jörg Baus, Antonio Krüger, Christoph Stahl
Chapter in: O. Stock and M. Zancanaro (eds.), Multimodal Intelligent Information Presentation, pages 71-93, Springer, 2005, ISBN: 1-4020-3049-5.

Estimating the Acoustic Context to Improve Speaker Classification.
Christian Müller
In Brevia proceedings of the Fifth International and Interdisciplinary Conference on Modeling and Using Context (CONTEXT-05). Paris, France.
[ DOWNLOAD NOTE ] [BibTeX] show abstract
This paper describes, how an existing approach on speaker classification can be extended in a way that context is taken into account. First, the main issues of this approach are summarized. With respect to context classification, preliminary results of large corpus analyses are presented. It is outlined, how knowledge about the context can be applied to improve speaker classification.

2004	Back to Top

Using Semantic Web Technology for Ubiquitous Location and Situation Modeling.
Christoph Stahl, Dominik Heckmann
The Journal of Geographic Information Sciences, CPGIS: Berkeley, Vol. 10, No. 2., December 2004, pages 157-165.
[pdf] [BibTeX] [ACM ref] show abstract
We use semantic web technology to utilize the world wide web for a large-scale ubiquitous (qualitative and geometric) location model, in order to achieve shareability and extensibility at low operational cost. Our intention is to model location and user characteristics, in order to realize location aware applications with a focus on pedestrian navigation in buildings and situated user interaction. We present the ubiquitous symbolic location and situational context model UbisWorld and the tool Yamamoto for the modeling of hierarchical geometrical maps.

Fluid Beam - A Steerable Projector and Camera Unit
Lübomira Spassova
Student and Newbie Colloquium at ISWC/ISMAR
[pdf] [BibTeX] show abstract
This paper presents an approach of using a steerable projector and camera unit in order to augment instrumented environments with projected virtual objects. The environment appears to be covered with a kind of invisible virtual layer on which images, texts, videos, computer desktops and other virtual objects can be placed. By steering the projector beam on particular surfaces the objects placed on them are made visible as if the projector were a virtual torch light.

Semantisches Web
Wolfgang Wahlster
Bullinger, H.-J. (ed.): Trendbarometer Technik: Visionäre Produkte, Neue Werkstoffe, Fabriken der Zukunft, p. 62 - 63, München, Wien: Hanser, 2004
[ DOWNLOAD NOTE ]

An Extension of the Differential Approach for Bayesian Network Inference to Dynamic Bayesian Networks
Boris Brandherm, Anthony Jameson
Special Issue "Uncertain Reasoning", International Journal of Intelligent Systems, Volume 19 Issue 8 (Part 1), 2004
[ DOWNLOAD NOTE ] [BibTeX] show abstract
We extend the differential approach to inference in Bayesian networks (BNs) (Darwiche, 2000) to handle specific problems that arise in the context of dynamic Bayesian networks (DBNs). We first summarize Darwiche's approach for BNs, which involves the representation of a BN in terms of a multivariate polynomial. We then show how procedures for the computation of corresponding polynomials for DBNs can be derived. These procedures permit not only an exact roll-up of old time slices but also a constant-space evaluation of DBNs. The method is applicable to both forward and backward propagation, and it does not presuppose that each time slice of the DBN has the same structure. It is compatible with approximative methods for roll-up and evaluation of DBNs. Finally, we discuss further ways of improving efficiency, referring as an example to a mobile system in which the computation is distributed over a normal workstation and a resource-limited mobile device.

Semantisches Web
Wolfgang Wahlster
Bullinger, H.-J. (ed.): Trendbarometer Technik: Visionäre Produkte, Neue Werkstoffe, Fabriken der Zukunft, p. 62 - 63, München, Wien: Hanser, 2004
[ DOWNLOAD NOTE ]

Special Issue: Conversational User Interfaces, it - Information Technology
Wolfgang Wahlster
6/2004, Munich, Germany: Oldenbourg, pp. 289-290
[ DOWNLOAD NOTE ]

Conversational User Interfaces
Wolfgang Wahlster
it - Information Technology, 6/2004, Munich, Germany: Oldenbourg, pp. 289-290
[ DOWNLOAD NOTE ]

Multi-modal Interaction with Mobile Navigation Systems
Rainer Wasinger, Antonio Krüger
W. Wahlster (Ed.): Special Journal Issue "Conversational User Interfaces", it - Information Technology 46 (2004) 6, München: Oldenbourg Wissenschaftsverlag (ISSN 1611-2776), 2004, pp. 322-331.
[BibTeX] show abstract
Navigation systems have seen significant advancements in recent years. They now exist for desktop computers, automobiles, and mobile devices. They are able to adapt to many different situational contexts and now provide a solid foundation for a wide range of location based services. Mobile multi-modal interfaces have seen similar advancements over the past few years as technologies previously designed for powerful mainframe computers are gradually swept into the reach of mobile devices. This paper describes a multi-modal user interface that has been integrated into a pedestrian navigation service called the Personal Navigator, and a closely interleaved shopping assistant called the Mobile ShopAssist. We describe the wide range of input interaction and output presentation that we believe mobile systems should provide and describe the importance for context-aware systems to adapt their user interfaces to the surrounding environment.

SearchLight - A Lightweight Search Function for Pervasive Environments
Andreas Butz, Michael Schneider, Mira Spassova
Pervasive Computing, Second International Conference
[pdf] [BibTeX] show abstract
We present a lightweight search function for physical objects in instrumented environments. Objects are tagged with optical markers which are scanned by a steerable camera and projector unit on the ceiling. The same projector can then highlight the objects when given the corresponding marker ID. The process is very robust regarding calibration, and no 3D model of the environment is needed. We discuss the scenario of finding books in a library or office environment and several extensions currently under development.

The Connected User Interface: Realizing a Personal Situated Navigation Service
Antonio Krüger, Andreas Butz, Christian Müller, Christoph Stahl, Rainer Wasinger, Karl-Ernst Steinberg, Andreas Dirschl
In Proceedings of the 9th International Conference on Intelligent User Interfaces (IUI 2004), ACM Press, 2004, pp. 161-168.
[ DOWNLOAD NOTE ] [BibTeX] show abstract
Navigation services can be found in different situations and contexts: while using the web with a desktop PC, in cars, and more recently also when being a pedestrian on PDAs. These services are usually well designed for their specific purpose, but fail to work in other situations. In this paper we present an approach that connects a variety of specialized user interfaces to achieve a personal navigation service spanning different situations. We describe the concepts behind the BPN (BMW Personal Navigator), an entirely implemented system that combines a desktop event and route planner, a car navigation system, and a multi-modal, in- and outdoor pedestrian navigation system for a PDA. Rather than designing for one unified UI, we focus on connecting specialized UIs for desktop, in-car and on-foot use.

The Museum Visit: Generating Seamless Personalized Presentations on Multiple Devices
Cesare Rocchi, Oliviero Stock, Massimo Zancanaro, Michael Kruppa, Antonio Krüger
Proceedings of IUI 2004, pp. 316-318
[ DOWNLOAD NOTE ] [BibTeX] show abstract
The issue of the seamless interleaving of interaction with a mobile device and stationary devices is addressed, in a typical situation of educational entertainment: the visit to a museum. Some of the salient elements of the described work are the emphasis on multimodality in the dynamic presentation and coherence throughout the visit. The adopted metaphor is of a kind of contextualized TV-like presentation, useful for engaging (young) visitors. On the mobile device, personal video clips are dynamically generated from personalized verbal presentations; on larger stationary screens distributed throughout the museum, further background material and additional information is provided. A virtual presenter follows the visitors in their experience and gives advice on both types of devices and on the museum itself.

2003	Back to Top

Proceedings of the 5th International Conference on Multimodal Interfaces
Wolfgang Wahlster, Sharon L. Oviatt, Trevor Darrell, Mark T. Maybury,
ICMI 2003, Vancouver, British Columbia, Canada, November 5-7, 2003. ACM 2003, ISBN 1-58113-621-8

Towards Symmetric Multimodality: Fusion and Fission of Speech, Gesture, and Facial Expression
Wolfgang Wahlster
Günter, A., Kruse, R., Neumann, B. (eds.): KI 2003: Advances in Artificial Intelligence. Proceedings of the 26th German Conference on Artificial Intelligence, September 2003, Hamburg, Germany, Pages 1 - 18, Berlin, Heidelberg: Springer, LNAI 2821, 2003
[ DOWNLOAD NOTE ] show abstract
We introduce the notion of symmetric multimodality for dialogue systems in which all input modes (eg. speech, gesture, facial expression) are also available for output, and vice versa. A dialogue system with symmetric multimodality must not only understand and represent the user's multimodal in-put, but also its own multimodal output. We present the SmartKom system, that provides full symmetric multimodality in a mixed-initiative dialogue system with an embodied conversational agent. SmartKom represents a new generation of multimodal dialogue systems, that deal not only with simple modality inte-gration and synchronization, but cover the full spectrum of dialogue phenomena that are associated with symmetric multimodality (including crossmodal refer-ences, one-anaphora, and backchannelling). We show that SmartKom's plug-an-play architecture supports multiple recognizers for a single modality, eg. the user's speech signal can be processed by three unimodal recognizers in parallel (speech recognition, emotional prosody, boundary prosody). Finally, we detail SmartKom's three-tiered representation of multimodal discourse, consisting of a domain layer, a discourse layer, and a modality layer.

M3I in a Pedestrian Navigation & Exploration System
Rainer Wasinger, Christoph Stahl, Antonio Krüger
5th International Symposium on Human Computer Interaction with Mobile Devices (Mobile HCI 2003), Udine, Italy, September 8-11, 2003, Springer: Lecture Notes in Computer Science, Vol. 2795, Chittaro, Luca (Ed.), ISBN: 3-540-40821-5, pp. 481-485.
[ DOWNLOAD NOTE ] [BibTeX] show abstract
In this paper, we describe a near-complete Pocket PC implementation of a mobile multi-modal interaction (M3I) platform for pedestrian navigation. The platform is designed to easily support indoor and outdoor navigation tasks, and uses several modalities for the presentation and user input. Whereas 2D/3D-graphics and synthesized speech are used to present useful information on routes and places, embedded speech and gesture recognition allow for situated user interaction.

Seamless Personalized TV-like Presentations on Mobile and Stationary Devices in a Museum
Michael Kruppa , Antonio Krüger, Cesare Rocchi, Oliviero Stock, Massimo Zancanaro
Proceedings of ICHIM 2003, Paris
[ DOWNLOAD NOTE ] [BibTeX] show abstract
Museums are noncompetitive environments where visitors are free to move around and learn concepts in connection to the objects exhibited. Presentations have to be engaging and rich. In this paper, we describe work aimed at a kind of contextualized tv-like presentation with the goal of making the museum visits more appealing in particular for young visitors. On the mobile device, personal video-clips are dynamically generated from personalized verbal presentations; on larger stationary screens distributed throughout the museum (so-called Virtual Windows), further background material and additional information is provided. A virtual presenter follows the visitors in their experience and provides advice on both types of devices and on the museum itself. Recent works have presented a multimedia mobile guides based on a cinematic metaphor. The language of cinematography, including shot segmentation, camera movements and transition effects, is employed in order to plan the animation and to synchronize the visual and the verbal parts of the presentation. In building the animations, a set of strategies similar to those used in documentaries were employed. Different methods to improve the usability of PDA, featuring small displays, by using them in combination with large displays have been presented. In this work, we have focused on automatically produced video-clips to be played on the small screen of the mobile device and using a life-like character either as an anchorman or a presenter. The life-like character plays the role of an accompanying agent, ready to move on the mobile device or to jump on the Virtual Windows, in order to provide continuous assistance and continuity to the presentation. The character helps in solving problems like how to reach a certain exhibit, and yielding explanations. User evaluations have shown that the introduction of a life-like character makes presentations more enjoyable and attractive (something that we regard as very important to keep younger visitors engaged). This work has been conducted in the context of PEACH project that has the objective of studying and experimenting with various advanced technologies that can enhance cultural heritage appreciation. Experimentations are carried on in Torre Aquila, a medieval tower in Trento, and in the premises of Völklingen Old Ironworks, a cultural heritage site dedicated to iron and steel industry in Saarbrücken.

Robust speech interaction in a mobile environment through the use of multiple and different media types
Rainer Wasinger, Christoph Stahl, Antonio Krüger
8th European Conference on Speech Communication and Technology (EUROSPEECH 2003 - INTERSPEECH 2003), Geneva, Switzerland, September 1-4, 2003, ISCA Archive, pp. 1049-1052.
[ DOWNLOAD NOTE ] [BibTeX] show abstract
Mobile and outdoor environments have long been out of reach for speech engines due to the performance limitations that were associated with portable devices, and the difficulties of processing speech in high-noise areas. This paper outlines an architecture for increasing speech recognition rates in a mobile pedestrian indoor/outdoor navigation environment, through the use of a media fusion knowledge component.

Spinning the Semantic Web. Bringing the World Wide Web to Its Full Potential
Wolfgang Wahlster, D. Fensel, J. Hendler, H. Lieberman
Spinning the Semantic Web. Bringing the World Wide Web to Its Full Potential
[ DOWNLOAD NOTE ]

SmartKom: Symmetric Multimodality in an Adaptive and Reusable Dialogue Shell
Wolfgang Wahlster
Krahl, R., Günther, D. (eds.): Proceedings of the Human Computer Interaction Status Conference 2003, June 2003, Berlin: DLR, pp. 47-62
[ DOWNLOAD NOTE ] show abstract
We introduce the notion of symmetric multimodality for dialogue systems in which all input modes (eg. speech, gesture, facial expression) are also available for output, and vice versa. A dialogue system with symmetric multimodality must not only understand and represent the user's multimodal input, but also its own multimodal output. We present the SmartKom system, that provides full symmetric multimodality in a mixed-initiative dialogue system with an embodied conversational agent. SmartKom represents a new generation of multimodal dialogue systems, that deal not only with simple modality integration and synchronization, but cover the full spectrum of dialogue phenomena that are associated with symmetric multimodality (including crossmodal references, one-anaphora, and backchannelling). We show that SmartKom's plug-an-play architecture supports multiple recognizers for a single modality, eg. the user's speech signal can be processed by three unimodal recognizers in parallel (speech recognition, emotional prosody, boundary prosody). Finally, we detail SmartKom's three-tiered representation of multimodal discourse, consisting of a domain layer, a discourse layer, and a modality layer. To conclude, we discuss the economic and scientific impact of the SmartKom project, that has lead to more than 50 patents and 29 spin-off products.

A User Modeling Markup Language (UserML) for Ubiquitous Computing
Dominik Heckmann and Antonio Krueger
In Peter Brusilovsky, Albert Corbett, Fiorella de Rosis Eds., UM 2003, LNAI 2702: Proceedings of the Ninth International Conference on User Modeling, Berlin Heidelberg: Springer, pp. 393-397 (Best Poster Award)
[ DOWNLOAD NOTE ] [BibTeX]

An Extension of the Differential Approach for Bayesian Network Inference to Dynamic Bayesian Networks
Boris Brandherm
Proceedings of the Sixteenth International FLAIRS Conference (FLAIRS-03), St. Augustine, Florida, USA, 2003
[ DOWNLOAD NOTE ] [BibTeX] show abstract
We extend the differential approach to inference in Bayesian networks (BNs) to handle specific problems that arise in the context of dynamic Bayesian networks (DBNs). We first summarize Darwiche's approach for BNs, which involves the representation of a BN in terms of a multivariate polynomial. We then show how procedures for the computation of corresponding polynomials for DBNs can be derived. These procedures permit not only an exact roll-up of old time slices but also a constant-space evaluation of DBNs. The method is applicable to both forward and backward propagation, and it does not presuppose that each time slice of the DBN has the same structure. It is compatible with approximative methods for roll-up and evaluation of DBNs.

Introduction to the Semantic Web
Wolfgang Wahlster, D. Fensel, J. Hendler, H. Lieberman
Fensel, D, Hendler, J. Lieberman, H., Wahlster, W. (eds.) (2003): Spinning the Semantic Web. Bringing the World Wide Web to Its Full Potential. Cambridge: MIT Press 2003, pp. 1 - 25
[ DOWNLOAD NOTE ]

Concepts for a combined use of Personal Digital Assistants and large remote displays
Michael Kruppa, Antonio Krüger
Proceedings of SimVis 2003, Pages 349-361, Magdeburg
[ DOWNLOAD NOTE ] [BibTeX] show abstract
We give an overview on different methods to improve the usability of Personal Digital Assistants, featuring small displays, by using them in combination with larger displays. We discuss several different methods that have been implemented and evaluated in former projects. Subsequently we categorize these projects i.e. the implemented methods to develop a classification of combination methods for small PDA-Displays and large remote displays. We extend the classification through new methods. Some of these new methods are implemented in a prototype to evaluate their usability in a museum scenario.

Implicit Feedback for User-Adaptive Systems by Analyzing the User's Speech
Wittig, F. and Müller, C.
In ABIS-03, Adaptivität und Benutzermodellierung in interaktiven Softwaresystemen. Karlsruhe, Germany
[ DOWNLOAD NOTE ] [BibTeX] show abstract
This paper describes an approach to recognize the gender and age of a user on the basis of her/his speech. Within a two-level framework, machine learning techniques are applied to learn several appropriate classifiers whose results are combined on the top level to improve the overall classification of the user. As a part of the user model, the information about the user’s age and gender can be an important basis for appropriate adaptation decisions.

Speech as a Source for Ubiquitous User Modeling
Müller, C. and Wittig, F.
In UM2003, User Modeling: Proceedings of the Ninth International Conference. Pittsburgh, USA
[ DOWNLOAD NOTE ] [BibTeX] show abstract
In this paper, we present an approach on how to use speech as a source for user modeling in a mobile and ubiquitous context. In particular, we exploit different abstraction levels of speech features to estimate the user’s age and gender. To solve the classification task, we compared several well known machine learning techniques such as artificial neural networks and support vector machines. The results of our study imply that one can indeed successfully extract higher level information from the raw speech data. We show how this approach is integrated into a generic resource adaptive system architecture. One particular instance of this system is an implementation of a mobile pedestrian navigation system.

Ressourcenadaptierende hybride Personennavigation
Jörg Baus
Dissertationen zur Künstlichen Intelligenz (DISKI), Band 268, ISBN 3-89838-268-0

Exploiting Speech for Recognizing Elderly Users to Respond to their Special Needs
Müller, C., Wittig, F., and Baus, J.
In Proceedings of the Eighth European Conference on Speech Communication and Technology (Eurospeech 2003). Geneva, Switzerland
[ DOWNLOAD NOTE ] [BibTeX] show abstract
In this paper we show how to exploit raw speech data to gain higher level information about the user in a mobile context. In particular we introduce an approach for the estimation of age and gender using well known machine learning techniques. On the basis of this information, systems like for example a mobile pedestrian navigation system, can be made adaptive to the special needs of a specific user group (here the elderly). First we provide a motivation why we consider such an adaptation as necessary, then we outline some adaptation strategies that are adequate for mobile assistants. The major part of the paper is about (a) identifying and extracting features of speech that are relevant for age and gender estimation and (b) classifying a particular speaker, treating uncertainty, and updating the user model over time. Finally we provide a short outlook on current work.

2002	Back to Top

Forschen für die Internet-Gesellschaft: Trends, Technologien, Anwendungen
Wolfgang Wahlster, C. Weyrich
BDI, FhG: Berlin, München 2002, 37 p
[ DOWNLOAD NOTE ]

Efficient Modeling of Temporally Variable User Properties With Dynamic Bayesian Networks
Boris Brandherm
Proceedings of the Eighteenth National Conference on Artificial Intelligence (AAAI-02), Doctoral Consortium, Edmonton, Alberta, Canada, 2002
[ DOWNLOAD NOTE ] [BibTeX] show abstract
Efficiency issues are considered to handle dynamic Bayesian networks which estimate a user's resource limitations on the basis of symptomatic evidence in order to realize a situational presentation of information. Depending on the kind of data and on the frequency it arrives different methods of preprocessing this data must be applied to incorporate it as evidence in the DBN.

Towards User Modeling in Ubiquitous Computing
Dominik Heckmann
In Artificial Intelligence in Mobile System (AIMS-2002),in conjunction with (ECAI-2002), Lyon, France, pages 58-65
[ DOWNLOAD NOTE ] [BibTeX] show abstract
This paper shows some ideas on the current progress in merging user modeling with ubiquitous computing and mobile computing. This work is under progress. The selected challenge is "to enable constant user modeling with different systems." The possible roles of mobile devices are addressed. A common language to exchange parts of the user models, as well as user behavioral data and inference procedures must be defined. A communication architecture is needed and is under development.

Empirically Grounded Decision-Theoretic Adaptation to Situation-Dependent Resource Limitations
Thorsten Bohnenberger, Boris Brandherm, Barbara Großmann-Hutter, Dominik Heckmann, Frank Wittig
Special Issue "Adaptivity and User Modeling", KI-Journal, Vol.3, 2002, pages 10-16
[ DOWNLOAD NOTE ] [BibTeX] show abstract
This article summarizes research on several interrelated general issues that can arise in the design and development of user modeling systems: the learning and subsequent adaptation of general user models on the basis of empirical data; the modeling of temporally variable properties of users, in particular time pressure and cognitive load; and the user-adaptive planning of interactions under uncertainty. The methods and results are integrated and illustrated with a prototype of a mobile assistance system for travelers in an airport.

Positionsabhängige Kartengenerierung für mobile Fußgängernavigationssyteme
A. Krüger, C. Stahl, J. Baus
Simulation und Visualisierung, Magdeburg 28.Feb.-1. März 2002. SCS-Verlag, ISBN 1-56555-234-2, ISBN 3-936150-15-x.
[ DOWNLOAD NOTE ] show abstract
Navigationssysteme werden in Zukunft den Sprung vom Auto in die Westentasche schaffen und so auch Fußgänger in den Genuß der personalisierten Wegbeschreibung kommen lassen. Dabei wird neben der verbalen Beschreibung der Route insbesondere geeignete Graphiken zum Einsatz kommen. Diese Graphiken müssen in Abhängigkeit einer Vielzahl von Parametern erstellt werden, so daß eine dynamische Generierung der Graphiken an dieser Stelle sehr sinnvoll ist. Der vorliegende Artikel befaßt sich daher mit der Frage wie inkrementelle 2D-Wegbeschreibungen aus einem 3D-Modell, auf den jeweilgen Beutzerkontext zugeschnitten, generiert werden können.

A ResourceAdaptive Mobile Navigation System
Jörg Baus, Antonio Krüger, Wolfgang Wahlster
Proceedings of IUI2002: International Conference on Intelligent User Interfaces 2002, ACM Press, New York
[ DOWNLOAD NOTE ] show abstract
The design of mobile navigation systems adapting to limited resources will be an important future challenge. Since typically several different means of transportation have to be combined in order to reach a destination, the user interface of such a system has to adapt to the user’s changing situation. This applies especially to the alternating use of different technologies to detect the user’s position, which should be as seamless as possible. This article presents a hybrid navigation system that relies on different technologies to determine the user’s location and that adapts the presentation of route directions to the limited technical resources of the output device and the limited cognitive resources of the user.

Frames of Reference, Positional Information and Navigational Assistance
Jörg Baus, Christian Kray
Proceedings of Flairs 2002, Menlo Park: AAAI Press
[ DOWNLOAD NOTE ] show abstract
We present an analysis of different types of frames of reference and what they are needed for in a setting of a system providing navigational assistance. Since the position of a user of such a system is a main factor in the establishment of different frames of reference, we review different technologies for measuring positional information, and present a qualitative classification for this information. Based on this and on the previous analysis, we then examine the influence of positional information and frames of reference on typical tasks that arise in the context of navigational assistance.

Positionsinformation und Navigationsaufgaben
Christian Kray, Jörg Baus und Antonio Krüger
Geoinformation mobil Zipf/Strobl (Hrsg.), Herbert Wichmann Verlag, Heidelberg
[ DOWNLOAD NOTE ]

2001	Back to Top

REAL: Ein ressourcenadaptierendes mobiles Navigationssystem
Wolfgang Wahlster, J. Baus, Ch. Kray, A. Krüger
Forschung und Entwicklung, November 2001, Vol. 16, 4
[ DOWNLOAD NOTE ]

Ubiquitous User Modeling for Situated Interaction
Dominik Heckmann
In M. Bauer, P. Gmytrasiewicz, J. Vassileva Eds., User Modeling: Proceedings of the Eighth International Conference (UM2001), Berlin: Springer, Sonthofen, Germany, LNCS 2109: Springer, Berlin Heidelberg, pages 280-282
[ DOWNLOAD NOTE ] [BibTeX] show abstract
The main contribution of my doctoral proposal will be the design of a standardized and expandable XML-based User Modeling Ontology Language, which enables ubiquitous systems to communicate about user models. The second contribution will be the investigation of combining simple partial user models from the point of view of the user modeling ontology language, as well as the specific example domain of speech and manual input, which will be realized by object-oriented dynamic Bayesian networks.

SmartKom: Towards Multimodal Dialogues with Anthropomorphic Interface Agents
Wolfgang Wahlster, A. Blocher, N. Reithinger
Wolf, G., Klein, G. (eds.), Proceedings of International Status Conference "Human-Computer Interaction", DLR, Berlin, Germany, October 2001, pp. 23 - 34
[ DOWNLOAD NOTE ] show abstract
SmartKom is a multimodal dialogue system that combines speech, gesture, and facial expressions for input and output. SmartKom provides an anthropomorphic and affective user interface through its personification of an interface agent. Understanding of spontaneous speech is combined with video-based recognition of natural gestures and facial expressions. One of the major scientific goals of SmartKom is to design new computational methods for the seamless integration and mutual disambiguation of multimodal input and output on a semantic and pragmatic level. SmartKom is based on the situated delegation-oriented dialogue paradigm, in which the user delegates a task to a virtual communication assistant, visualized as a life-like character on a graphical display. SmartKom is a ultilingual system that analyses and generates German and English utterances. We describe the SmartKom architecture, the use of an XML-based mark-up language for multimodal content, and the most distinguishing features of the fully operational SmartKom 2.0 system.

SmartKom: Multimodal Communication with a Life-Like Character
Wolfgang Wahlster, A. Blocher, N. Reithinger
In: Proceedings of Eurospeech 2001, 7th European Conference on Speech Communication and Technology, Aalborg, Denmark, September 2001, Vol. 3, pp. 1547 - 1550
[ DOWNLOAD NOTE ] show abstract
SmartKom is a multimodal dialog system that combines speech, gesture, and mimics input and output. Spontaneous speech understanding is combined with the video-based recognition of natural gestures. One of the major scientific goals of SmartKom is to design new computational methods for the seamless integration and mutual disambiguation of multimodal input and output on a semantic and pragmatic level. SmartKom is based on the situated delegation-oriented dialog paradigm, in which the user delegates a task to a virtual communication assistant, visualized as a life-like character on a graphical display. We describe the SmartKom architecture, the use of an XML-based mark-up language for multimodal content, and some of the distinguishing features of the first fully operational SmartKom demonstrator.

Robust Translation of Spontaneous Speech: A Multi-Engine Approach
Wolfgang Wahlster
Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, Seattle, Washington, August 2001, San Francisco: Morgan Kaufmann, Vol. 2, pp. 1484 - 1493
[ DOWNLOAD NOTE ] show abstract
Verbmobil is a speaker-independent and bidirectional speech-to-speech translation system for spontaneous dialogs that can be accessed via GSM mobile phones.It handles dialogs in three business-oriented domains, with context-sensitive translation between four languages (English, German, Japanese, and Chinese). We show that in Verbmobil's multi-blackboard and multi-engine architecture the results of concurrent processing threads can be combined in an incremental fashion. We argue that all results of concurrent processing modules must come with a confidence value, so that statistically trained selection modules can choose the most promising result. Packed representations together with formalisms for underspecification capture the uncertainties in each processing phase, so that the uncertainties can be reduced by linguistic, discourse and domain constraints as soon as they become applicable. Distinguishing features like the multilingual prosody module and the generation of dialog summaries are highlighted. We conclude that Verbmobil has successfully met the project goals with more than 80% of approximately correct translations and a 90% success rate for dialog tasks. One of the main lessons learned from the Verbmobil project is that the problem of speech-tospeech translation can only be cracked by the combined muscle of deep and shallow processing approaches.

Some Remarks on Automated Sketch Generation for Mobile Route Descriptions
Jörg Baus, Andreas Butz, Antonio Krüger, and M. Lohse
Proceedings of the 1st Symposium on Smart Graphics, ACM Press, New York , 2001
[ DOWNLOAD NOTE ] show abstract
The next generation of route description systems will not only give way information at the beginning, but also during the way-finding task. The information will be brought to the user by stationary and mobile displays (e.g. info kiosk or PDA). Especially graphics play an important role to convey way finding information. In this paper we explain an approach, to generate such graphics that are tailored to the use on mobile devices.

Recognizing Time Pressure and Cognitive Load on the Basis of Speech: An Experimental Study
Müller, C., Großmann-Hutter, B., Jameson, A., Rummer, R., and Wittig, F.
In UM2001, User Modeling: Proceedings of the Eighth International Conference. New York - Berlin: Springer
[ DOWNLOAD NOTE ] [BibTeX] show abstract
In an experimental environment, we simulated the situation of a user who gives speech input to a system while walking through an airport. The time pressure on the subjects and the requirement to navigate while speaking were manipulated orthogonally. Each of the 32 subjects generated 80 utterances, which were coded semi-automatically with respect to a wide range of features, such as filled pauses. The experiment yielded new results concerning the effects of time pressure and cognitive load on speech. To see whether a system can automatically identify these conditions on the basis of speech input, we had this task performed for each subject by a Bayesian network that had been learned on the basis of the experimental data for the other subjects. The results shed light on the conditions that determine the accuracy of such recognition.

Die Selektion dimensionaler räumlicher Präpositionen automatisch und nicht ressourcenadatierend
H.D. Zimmer, H. Spieser, and J. Baus
Kognitionswissenschaft, Band 9, Heft 3, Oktober 2001, S. 115-121
[ DOWNLOAD NOTE ]

A Hybrid Indoor Navigation System
Andreas Butz, Jörg Baus, Antonio Krüger, and M. Lohse
Proceedings of IUI2001: International Conference on Intelligent User Interfaces 2001, ACM Press, New York
[ DOWNLOAD NOTE ] show abstract
We describe a hybrid building navigation system consisting of stationary information booths and a mobile communication infrastructure feeding small portable devices. The graphical presentations for both the booths and the mobile devices are generated from a common source and for the common task of way finding, but they use different techniques to convey possibly different subsets of the relevant information. The form of the presentations is depending on technical limitations of the output media, accuracy of location information, and cognitive restrictions of the user. We analyze what information needs to be conveyed, how limited resources influence the presentation of this information, and argue, that by generating all different presentations in a common framework, a consistent appearance across devices can be achieved and that the different device classes can complement each other in facilitating the navigation task.

2000	Back to Top

Information Processing in Health Care at the Start of the 3rd Millennium: Perspectives from the Viewpoint of Artificial Intelligence
Wolfgang Wahlster
Journal Methods of Information in Medicine, No. 6, 2000, pp. 9 -14

Mobile Speech-to-Speech Translation of Spontaneous Dialogs: An Overview of the Final Verbmobil System
Wolfgang Wahlster
Wahlster (ed.) Verbmobil: Foundations of Speech-to-Speech Translation. Berlin, Heidelberg, New York, Barcelona, Hong Kong, London, Milan, Paris, Singapore, Tokyo: July 2000, Springer. pp. 3-21
show abstract
Verbmobil is a speaker-independent and bidirectional speech-to-speech translation system for spontaneous dialogs in mobile situations. It recognizes spoken input, analyses and translates it, and finally utters the translation. The multilingual system handles dialogs in three business-oriented domains, with context-sensitive translation between three languages (German, English, and Japanese). Since Verbmobil emphasizes the robust processing of spontaneous dialogs, it poses difficult challenges to human language technology, that we discuss in this paper. We present Verbmobil as a hybrid system incorporating both deep and shallow processing schemes. We describe the anatomy of Verbmobil and the functionality of its main components. We discuss Verbmobil’s multi-blackboard architecture that is based on packed representations at all processing stages. These packed representations together with formalisms for underspecification capture the non-determinism in each processing phase, so that the remaining uncertainties can be reduced by linguistic, discourse and domain constraints as soon as they become applicable. We present Verbmobil’s multi-engine approach, eg. its use of five concurrent translation engines: statistical translation, case-based translation, substring-based translation, dialog-act based translation, and semantic transfer. Distinguishing features like the multilingual prosody module and the generation of dialog summaries are highlighted. We conclude that Verbmobil has successfully met the project goals with more than 80% of approximately correct translations and a 90% success rate for dialog tasks.

Facts and Figures about the Verbmobil Project
Wolfgang Wahlster, R. Karger
Wahlster (ed.) Verbmobil: Foundations of Speech-to-Speech Translation. Berlin, Heidelberg, New York, Barcelona, Hong Kong, London, Milan, Paris, Singapore, Tokyo: July 2000, Springer pp. 22-30
show abstract
In this chapter the organizational and funding structure of the Verbmobil project is summarized and the major technical data about the final Verbmobil system and the Verbmobil archives are compiled.

Verbmobil: Foundations of Speech-to-Speech Translation
Wolfgang Wahlster
Berlin, Heidelberg, New York, Barcelona, Hong Kong, London, Milan, Paris, Singapore, Tokyo: July 2000, Springer

Pervasive Speech and Language Technology
Wolfgang Wahlster
Wilhelm, R. (Ed.): Informatics - 10 Years Back, 10 Years Ahead. Berlin, Heidelberg, New York: Springer, Lecture Notes in Computer Science Vol. 2000, pp. 274- 293
[ DOWNLOAD NOTE ]

Some Aspects of Scouting Smart Environments
Jörg Baus, C. Breihof, Andreas Butz, M. Lohse, Antonio Krüger
Proceedings of the AAAI Spring Symposium on "Smart Graphics", March 20th-22nd 2000, Stanford, CA, USA .
[ DOWNLOAD NOTE ] show abstract
We discuss a virtual presenter in the form of a 3D avatar whose task it is to show people a way through a (3D model of a) public building. Our thesis is that when looking for way descriptions, seeing such a walkthrough guided by a virtual scout is easier to remember than a purely textual description of the way or the kind of annotated floor plans used widely in today's public buildings. Furthermore, we discuss some technical aspects of how such a scout can be constructed and what underlying data and processes are needed for the automated generation of guided way descriptions. One of our main goals is to give the avatar the possibility to react to her spatial environment in an appropriate matter, e.g. pointing to relevant objects and following a path that can be easily memorized by the viewer. The walkthroughs described here will be used within a building navigation system that also includes navigation via a handheld device, (such as a PDA,) once the user has left the information booth. Both the 3D visualization at the info booth and the more sketch-like presentation on the PDA are generated from the same data and by the same system, and thus can refer to each other for a much more consistent overall appearance. Last modified: Wed Dec 15 15:46:32 MET 1999

Künstliche Intelligenz: Werden Computer zu intelligenten Assistenten für jedermann
Wolfgang Wahlster
Brockhaus-Redaktion (eds.): Visionen 2000. Mannheim: Brockhaus 172- 175
[ DOWNLOAD NOTE ]

Software-Offensive mit Java, Agenten und XML
Wolfgang Wahlster
Velbert: Online Verlag, ISBN 3-89077-211-0

Smart Graphics in Adaptive Way Descriptions.
Antonio Krüger, Jörg Baus, Andreas Butz
Proceedings of: Advanced Visual Interfaces 2000 (AVI2000), Italy.
[ DOWNLOAD NOTE ] show abstract
While car navigation systems are widely commercialized already today, pedestrian information systems are still in the early research stage. However, recent progress in mobile computing has opened perspectives for pedestrian navigation systems. In this context, graphics is and will still be an important modality to convey all types of route information. This paper will address the question how to generate graphics for navigation systems that help pedestrians, e.g., airport passengers, city tourists or conference attendees, to find their way in complex environments. We will discuss how the presentation of graphics can be tailored to various technical and cognitive constraints, and we will demonstrate our ideas within a scenario where a passenger of an airport gets navigational help from a stationary info booth and afterwards on her way via a handheld device (PDA). Both the 3D visualization at the info booth and the sketch-like presentation on the PDA are generated from the same data and by the same system, yet are adapted to the specific situation, output medium and user as far as possible

Augmenting Buildings with Infrared Information
Andreas Butz., Jörg Baus, and Antonio Krüger
Proceedings of the International Symposium on Augmented Reality ISAR 2000, IEEE Computer Society Press, 10662 Los Vaqueros Circle, P.O. Box 3014, Los Alamitos, CA 9020-1264 (to appear)
[ DOWNLOAD NOTE ] show abstract
We describe a building information and navigation system based on Palm Pilot PDAs and a set of strong infrared transmitters, located throughout a building. The infrared senders stream localized data, thus effectively augmenting areas of space with localized information. This information can be perceived by just entering those areas with the PDA in your hand. We show that this form of augmentation of an environment can serve a multitude of purposes and requires neither the employment of classic 3D augmented reality nor to carry around wearable computers nor to wear head mounted displays.

1999	Back to Top

Agent-Based Multimedia Interaction for Virtual Web Pages
Wolfgang Wahlster
Intelligent User Interfaces 1999, Proceedings of the ACM Conference, Los Angeles, NY: ACM Press

Experimentelle Untersuchung von Spracheingaben unter kognitiver Belastung zur Benutzermodellbildung
Großmann-Hutter, B. and Müller, C.
In ABIS-99, Adaptivität und Benutzermodellierung in interaktiven Softwaresystemen. Magdeburg, Germany: Otto-von-Guericke University
[BibTeX]

1998	Back to Top

Recursive Matrix Systems: A Highly Parameterizable Formal Rewriting System
Dominik Heckmann
10th European Summer School in Logic, Language and Information, Student Session (ESSLLI-98), Saarbruecken, Germany, pages 149-161
[ DOWNLOAD NOTE ] [BibTeX] show abstract
A formal rewriting system is defined. It is capable of describing context­free, mildly­context­sensitive and context­sensitive languages as well as lan­ guages in between. It allows to compare formal language properties in one uniform framework. The main idea of RMS is adding a vertical dimension to the usual string deriving grammars.

Intelligente Navigation in 3D-Welten: Zur Rolle graphischer Abstraktion
Antonio Krüger, Christoph Stahl
Simulation und Visualisierung, SCS-Verlag, Magdeburg, 5.-6. März 1998.
[pdf] show abstract
Das Navigieren in komplexen 3D-Welten wird durch zwei Faktoren erheblich erschwert: Zum einen stellt die enorme Datenmenge große Anforderungen an die Hardware der Visualisierungsplattform, zum anderen wirkt der Detaillreichtum, der diesen Aufwand erforderlich macht, oft ablenkend und kann im Extremfall dazu führen, daß der Benutzer den Überblick verliert und die Navigation von einem bekannten Startpunkt aus neu starten muß. Dieser Beitrag diskutiert einen Lösungsvorschlag, der versucht, beiden Problemen gerecht zu werden. Durch die graphische Abstraktion von Modellteilen und der damit einhergehenden Vereinfachung des 3D-Modells werden Rechnerressourcen gespart und dem Navigierenden wird gleichzeitig ein Überblick ermöglicht, da potentiell ablenkende Details unterdrückt werden. Während der Navigation wird der Detaillierungsgrad automatisch durch das System oder interaktiv durch den Navigierenden angepaßt. Ziel dieser semi-automatischen graphischen Abstraktion ist ein auf das Navigationsziel und auf die Leistung des visualisierenden Rechners möglichst optimal zugeschnittener Abstraktionsgrad.

1997	Back to Top

Proceedings of the 35th Annual Meeting of the ACL and EACL-97
Wolfgang Wahlster, P. Cohen
San Francisco: Morgan Kaufmann

Multimodal Interfaces
Wolfgang Wahlster, S.W Oviatt
Human-Computer Interaction Journal, Vol. 12, No. 1-2, 1997

1996	Back to Top

Fortschritte der objektorientierten Softwaretechnologien
Wolfgang Wahlster
Velbert: Online Verlag, ISBN 3-89077-156-4

1995	Back to Top

Visual Grounding of Route Descriptions in Dynamic Environments. SFB 314 (VITRA), Bericht Nr. 117, Juli 1995
W. Maaß, J. Baus, J. Paul
To appear in Proceedings of the AAAI Fall Symposium on "Computational Models for Integrating Language and Vision", MIT, Cambridge, MA, USA, 1995
[ DOWNLOAD NOTE ] show abstract
A software agent who gives incremental, i.e. step-by-step, route descriptions while moving through an environment is an interesting starting-point for an integrated view on visual perception and natural language generation. We present a computational model, called MOSES. In particular we show how visual data is transformed into visuo-spatial representations. An object selection process based on visual features starts at a high-level description of objects in a synthetic three-dimensional environment. We found by experiments that incremental route descriptions can be classified by a small set of syntactic and semantic structures. By consideration of temporal constraints, visuo-spatial structures, path-related intentions, as well as rhetorical abilities of the speaker, a selection process extracts description schemata as input for the language generation process. These schemata are modeled by a modified subset of Jackendoff's conceptual semantics formalism.