Publications

Conference Proceedings

Schraffenberger, H. & van der Heide, E. (2015). Sonically Tangible Objects. In A. Clifford, M. Carvalhais & M. Verdicchio (Eds.), xCoAx 2015: Proceedings of the Third Conference on Computation, Communication, Aesthetics and X (pp. 233–248). Glasgow, Scotland, UK.

Abstract: A unique power of virtual objects is that they do not have to look, feel or behave like real objects. With this in mind, we have developed a virtual cube that is part of our real, physical environment but, unlike real objects, is invisible and non-tactile. ‘Touching’ this virtual object triggers binaural sounds that appear to originate from the exact spot where it is touched. Our initial experimentation suggests that this sound-based approach can convey the presence of virtual objects in real space and result in almost-tactile experiences. In this paper, we discuss the concept behind, implementation of and our experience with the sonically tangible cube and place our research in the context of tangible interaction, perception and augmented reality.

Schraffenberger, H. & van der Heide, E. (2014). The Real in Augmented Reality. In M. Carvalhais & M. Verdicchio (Eds.), xCoAx 2014: Proceedings of the Second Conference on Computation, Communication, Aesthetics and X (pp. 64–74). Porto, Portugal.

Abstract: What is augmented in Augmented Reality (AR)? This fundamental question has received surprisingly little attention in AR research. In this paper, we review exist- ing views and show how little consensus there is on the topic. Subsequently, we approach the question from a theoretical and technology-independent perspective that focuses on the relationships between the virtual and the real. We consider both spatial as well as content-based augmentations and distinguish between augmented environments, augmented objects, augmented humans, augmented content and augmented perception. We dis- cuss our findings and suggest possible future directions, such as research into multimodal and crossmodal AR.

Gómez-Maureira, M. A., Teunisse, C., Schraffenberger, H., & Verbeek, F. (2014). Illuminating Shadows: Introducing Shadow Interaction in Spatial Augmented Reality. Creating the Difference, 11.

Abstract: In this paper we present a new mode of interaction in ‘Spatial Augmented Reality’ (SAR) setups, using shadows as interaction input as well as display area. We claim that the combination of shadow interaction and SAR offers a novel, enjoyable and interesting way of interacting with information in a physical manner. This is especially relevant for contexts such as museum exhibits, where digital information and physical objects relate to one another. The results of our usability experiment with a zebrafish model show that users enjoy the combination of shadow interaction and SAR, as well as see a use for it in exhibition environments.

Schraffenberger, H. & van der Heide, E. (2013a). From Coexistence to Interaction: Influences Between the Virtual and the Real in Augmented Reality. In K. Cleland, L. Fisher, & R. Harley (Eds.), Proceedings of the 19th International Symposium on Electronic Art, ISEA2013 (pp. 1–3). Sydney.

Abstract: This paper examines the interaction and influences between the virtual and the real in Augmented Reality (AR). We explore how real objects can affect virtual objects and vice versa. Our work is based on heoretical considerations, a review of existing research and artworks as well as our own initial series of experiments. We argue that virtual and real objects can not only simulate influences that exist between real entities, but also influence each other in new and imaginary ways that have no equivalent in the physical world.

Schraffenberger, H. & van der Heide, E. (2013b). Towards Novel Relationships between the Virtual and the Real in Augmented Reality. In G. De Michelis, F. Tisato, A. Bene, & D. Bernini (Eds.), Arts and Technology (Vol. 116, pp. 73–80). Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering. Springer Berlin Heidelberg.

Abstract: In Augmented Reality (AR), virtual and real content coexist in the same physical environment. However, in order to create AR, solely adding virtual content to a real space does not suffice. In this paper we argue that an augmentation adds and relates something virtual to something real. Subsequently, we discuss both existing and promising future relationships between the virtual and the real. We explore what AR is and what it could possibly include from a technology-independent and conceptual point of view. By comparing our take on AR with common manifestations of AR, we identify possible directions for future research and AR (art) works, such as the use of non-visual modalities and the design of novel interactions between the virtual and the real.

Schraffenberger, H. & van der Heide, E. (2012) Interaction Models for Audience-Artwork Interaction: Current State and Future Directions. In Anthony L. Brooks (Ed.), Arts and Technology – Second International Conference, ArtsIT 2011, Esbjerg, Denmark, December 10-11, 2011, Revised Selected Papers, LNICST 101, Springer.

Abstract: Interactive art is of great relevance to the arts, sciences and technology alike. A common field of interest among researchers of different disciplines, practising artists and art institutes is the interaction between audience and artwork. This paper reviews existing research concerning interaction in interactive art and discusses its applicability for describing and classifying audience-artwork interaction. In pointing out possible future directions, we identify a need for models describing the relation between the audience’s and artwork’s actions and reactions as well as the necessity for future research looking at interaction as a continuous bi-directional process between work and audience.

de Jong, S., Jillissen, J., Kirkali, D., de Rooij, A., Schraffenberger, H. & Terpstra, A. (2010). One-press control: A tactile input method for pressure-sensitive computer keyboards, extended abstract at 28th ACM Conference on Human Factors in Computing Systems (CHI 2010), April 2010, Atlanta (USA).

Abstract: This work presents One-press control, a tactile input method for pressure-sensitive keyboards based on the detection and classification of pressing movements on the already held-down key. To seamlessly integrate the added control input with existing practices for ordinary computer keyboards, the redefined notion of virtual modifier keys is introduced. A number of application examples are given, especially to point out a potential for simplifying existing interactions by replacing modifier key combinations with single key presses. Also, a new class of interaction scenarios employing the technique is proposed, based on an interaction model named “What You Touch Is What You Get (WYTIWYG)”. Here, the proposed tactile input method is used to navigate interaction options, get full previews of potential outcomes, and then either commit to one or abort altogether – all in the space of one key depress / release cycle. The results of user testing indicate some remaining implementation issues, as well as that the technique can be learned within about a quarter of an hour of hands-on operating practice time.

Journals

Schraffenberger, H., & van der Heide, E. (2014). Everything Augmented: On the Real in Augmented Reality. Journal Of Science And Technology Of The Arts, 6(1), 17-29.

Abstract: What is augmented in Augmented Reality (AR)? In this paper, we review existing opinions and show how little consensus exists on this matter. Subsequently, we approach the question from a theoretical and technology-independent perspective. We identify spatial and content-based relationships between the virtual and the real as being decisive for AR and come to the conclusion that virtual content augments that to which it relates. Subsequently, we categorize different forms of AR based on what is augmented. We distinguish between augmented environments, augmented objects, augmented humans and augmented content and consider the possibility of augmented perception. The categories are illustrated with AR (art) works and conceptual differences between them are pointed out. Moreover, we discuss what the real contributes to AR and how it can shape (future) AR experiences. A summary of our findings and suggestions for future research and practice, such as research into multimodal and crossmodal AR, conclude the paper.

Schraffenberger, H. & van der Heide, E. (forthcoming). Audience-Artwork Interaction. International Journal of Arts and Technology.

Abstract: Interactive art is of great relevance to the arts, sciences and technology alike. A common interest among researchers of different disciplines, practising artists and art institutes is the interaction between audience and artwork. This paper introduces the term audience-artwork interaction to facilitate research in this field and raises the question: how can we describe the interaction between audience and artwork? We discuss relevant research such as definitions and taxonomies of interaction, classifications of interactive systems and audience behaviour studies. A working definition of audience-artwork interaction is established. Furthermore, a model which describes interaction as a dialogue between the audience and a dynamic work is presented. We identify possible future directions and point out the need for models which describe the audiences and artworks actions and reactions as well as the relationships among them.

Editorial work

Schraffenberger, H., Kniveton, M., Kolstee, L., Kolstee, Y., Verlinden, J., & Wesdorp, R. (Eds.) (2014, May). AR[t], Augmented Reality, Art and Technology, issue 5.
Schraffenberger, H., Kniveton, M., Kolstee, Y., Verlinden, J., & Wesdorp, R. (Eds.) (2013, November). AR[t], Augmented Reality, Art and Technology, issue 4.
Schraffenberger, H., Kniveton, M., Kolstee, Y., & Verlinden, J. (Eds.) (2013, May). AR[t], Augmented Reality, Art and Technology, issue 3.
Kolstee, Y., Schraffenberger, H., Vahrmeijer E., & Verlinden, J. (Eds.) (2012, November). AR[t], Augmented Reality, Art and Technology, issue 2.
Kolstee, Y., Schraffenberger, H., Vahrmeijer E., & Verlinden, J. (Eds). (2012, April). AR[t], Augmented Reality, Art and Technology, issue 1.

Books

Schraffenberger, H. (2011). Whether you like art or not. Short essays on the liking of modern art. Supervised by Edwin van der Heide. Self-published.

Posters

Kripe, Z., Schraffenberger H., & Terpstra, A. (2010). Interactive Art and Persuasive Technology: The Case Study of the Formamat, Poster presented at the 5th International Conference on Persuasive Technology (Persuasive 2010), June 2010, Copenhagen.

Video Publications

Kripe, Z., Schraffenberger H., & Terpstra, A. (2010). The Formamat: Investigating the Dispensability of Pervasive Data, the 8th International Conference on Pervasive Computing (Pervasive 2010), May 2010, Helsinki. Video published on the USB-proceedings.
Formamat from Hanna Schraffenberger on Vimeo

Magazine Articles

Schraffenberger, H. & Verlinden, J. (2014, May). The great pig in the sky. In AR[t], Augmented Reality, Art and Technology, 5, ISSN: 2213-2481.
Schraffenberger, H. (2012, November). Chasing virtual spooks, losing real weight. In AR[t], Augmented Reality, Art and Technology, 2, ISSN: 2213-2481.
Schraffenberger, H. (2012, April). Interview with Helen Papagiannis. In AR[t], Augmented Reality, Art and Technology, 1, ISSN: 2213-2481.
Schraffenberger, H. (2012, April). Lieven van Velthoven — the racing star. In AR[t], Augmented Reality, Art and Technology, 1, ISSN: 2213-2481.

Newspaper Articles

Schraffenberger, H. (21 April 2011). Kunst? Welke kunst? In Mare – Leids Universitair
Weekblad, no. 27.

Online Columns

Schraffenberger, H. (17th May 2012). What matter does! Blog of the DEAF festival 2012.
Schraffenberger, H. (18th May 2012). The beauty of a flower. Blog of the DEAF festival 2012.