Interactive Installations for Spatial Access to Artistic Sketchbooks

A book is a book – or is it? With present-day, affordable technology, we can scale a book to become a spatial object, or even a space in itself, of almost arbitrary size. We describe our design of and experiences with a generic interactive installation, called Viskbook, that provides a convincing illusion of oversize books, allowing the spectator to turn over the pages using natural and intuitive gestures. It provides a new media for exhibiting books in museums and elsewhere. It can be used for vulnerable and irreplaceable unique books, that otherwise would remain in the safe-box, and for books that may have no non-virtual counterpart and which may go beyond what we normally expect from a book. Viskbook has been used successfully for displaying historical sketchbooks, integrated in thematic art exhibitions, and for experimental books based on our own artistic material selected. An important conclusion is that oversizing a book, in addition to an enhanced visibility, creates a novel and spatial experience of the book. Received on 31 August 2016; accepted on 15 June 2017; published on 03 July 2017


Introduction
"Culture is habits" the Danish historian and Minister of Culture, Hartvig Frisch, claimed as the opening remark of his famous book "History of European Culture" [3].What you experience meeting our (possibly) giant sized innovation, an up to four meters high interactive sketchbook, is a highly deliberate break of habits in dealing with books.From the dimensions of the book and the close distance to it, follows the fact that the field of viewing cannot be focused entirely simultaneusly, knowing that our sharp view only covers little of the entire field of vision in our horizontally oriented stereo view [4, p. 113 ff].
Different media represent different potentials for experiencing space.Different media also offer different use of your body during spatial experience and familiarization.One of the greatest masters in relation to create fascination is the Danish poet Hans Christian Andersen.He very often used the parameter of size to provoke and engage our imagination.He often makes the diminutive gigantic and vice versa.An ordinary window frame can be transformed into a whole world or universe of its own.In the "The Tinder-Box", Andersen let no less than tree spatial monsters of dogs loose, the copper, the silver, and the gold protector.The size of the dogs grows bigger with the value of their protections, leading to their terrifying huge eyes.Humans in general and academics in particular pay little attention to the phenomenon of the common presence of their body as a basic precondition for our common and more specific behavior [7], a point of view that is sharply underlined by French phenomenologist philosopher Maurice Merleau-Ponty, who writes in "Phenomenology of Perception" [9]: "We must therefore avoid saying that our body is in space, or in time.It inhabits space and time" (p.139).This "inhabitation" of space and time is central for understanding the experiencing person's interaction, responding to spatial signals and spaces and turning pages in an ongoing activity, getting access to the inside of the drawings, being intertwined with the life of the huge pages."I am not in space and time, nor do I conceive space and time; I belong to them, my body combines with them and includes them.The scope of this inclusion is the measure of that of my existence" (p.140).The experiencing person gets an opportunity to be included in an existential "being in the world" following his or her body's own concrete body-experience of being alive.As such, humans experience the surrounding world depending of the position of their body in exactly that world: "It is a space measured from me as point zero of the spatiality.I do not see the space from outside as an outer shell, but I experience it from inside, being surrounded by it.All in this world is not in front of me, but around me" [10, p. 41].The bodily rooted phenomenon of "being inside" or "part of" the world being around us rather than in front of us is addressed spatially strongly in our interactive sketchbook project.
We describe and develop an interactive installation technology that appears as a huge book, of potentially unlimited size.It provides innovative and unseen opportunities for displaying artists' sketchbooks as part of art exhibitions, as well as a platform for innovative typologies of artworks.

Related Work & A Possible Pre-Historic Connection
Museums of various kinds use increasingly interactive technology to attract a larger audience and to indicate a continued development of their exhibitions.Interactive installations may be artworks themselves or be vehicles for displaying existing art.A good systematic analysis of the potentials of the first category is found in Kwastek's recent book [6], and we find it also very useful for the second.At first glance, our installation may belong to the second kind but borders are crossed when we develop content, such as drawings, specifically for the installation.Our physical set-up, explained in Section 4, with two projectors pointing into a corner is very similar to that of [5], developed for gamingfor which our discussion of Maurice Merleau-Ponty also will apply; see [11].
The habit of depicting important objects in our surrounding world is universal for humans, as demonstrated so clearly by the cave painters.The Paleolitics', i.e., our cave painters', experiences inside the caves are interesting phenomena to bring forward in relation to experiences with a huge, virtual book.Certain spatial phenomena related to their being in the caves may foster comparative analyses of aspects of the two forms of performances taking place inside the caves and in our book installation.Caves are dark in general (also preferred for the present installation, but not total darkness) so the cave painters themselves had to bring lights to be able to orient themselves, to work and create.The cave painters have been: "inside", simultaneously surrounded and bodily mobile, interactively occupied of the substance of the rockwall of chalk they scratched in, drew and painted on.To specify presentday perceptions, we point at three typical different ways of experiencing sketchbooks as phenomena.They are named according to the body position of the involved person.
• "Outside the sketchbook" has got the natural quality, like any other minor thing, that the book as handheld is an entity you can grip and in most cases turn pages in and, if you make drawing, visualize impressions in.
• "Towards the sketchbook".For many other media a raised body position is the most characteristic, looking at a stationary computer screen, TV or a film.
• "Inside the sketchbook".The dimension of size is highly important here and that your bodily position is quite close to the depictions.Figure 1 shows a sketch of one of the dimensions of being inside the book.Displaying valuable books in a museum is of course not new, and different ways of surpassing the static, two pages view have been used.For example, the Trinity College Library in Dublin, Ireland, shows volumes of the Book of Kells, opened at different pages each days, which is makes a difference for regular visitors.We also often see the actual book complemented by photographic reproductions of all or selected pages, as exemplified in Figure 2.This museum is located in an old mansion, and this showcase is situated in a room with wall paintings that are based on illustrations from pages shown in exactly that book, which gives a very interesting spatial relationship showcase↔room (not visible in the photo).
Touch screens are often used for displaying books as they are inexpensive, they do not take up much space and (differently from our installation) pages are shown in roughly their original size, and the experience is inherently 2-dimensional.We have seen several examples using or inspired by software intended for web page versions of supermarket leaflets.Figure 3 shows a touch screen installation displayed side by side with a showcase with the original book (not shown in the photo).The pages are turned by "dragging" the page by a finger, and, as shown, a limited 3D effect is used to strengthen the similarity with the real book; this should be compared with the more convincing 3D effects shown in Figures 4 and 6.

Sketchbooks, What and Why
For the artist, a sketchbook may always be at hand, using it as a medium for collecting immediate perceptions and ideas, it can preserve a figurative freshness that is often difficult to transfer in the studio  For the interested audience, sketchbooks are a wonderful source for meeting and experiencing the creation process for the completed and perhaps famous paintings.It provides also original insights into the artist's life and life time, which in turn deepen the general understanding of the artist's oevre.
However these books are rarely opened and almost never displayed for the audience.Tragically many sketchbooks reside in the eternal darkness of collections in safety boxes.The Viskbook installation makes it possible and easy to exhibit these books, giving the audience access to their complete contents.All over the world, we find this problem, and we see a large potential and upcoming demand for satisfactory solutions.
In the following we analyze requirements for installations aimed at existing, historical books, and in Section 5 we report on a more experimental application of the Viskbook technology.

Technologically Enhanced Experience as Trompe l'oeile
The design of an interactive installation for oversize sketchbooks is to some degree comparable with the trompe l'oeile genre in painting.The intriguing painting in  reading of the image -to create a convincing illusion of a real book that invites to be studied.The partly visible pages expose various signs (in the semiotic sense), chosen carefully as to make it impossible for the spectator not to hypothesize about what the complete pages may look like 1 and to ponder over the content of the entire book.The painting activates the users mind, it initiates mental processes that appear as almost interactive as the spectator's gaze travels around the image to experience and realize.
Our objective is slightly different from the painter's (whatever that may be).By means of available, affordable and robust technology, we want to create an illusion of oversized books inviting to active investigation.We want to cheat the spectator to believe that he she can browse back and forth in the book to study the different pages and spreads in detail, perhaps to identify relationships between different parts or to experience the flow in the succession of pages, leading to an overall impression of the book displayed.Contrary to the painting of Figure 4 (and what is difficult to associate with a painting), our installation is intended to be generic, i.e., it must function more or less independently of which actual book is displayed through the installation. 2This means 1 ... comparable to what has practiced in fashion design in all times. 2 However, we should not expect our installation to work equally well for all sorts of material.First of all, the original book should be that we cannot thrill the spectator's attention so delicately and precisely with visual cues and signs as did the painter of Figure 4 (and it would anyhow be illusory to expect us even to get near the painter's masterly level), and we concentrate instead on the experienced materiality and handling.
Simulating materiality involving scaling up or down physical dimensions may imply some interesting problem: a book perhaps several meters tall, made of paper etc., that "behaves" similarly to a normal size book cannot exists.There are good physical reasons why elephants have legs that appear thick and clumsy as opposed to the mosquito's thin and elegant extremities: we would not need to be scared when encountering an elephant-sized mosquito, scaled linearly in all dimensions, but we could watch it sadly break down under its own weight.However, the oversize book is mentally no problem, similar to myriads of up and down scaled creatures appearing in comics and other literature, e.g., Andersen's dogs referred to previously.
Let us, as a parenthesis, consider the thinking experience of manufacturing a 2m tall book from traditional materials.In an upright position, the pages of a spread would be hanging and folding in an ungraceful and sadly looking way, and the noise produced in an attempt to turn a page would be unbearable (imagine the sound of a flip chart multiplied by eight!).But using computer technology and current peripherials, scaling of size is not an issue, and sound can be designed in a detailed way according to the illusion we want.We can even imagine an installation equipped with a dispenser of smell of old books that ejects a suitable dose each time a simulated page is simulated to be turned (which, however, is in conflict with our focus on affordable and robust technology).In the following section, we discuss a few more design criteria and detail our design.

The Viskbook Technology
Here we describe our proposal for a technology intended for displaying the content of sketchbooks.The primary design goal has been to create a convincing illusion of an oversize book having a natural interaction of turning the pages in the book.In other words, even based on computer technology, it should not be experienced as a "computer installation", which means that help text, navigation tools, intelligent autonomy or the like are totally banished.
In order to make the installation accessible for a wide range of budgets and to make it easy to set up and take down, we have aimed at using payable and replaceable standard components, and minimize interesting in itself, and limitations in contrast and resolution of the chosen projectors or monitors may exclude certain types of content.used, one placed at each edge, shooting at the opposite wall.The projectors need to be of the so-called shortthrow type in order to produce a sufficiently large image on a short distance, and they can be hidden by a screen as shown.The unavoidable geometric distortion is corrected by software.The figure shows also the typical position of a spectator, waving his or her hand in one or the other direction to turn a page, thus being able to go through the pages from one end to another or go a bit back and forth.Not shown in the figure are two loudspeakers behind the screens also hiding the projectors; in the basic version of Viskbook, sound is only used to produce a discrete "flap" sound when a page has been turned.In order to strengthen the illusion of the book as physical object, we use a 3D engine developed for computer games to obtain an effect as if the page is arching through the space in front of the book; see Figure 6.The choice of sensor for recognition of gestures for turning pages is not essential; the figure shows a Kinect sensor (developed by Microsoft for their Xbox 360 play station), but other devices have also been used.A standard laptop computer with two video outputs is sufficient to run the entire installation.

Technical Observations
From a technical point of view, the design principles seem robust and do their job.More specifically, we noticed the following.• The 3D effect associated with the turning of a page is very convincing.The spatial design, using a real corner as opposed to a flat screen, gives a significant amplification of the computer generated and normally flat "3D images".
• Although we have made no comparative studies, it is our clear impression that the simple sound effect (the "flap") adds a lot to the feeling of materiality.
• This arrangement of the projectors with crossing beams shot into a corner eliminates completely any problems with shadows, that typically appear in installations using front-projection.
• A Kinect sensor located as shown in Figure 6 is difficult to calibrate so it perfectly fits all spectators (of different heights); 3 we are currently developing more reliable solutions with other sorts of sensors.
• The installation scales immediately to larger sizes; heights up to 4 metres have been tested in our laboratory.It will be a manageable software development task to scale to very large sizes with several synchronized projectors for each of the two pages.The real problem here will be fixing of the projectors and finding a suitable location for such an installation.
• A portable version of the Viskbook prepared in the lab can be set up and calibrated in a few hours, making it interesting also for fairs and other events.

Possible Extensions
The Viskbook technology as described above has been optimized for giving a faithful and respectful illusion of a given book.However, the technology provides a potential of going beyond what we normally expect from something that we designate "a book".The following ideas have been tested in our laboratory; some experiences are explained in Section 5, below.
• Animated pages; e.g., a video or a drawing extended with cartoon-like effects.
• Content may be different sorts of images; we experienced that drawings are the most robust concerning the lighting condition and limited resolution of projectors, while photos and reproductions of paintings require more care.
• Adding real sound associated to the images in the book may be used without destroying the illusion of a material book.
• Cheating with the content of the book, so, e.g., going one page forth and one back again may yield a different view.
• Allowing spectators to manipulate the book content.
Using real sound has been tested in exhibitions with good results, while the others still need practical experimentation to prove their relevance.Obviously the use of such effects should be made with care, as on one hand maintain a believable materiality and on the other create interestingly enhanced experiences.

The experimental book installation "Italian Drawings"
The Viskbook installation described above is a result of a series of more experimental setups developed in our laboratory and exhibited at libraries and cultural institutions.The hardware was essentially the same as described above, but the contents of the virtual book consisted of drawings produced by a member of the research team, Bjørn Laursen.These drawings were selected from a comprehensive collection of drawing made on different locations in Italy, thus the title "Italian Drawings", complemented by new drawings produced for or adapted to the installation.This implied several advantages for our experimental research, compared with the situation where we start from a given book.We have no copyright issues or other restrictions, and we could experiment freely with "semantic threads" through the sequence of pages, drawn from a database of more that 700 drawings, represented by thumbnail versions of the drawings and a very simple semantic tagging.
As an example of animated pages, we included socalled topological drawings [8], that are drawings never shown in full to the spectator, but with only a small part visible at a time through a moving peephole.This way of viewing an image provides a very different experience as opposed to the traditional way, where we first make an overview and then perhaps dive into selected details.Topological drawings strengthen the focus on the details and trigger the spectators imagination to figure out what the full drawing might look like, very similar to the partially visible pages in the trompe l'oeile painting shown in Fig. 4.
Being freed from the traditional museum setting, we could also experiment with different uses of sound, of which we identified the following relevant categories.
1.The page-turning sound described above.
2. Empirical sounds or soundscapes from the concrete destinations where the drawings were made.
3. Effect sounds related to the destinations.
4. Using stereophonic sound to couple specific sounds to one or the other of the two visible pages; this allows a dialogue or other "Ping-pong" effects.
5. Explicit speak from the author relating destination and drawing process.
6. Implicit speak -authors inner reflections of the process of experiencing -during the drawing process.
7. Voice-over speak by an introducer.

Music.
These dimensions in sound design give us a wide spectrum for creating variation in the way our visitors will experience the whole installation and drawing collection.

Experiences and Conclusions
The Viskbook has been shown publicly at the following events.
• Istituto Italiano di Cultura di Copenaghen, November 22-December 20, 2012."Italian Drawings" by Bjørn Laursen; a virtual sketchbook composed of drawings from different cities in Italy: Rome, Naples, Venice and Florence.
At the Rørbye exhibitions [2], our installations displayed two specific sketchbooks, integrated as elements in larger exhibitions.This confirmed our hypotheses about a strong interest among the audience to discover the complex and varied content of the sketchbooks, and it complemented well the static exhibiting of authentic the works; see Figure 7.We experienced also that the presence of an interactive installation created new social interactions among the visitors, guiding each other in its use, leading to conversations about what was seen in the book.In one exhibition a one-line instruction was given as part of the curator's introduction to the sketchbook; in the other, there were no instructions, but a circular marking on the floor indicating a recommended position for browsing the book.The pattern of use were the same in both cases: the visitors found out themselves.Focusing the sensor on the natural (and marked) position of the visitor "in control" of the book reduced confusion to a minumum in case of several visitors watching the book.We learned also that an ultimate robustness of the interaction is essential (as opposed to the "95% performance" of that version).The Italian Drawings exhibitions displayed virtual books composed from several actual sketchbooks, selected in order to produce strong coherence.The location in rooms not dedicated to exhibition exposed that conditions of light were critical for the visual contrast and overall experience of the installation.
In all cases, we experienced a very positive interest for this type of installation, and further research may show whether this is due to novelty value only or true qualitative enhancements.As it appears, a more systematic evaluation of the approach is still to be done.

Figure 1 .
Figure 1.An oversize book installation may materialize the sense of being inside the book, transforming Hans Christian Andersen's imaginations into spatial perceptions.The biggest version of our Viskbook installation is H=4.2m,W=2x3.2m.(Drawing: B.Laursen)

Figure 3 .
Figure 3. Touch screen presentation of a Libro d'ore, parchment book, illustrated with miniatures, by unknown Flemish artist from the beginning of the Sixteen Century; Galleria Regionale di Palazzo Bellomo, Siracuse, Italy.-Compare also with Figures 4 and 6.

(
Photo: B.Laursen) later.It is a flexible, semantic memory media combining multiple elaborated sketches, notes of impressions etc.

Figure 4 ,
attributed to Ludger tom Ring (1522-1584), is spot on for our comparison.The artist has unfolded his technical virtuosity with brushes and paint to make a non-existing materiality stand forward from the objectively flat canvas -combined with an impressive psychological insight into the spectator's 3 EAI Endorsed Transactions on Creative Technologies 04 2017 -07 2017 | Volume 4 | Issue 12 | e3

4Figure 5 .
Figure 5.A sketch of the Viskbook installation seem from above.

Figure 6 .
Figure 6.The 3D effect used in Viskbook when a page is turned.(Photo:H.Christiansen)

Figure 7 .
Figure 7.A Viskbook installation in the Martinus Rørbye exhibition at Nivaagaard.The photo is taken during the exhibition, and shows how the installation creates a special attention and at the same time is an integral part of the exhibition.