Biketopia

In November 2017 me and my buddy Daniel followed an invitation from Michael Johansson to Sweden to work as Artist in Residence on an installation that will later be a small part of the cube_x project within the EU funded PS2 project. As usual, the timeframe was tight.

Our goal: Creating the ideal city sketched by Leonardo da Vinci in Virtual Reality. However, users should not explore the city on foot, but by bicycle.

Our installation at the Architecture Biennale 2016 in Venice (as part of the Ideal Spaces Working Group 2016 [1]) had allowed us to gain some experience, which made the construction of the floor plan easier for us. While back then only prerendered camera movements were shown, we now had to build a new 3D model from scratch that could fulfill the following features:

1. All relevant places and places must be accessible by bicycle. This aspect is relevant in so far as Leonardo’s construction provides several height levels and the entire city is built around a dense network of canals.
2. The cyclist has to be able to move freely in 360°, which means that the city has to be elaborated much more extensively.
3. The perspective is always linked to the physical height of the cyclist, i.e. interesting things should take place in the typical field of view.
4. The entire application must run in real time

Construction: How to build a 3D model with sketches from 1486?

Leonardo’s project was by no means a finished construction blueprint, but rather a conglomeration of loose thoughts, drafts and ideas that he wanted to put on paper quickly. Accordingly, neither exact floor plans nor dimensions of the city’s structre are known. We therefore decided to referain from working in absolute units, but to capture the atmosphere of the imaginary place as best as we could. That’s why it was particularly important to us to preserve the proportions of Leonardo’s drawings as accurately as possible in 3D.

As basis for our work we used Leonardo’s drawings and tried to transform the freehand lines into straight vector shapes that we could then use in our 3D software. Always in mind that we wanted to reproduce the underlying construction idea as exactly as possible.

Just a quick example: The arcades Leonardo planned are not based on a “full” semicircle, but are the result of a cut arc of a circle. This detail may seem inconspicuous at first glance, but since every building in the city features the same arcades, such marginal elements are suddenly important and define the overall look and feel.

Leonardo’s concept was never built. Therefore it seems important that the city’s metaphysical character remains present.

It would be wrong to fill this imaginary place with algorithmically generated (artificial) life we know from NPCs in computer games.
Nevertheless, this space needs “action” in the broadest sense. Between Leonardo’s city-sketches there are also various considerations about inventions that can be used within the city.
For example, da Vinci sketched a multitude of bridges that were either mobile or otherwise movable and swiveling. We decided to integrate these objects into the city in order to adress two aspects:

• bring the place to life through animations and increased level of detail
• solve the inherent problem of access to all places by bicycle

The iconic “Swinging Bridge” can be rotated to enable ships to pass. Leonardo cleverly hides the necessary counterweight within the curved bridge design. We tried to ensure that the rebound when stopping the rotation is as physical correct as possible.

Vegetation

In the sketches of da Vinci there is no evidence that parks or green areas should be created within the city. However, in order to represent the urban environment, it was necessary to at least indicate the presence of nature. Here, too, we carried out a large number of experiments to find out what kind of representation would be suitable for this particular purpose.
Very quickly it turned out that we did not want to work with the Speed Tree vegetation of Unity3D or other “realistic” plants.
Another idea of ours was to emphasize the strictly functional character of the city by using algorithmically generated tree-like structures. We created some scribbles with L-systems, but this style wasn’t appropriate either.

Finally, we modelled by hand four different abstract trees with phenotypes similar to those of the northern Italian cypress trees.

The follow text is taken out from our drafts of the paper: Worldmaking: Designing for Audience Participation, Immersion and Interaction in Virtual and Real Spaces (click here for a full text PDF: worldmaking@researchgate).

Since there are clearly divergent requirements for the shown environments from a purely technical perspective – especially real-time requirements are to be mentioned here – it should be briefly outlined at this point how the aesthetics of our rendering can be transferred into a game engine. 

The core problem is that a single frame of the pictures shown above take about 40-60 seconds to calculate on a workstation. Since interactive VR applications require 60 frames per second to be displayed, a different approach is required here. Specifically we did the following steps:

1. reduction of world size: Depending on the possible perspective, it is possible to predict which areas of an environment the user can actually walk through. Especially when locomotion in the environment requires real muscle power, such as on a bike, the world shown can be much smaller and more compact than in environments that also allow a bird’s eye view. If additionally paths are skilfully directed (for example through impassable waters or obstacles inserted into the world), then the impression of a harmonious environment arises despite a comparable compact world size.
2. reduction of geometry: Especially when working with 3D models from unrelated sources the meshes require post-processing. Internal geometry, which will not come into the user’s field of view anyway, can be deleted in the same way as geometry which is not in the user’s field of view due to the observer’s perspective. However while performing this job, it is important that the basic mesh geometry is not destroyed. It is also necessary to check the normal vectors of each vertex.
3. level of detail: Especially during the balloon flight, where the basic route is predetermined and cannot be changed by the visitor, the viewing distance at which the observer perceives particular 3D geometry can be easily estimated in advance. Since many details of buildings cannot be seen from a bird’s eye view anyway, the 3D geometry can be significantly simplified here without significant loss of quality. Objects that will always remain in the distance (e.g. trees in the surroundings) were even replaced by 2D billboards with alpha map that featured only one (!) vertex per tree. 
4. shadow baking: Since most of our environments consist of static, fixed objects, it is possible to bake the shadow into the texture. This method allows the high-quality raytracing to be used for rendering, because the shadow calculation is no longer performed in real time, but is already firmly linked to the model during production. To use this method, the position of the sun must be determined very early in the production process and can only be changed subsequently with great effort. In addition, it is necessary to process the UV maps of the 3D models properly and to prepare them in such a way that neither overlaps nor excessive distortions occur. 
5. baking of geometry: Despite all reduction of geometry, it is still necessary to preserve some details, because they provide unique character that makes the environment special. In concrete terms, the columns’ capitals of our da Vinci environment can be considered iconographic that must be preserved in the spirit of the desired aesthetics. Unfortunately, the detailed capitals are not real-time capable: A single column features over 75,000 polygons in the initial version. A reduction by a factor of 3 was possible without the visual quality suffering too much, but each further reduction step changed the character of the space enormously. The 25,000 polygons per column were noticeably too much to be displayed fluidly in a real-time environment, especially considering that 20 or more columns have to be displayed depending on the field of view. In order to preserve the visual character as best as possible, we refrained from further simplifications and worked with the “normal baking” method, in which high polygon parts of a model are baked into the normal map of a low polygonal version of the model. Since the capitals are not at eye level with the viewer, we expect a credible representation that adequately communicates the impression of space.  
6. instantiating: Depending on the environment, parts of the geometry may be replaced by instances. This works especially well in environments that are very repetitive and essentially consist of repeating basic patterns.
7. matte painting: One of the most important elements of our environments were the skymaps used. During the balloon flight we chose them not only for representing sky and sun, but also to picture the landscape in the surrounding area (mountain ranges of Black Forest and Vosges). Here in particular, enormous savings can be made in geometry, since not only topography but also vegetation can be baked into a two-dimensional texture. The entire environment was remodeled in Cinema4D, provided with vegetation, sun, sky objects (clouds, dust particles) and lighting effects (atmosphere, ozone) and exported as an 8000×4000 pixel spherical panorama. This image was then placed as inner texture around a sphere that functioned as skydome in Unity3D. 

We are currently working on integrating a fluid interaction pattern into the VirZoom bike.

As a first beta-test Michael was able to exhibit our prototype in Malmö/Sweden with very promising results. Nevertheless our work with DaVinci is still highly work in progress.

Video Copyright: Michael Johansson

Footnotes

[1] The Ideal Spaces Group 2016 consisted of: Ulrich Gehmann, Michael Johansson, Matthias Wölfel, Daniel Hepperle and Andreas Siess and many more. Please refer to the project page for a full list and further information.