Santa Cruz del Islote is a remote Colombian island just over the size of a football field and home to around 500 residents. Situated two hours from the mainland, they don’t have access to things we take for granted like running water or electricity grids. Resource conservation is simply a way of life. All this makes the tiny island an interesting place to explore, which is why Colombia’s largest bank, Bancolombia, decided to use it as their base for the School of Sustainability.
Developed as a Google Street View-like experience, virtual visitors can explore the island in detail, get to know the community, and learn nine core lessons in sustainability and resource conservation.
Together with fellow crafters and makers Google and Sancho BBDO, our goal at MediaMonks was to create an immersive, cohesive island experience that hadn’t been done to this level of detail before, while also making sure that people left their virtual tour with a new appreciation for resource conservation and living a more sustainable lifestyle. So how was the experience created?
Capturing the island in stunning detail
We travelled to the island with a full crew. Armed with a tripod and SLR, our photographer focused on capturing the island with beautiful shots from every angle. These shots would then go into 140 photospheres placed around the island, with each giving the viewer a high fidelity 360-degree view.
Each shot was carefully framed to ensure that there was a snapshot of island life in each photosphere, with the idea to offer a virtual tour through the lives of the residents. It was a pretty labour intensive method and took us several days of photography.
Creating a more immersive street-view experience
While it was up to our photographer to capture island life in beautiful photos, we also had to make sure that the transition between each shot was as seamless (and similar) as possible, in order for the user to feel they are actually walking around the island and not just moving between a series of images. To create a cohesive transition between each photosphere, we used a variety of tricks to ‘glue’ them together.
To get a fluid transition, we needed a (low poly) 3D mesh of the island, so we could project the current photosphere on this mesh at the start of a transition, and then fade the projection to a projection of the destination photosphere while moving in 3D.
This is similar to what Google does for their own Street View. Google creates the low poly mesh needed, by using a 3D scanner placed at the same location as the camera. Because we used a classical camera to create the photospheres and couldn’t use a 3D scanner, we had to make the 3D model in a different way.
Photogrammetry was the technique we used to map the island in 3D. At heart, photogrammetry is the technique of automatically creating a 3D model of an object, by analyzing photos of the object taken from several angles.
We used a drone for aerial photography and shot thousands of photos at ground level and then created the high poly 3D mesh we needed with photogrammetry-software called Reality Capture.
In the end, we optimised this mesh by hand. The resulting low poly mesh is used for the transitions between the different photospheres.
Imitating real life
The photosphere-to-mesh projection during transitions really helps in getting a more immersive experience. It gives the photospheres a 3D position in world space, and the warping of the photospheres during a transition (based on the projection on the low poly mesh) helps to give the user the feeling that they are moving through a 3D environment.
To create an even more immersive experience we added elements to the scene that are stable and consistent between each photosphere - helping the viewer experience a more coherent transition as they explore the island.
Around the island, we placed lots of clickable hotspots that take a user to a story or interesting fact from an islander. We made these hotspots viewable from multiple photospheres, so a user can see them from a distance while traversing across the island. The hotspots have a 3D position, so they behave like real objects in space when the user walks from sphere to sphere.
As well as the clickable hotspots, we placed audio sources with atmospheric sounds in the 3D space. These sounds were recorded directly from the island and can be heard from multiple photospheres. The spatial audio helps the viewer to locate where the sound may be coming from. As the user comes closer or pass by it, audio intensity and panning changes, based on the distance to the audio source and the view direction of the user.
Furthermore, a completely custom sky was created, with animations of frigate birds passing over. This required us to manually cut out the sky from every photograph which was quite a labour intensive method.
Finally, we added a layer of lens flare and a subtle vignette effect, just to really help ensure a smooth transition and make the different photospheres as similar looking as possible.
The 3D map
For the map and the overview of the island you see during the intro sequence at the website, we used a simplified version of the high poly mesh from the photogrammetry, again using Reality Capture.
Because all photospheres are positioned in 3D, we could easily create near-seamless transitions between this overview and the photospheres. Hopefully, this gives the user an even better understanding of their position on the island, and strengthens the illusion of walking around in an immersive environment.
Being able to explore a real-life location you might never see is one of the gifts recent technology has given us. Remember when Google Street View first came out and it was a treasure to hover over Manhattan? Then came along Canadian artist Jon Rafman’s incredible Nine Eyes project capturing real-life scenes across the globe and GeoGuessr taking us from the outback of Australia to the fjords of Norway. Humans love exploring. And by mapping Santa Cruz del Islote, we hope that we have created a more immersive street view experience that everyone wants to pay a virtual visit to.