Maritime's Amazing World of Mixed Reality

New holographic technology is set to revolutionize training procedures, fleet management, ship performance and engineering maintenance in the maritime sector

Have you ever wished you could move around your fleet of ships, in full 3-D, without ever moving from your desk? Have you ever wanted to be able to delve into the complex inner workings of one of your vessels and inspect its internal structure while sitting in your office? Have you ever been performing maintenance work within a boiler room and wished you had a 3-D plan of what’s in front of you and receive expert guidance on how to fix any problems you detect? The stuff of imagination? Not now. Intrigued? Then read on...

Mixed reality: not just virtual reality, but a whole new concept in which any object with a complex internal structure can be fully, not just represented, but reproduced as a fully three-dimensional object with which it is possible to interact through your own physical movement. Fostech AS, headquartered in Fosnavaag, Norway, has been developing and pioneering this amazing technology since2016.

Fosnavaag, Norway: The ‘Silicon Valley’ for Maritime
 “This part of Norway has had the same kind of influence over the global maritime industry as Silicon Valley has had on the global tech industry in recent decades,” said Havard Notøy, Fostech’s Founder and CTO. “This heritage comes from the region’s fostering of fishermen operating in some of the most harsh environments to be encountered and the companies that operate here are proud of the fact that state-of-the-art ship design, equipment and solutions from the region are now in operation all over the world. This is the reason why Fostech focuses on delivering state-of-the-art holographic solutions for the global maritime industry in recognition of the push towards autonomous vessels and operations, and therefore a reduced onboard crew presence.

 “This means that remaining crewmembers need to be able to cover a wider range of equipment and hence have a wider range of product knowledge. We recognized that to obtain this knowledge, a new and improved type of product documentation was needed and had the business idea of developing mixed reality solutions that add value for maritime customers. By adding value, we mean improving quality, safety and productivity, or improving the customer’s profit in any other way, and so we are now developing ways of visualizing and physically, not just virtually, interacting with any maritime object with deep and complex internal structure, including mechanical equipment such as engines and compressors, structural objects such as cranes, and, of course, the vessels employed by the industry.”

Mixed reality – where real-world and digital objects interact
So what exactly is mixed reality? A mixed reality (MR) experience combines elements of augmented reality (AR) and virtual reality (VR) to enable real-world and digital objects to interact. Augmented reality simply adds digital elements to a live view, often by using the camera on a smartphone (examples of AR experiences include Snapchat lenses and the game Pokemon Go), while virtual reality implies a complete immersion experience that shuts out the physical world. Using VR devices such as HTC Vive or Oculus Rift, users can be transported into a number of real-world and imagined environments. Mixed reality technology is just now starting to take off, with Microsoft’s HoloLens as one of the most notable early mixed-reality apparatuses. The current estimate of the size of the market in 2023 for the global delivery of AR, VR and MR hardware and software is in the range of $100 billion to $200 billion and VR, AR and MR are currently among the fastest growing tech industries globally.

The areas of the maritime industry where Fostech’s MR technology has the potential to deliver solutions include sales and marketing, maritime video communications, education and training, sales and maintenance, and other operational solutions, and for companies interested in the potential applications of MR in their maritime business, Fostech recommends that they “dream big but start small”, understand their “as-is” working processes, and involve end-users in their decision-making. It is essential that they determine what drives customer value and that they look to see what areas of business can be improved, for example production efficiency and determining optimal production flow, achieving the highest possible quality levels and “getting it right first time”, and, finally, ensuring that critical procedures such as the most expensive or most time-consuming ones are optimized as fully as possible.

Benefiting the maritime sector
Hans PetterHildre, Head of the Department of Oceanspace Operations at NTNU, the Norwegian University of Science and Technology, had this to say about the new technology:

“The maritime industry has begun a long journey towards autonomous ships. It is going to take many years before we see extensive shipping traffic without crews, but the voyage towards unmanned ships creates vast, new opportunities. We will see a gradual reduction in operating crews, which will lead to major demands on remaining crew members. This will be especially demanding in the engineering field. The question is how an engineer can carry out troubleshooting and maintain service on increasingly complex equipment. Technology supporting these activities is a prerequisite for succeeding with crew reductions. Technology must also attend to communication with onshore experts operating a wide range of equipment. With focus and close contact to the maritime industry, Fostech could be an important participant on this voyage.”
While HåvarRisnes, Head of StartUp and ScaleUp at ÅKP Blue Innovation Arena, the local Sunnmøre region’s hub for innovation, business development and social development, commented:

“Fostech enters an increasingly relevant niche in VR and AR in their fresh effort. As an incubator company at ÅKP, Fostech is closely linked to the DigiCat catapult and the clusters of Norwegian Rooms, Blue Legasea and GCE Blue Maritime. Businesses need companies that utilize the latest technology and adapt it to relevant customer groups. Fostech’s idea of an ‘appstore’ for their products are exciting and innovative. They have attracted contributors from other countries and with different experiences. ÅKP really values Fostech’s efforts within the maritime sector.”

Proprietary presentation platform
Fostech has developed the Fostech HoloPodium v2.6 Configurator, a proprietary presentation platform/configurator that manages 3D animations, voice instructions, text instructions, warnings and notes to reduce MR systems development time and costs by about 50 percent and ultimately making it less expensive to develop MR solutions compared to the traditional animated movies of old. The Fostech HoloPodium supports a number of animation systems including Microsoft Hololens, PC and various mobile platforms. The next developments will be to include multiple same-time users to allow classroom-based use of the system and to enable external participation: the company is currently looking for a commercial partner to support its work in these areas and complete these technology development stages.

Maritime APP Store
Fostech is also developing its own Maritime APP Store (www.themaritimestore.com),an online augmented/mixed reality store meant for the maritime sector, where customers can subscribe to hardware and selected software solutions. The intention is that most of the company’s customer projects can be published on the Store, with shipping companies as the ‘number one’ customers being featured, while one of the benefits of the Store for equipment manufacturers is that they will have full control of where their solutions will be distributed. Notøy commented that nothing similar exists today and that the solution has the potential to become some kind of a disruptive force in the industry.

Investment opportunity
Fostech was established in late 2015 and started working solely on mixed reality and Hololens in May of the following year. Notøy commented: “We have built lots of knowledge about the capabilities and solution directions of this exciting new platform and 2018 has been the year in which we compiled all this knowledge and developed our own presentation solution, where the final product is a unique Unity plug-in configurator, enabling us to develop high-quality solutions at a much lower cost compared to only a few months ago. In addition, we have developed a Beta version of the Satellite Video Communication application and we have plenty more exciting solution proposals for the maritime industry in our drawer. If you are interested in investing in this opportunity, please contact Fostech.”

An exciting future
“Microsoft in a way took the AR/MR market a little by surprise when the company launched its Hololens device in April 2016,” stated Notøy. “Nowadays, most of the other big players are talking about launching their own solutions in the next few years. At Fostech, we see in the work we do, and the work that others are doing, that the technology is already bringing value for many enterprises around the world. And since Microsoft is launching Hololens version 2 next year (with a new AI chip, an improved field of view, depth sensors, tracking, plus battery and processing power), I think we have only seen the top-of-the-iceberg of what this technology will enables us to do in the years to come. But most importantly, it’s already here, and it is already making a difference.”