Client

Explorers

Website

www.explorerslive.com

SERVICES

Mobile development (iOS)

Back-end development (.NET/C#)

Project management

Case in brief

We pride ourselves at Olympiatech on being at the forefront of technological advancements, and our recent project with Apple's Vision Pro and Vision OS is a testament to our commitment. In our partnership with the startup MedicalVR, we're showing our dedication to blending tech and healthcare in new ways. By leveraging the benefits of patient specific CT-scans, Virtual Reality (VR) and segmentations, MedicalVR is developing the future of pre-operative planning. Patient-specific CT-scans are segmented and transferred into a 3D-model creating a digital twin of the patient. Allow optimal pre-operative planning, patient information, training and education.

In this case we will offer a comprehensive overview of the project, its goals, and the technology behind it. Our focus on user experience and adaptability, even under tight constraints, positions them as a leader in innovative solutions. This endeavor not only showcases their technical prowess but also our vision for a future where technology plays a pivotal role in enhancing healthcare outcomes.

From concept to working prototype in 3 weeks

Tech stack: Vision OS | SwiftUI | RealityKit

Team of 3 people (2 engineers)

Project Overview

Primary Goal of the Project

The main goal was twofold: for the user side, to create a user-friendly tool for preliminary eyesight assessment; and for the medical side, to offer doctors a precise 3D visualization tool to aid in their diagnostics and surgical planning.

Project Description

Firstly, it aimed to help general users assess their eyesight quality through an intuitive application. This app conducted various tests, and based on the results, could suggest if there might be underlying eye issues needing medical attention.

The second part of the application was designed for medical professionals. It aimed to provide doctors with a tool to visualize 3D models of scanned organs, such as the lungs. This tool is revolutionary in potentially reducing the need for exploratory surgeries by precisely pinpointing tumor locations or other medical concerns.

Technology stack

UI toolkit

SwiftUI is the best way to build a new visionOS app or bring your existing iPadOS or iOS app to the platform.

Rendering Engine

Present 3D content, animations, and visual effects in your app with RealityKit, Apple’s 3D rendering engine.

SDK

Development for visionOS starts with Xcode, which supports the visionOS SDK. Add a visionOS target to your existing project or build an entirely new app.

Our approach

Goals and Features

The primary goal of this initiative was twofold:
1. Deliver an easy-to-use eyesight assessment tool for the general public.
2. Provide medical professionals with a precise 3D visualization tool to enhance diagnostics and surgical planning.

The project leveraged the Vision Pro headset's eye-tracking and immersive visual capabilities. These features were crucial in developing an accurate eyesight testing application and an efficient 3D visualization platform for medical purposes.

Technical Aspects

On the technical side, our developers utilized SwiftUI for crafting intuitive user interfaces. For the intricate task of 3D rendering, especially vital for the medical visualization application, we employed RealityKit. All development was facilitated in Xcode, ensuring seamless integration with the AR/VR headset's functionalities.

UI and User Experience

The user interface was carefully crafted, keeping the end-users in mind. For the eyesight test app, we prioritized simplicity and clarity. We used SwiftUI for its proficiency in developing fluid, native interfaces. The medical app, due to its specialized nature, had a more complex UI. However, the focus remained on ensuring clarity and accuracy of the 3D models.

The outcome

Challenges and Solutions
Like all pioneering ventures, this project wasn't without its challenges. Our team had to quickly adapt to Swift programming language and certain facets of Vision Pro and Vision OS, especially given the tight timeline of just six days. Another significant challenge was the limited documentation available since the product was still in pre-release. This situation demanded real-time problem-solving and rapid learning. Notably, the operating system's privacy constraints presented a hurdle. They prevented the collection of eye movement metrics, a feature we hope will be available in future OS iterations.

Learnings and Achievements
This project taught us how important it is to harmonise hardware capabilities with good software innovations.The experience of adapting to SwiftUI and RealityKit was invaluable. Despite the project being a Proof of Concept (POC) and the inherent challenges, the positive feedback we got showed how well our team performed.

Guidance and The Road Ahead
For others thinking of doing similar projects, it's good to really get to know the operating system. With emerging technologies, using all available resources, from developer communities to hands-on experimentation, is crucial. As for future developments, we foresee the eyesight app benefiting from a broader range of tests and AI-enhanced scoring. The medical app, with advancing 3D rendering technology, could offer even more detailed organ models, greatly aiding medical professionals.

In conclusion, our work with Apple's Vision Pro and Vision OS showed our dedication to new solutions. This project highlighted our skills and our goal to make tools that really help people. As tech gets better, we'll keep using it to make a difference.