From delivering a new level of immersion in training to enabling real-time connections between experts and field technicians, extended reality (XR) technology is transforming the world around us.
Augmented reality (AR), mixed reality (MR) and virtual reality (VR) technologies have already proven their value in a variety of use cases.
But what’s really involved in implementing them? And above all, how can you be sure that an XR app will deliver value for your organization? In this article, I’ll answer those questions and more — walking you through the questions you should ask yourself before deciding what to create and how to create it, along with important deployment issues.
Defining common scenarios
Almost anything is possible in XR. But for an article like this, we need an example scenario that’s both common and broadly applicable for a wide range of organizations.
We’ve chosen to look at scenarios that involve mapping spaces and identifying items within those spaces. That could mean using XR to:
Locate items in a warehouse.
Find and detect specific landmarks.
Locate vehicles or other assets on construction sites.
Help people navigate buildings or facilities.
While those use cases all have a lot in common, they can all require different technologies and strategies.
Chapter 1: Before you begin
Start your journey with a clear view of your goals
As with any new application or technology, the first step towards a successful implementation is defining what you want XR to help you achieve. Whether you want to improve on-site safety, increase training efficacy, or provide your customers with an engaging experience, the goals you define will influence every decision you make along your XR journey.
Defining your goals upfront will help you choose the appropriate type and level of technology needed to make your vision for XR a reality. You might not need the latest, greatest and most expensive technology to achieve your goals. Understanding what you want to achieve can help you control the cost of your journey and avoid adding unnecessary complexity with features and tools you don’t need.
Goals come first, but privacy, security, and safety should be a close second.
Every new technology you apply across your organization impacts privacy and security — and XR is no different. Privacy and security considerations need to be made early in your journey, as they can define what is and isn’t possible within your chosen use cases.
Part of the magic of AR comes from combining visible information about your surroundings with additional information about that place or objects in that place. However, from a business perspective, you need to consider how that will impact security and privacy, and ask the right questions upfront. For example:
Is it OK to create a map of your warehouse and the locations of objects in that warehouse? Is it OK to use a camera to do this?
What if the technology that does the mapping stores the information on a third-party cloud service?
Is it OK for customers to use their cameras inside your stores?
Is it OK for you to store information about the location of your customers?
Do you need to notify users that you are tracking their movements?
Answering those questions will help you understand how your policies will need to evolve to safeguard customers and data, and identify any points of friction that might hinder your project. Keep in mind that some VR headsets also use external cameras for tracking, so even if the user can’t see their physical location, the VR headset may be able to.
At this stage, it’s also important to spend some time thinking about user and employee safety. As people explore and engage with your virtual or augmented environments, you don’t want them to be tripping over physical objects, nor do you want them using a device that itself does not meet your safety standards.
While devices and equipment designed for industrial use often have safety requirements that they need to meet by default, there are generally less specific and rigorous standards for consumer devices. So when using consumer devices in a commercial setting, in addition to reviewing your privacy and security protocols, you should also consider your safety requirements, including equipment certifications.
Chapter 2: Planning your deployment
Bringing the right hardware and software together
XR deployments involve both hardware and software elements, and the key to a successful deployment is making the right decisions on both sides simultaneously. On the hardware side, you’ll need to consider:
How users or customers will interact with your deployment: Will you use phone-based AR, or will they need to use specialist equipment like VR headsets provided and managed by you?
The locations where your deployment will be used: Will you need to deploy cameras, detection tools, wifi extenders, or other specialized hardware in the areas where users will use the technology you provide?
How advanced your use cases are: What level of technology do you need to make your proposed use cases a reality? Do your plans really call for the newest and most sophisticated hardware, or could proven tools do the job well?
Which devices you’re supporting, and how many you’ll need: Acquiring XR hardware can be expensive, so it’s important to establish how many devices you’ll need and how you’ll support them.
On the software side of the equation, you’ll also need to consider:
How you’ll deploy your software: Will your AR app be available globally through a public app store, or will you deploy your apps locally? If you’re using a 3rd-party store, how will their release lead times impact your plans?
How you’ll deploy updates to remote hardware: If you’re deploying hardware in the field, how will that hardware receive updates and need to be maintained?
The platforms you want to use and their limitations: Many of the platforms powering XR technology aren’t entirely transparent. Vendors retain some secrets to protect their intellectual property, so often, it’s up to you to test the tolerances and nuances of that technology for yourself to determine its viability.
Where users will go when they need support: If your technology is deployed in-store, for example, will you need to train staff to function as frontline technical support operatives too so they can resolve customer questions? And are there any ways to empower users to answer their own questions?
Making the right choices for your chosen use cases
Ultimately, the decisions you make around each of those considerations will hinge on the use cases you’ve identified for XR, the users involved in those scenarios, and the technical needs of those users. Each use case can bring its own technical considerations, so let’s walk through an example scenario to see what that looks like in practice.
Let’s say that a university wants to create an AR map app to help students and visitors navigate its facilities and get to the right places at the right time. By picking that use case apart, we can understand the technical requirements and priorities that need to factor into the university’s hardware and software decisions.
Users: We know that this app will likely be helpful for both employees and the public, so it must be publicly available.
Desired experience: For this map app to be valuable, it needs to be usable on the move, as someone navigates the campus as a whole and individual buildings.
User needs: For most users, all they will want from the app is to find where they need to go quickly. Ease of use and speed to get started are paramount considerations for software design and deployment.
Granularity: Finding specific rooms and areas of a large complex is far easier than identifying small objects, so we can likely rely on beacons and GPS. There’s no need to switch to any kind of optical scanner as we would for finding a specific item on a shelf.
The physical environment: Both the university campus and the rooms and areas within it are in static locations, so a GPS-based system could be used to navigate the campus. However, once you enter a multi-story building, you will need to be able to differentiate what floor you are on, so you will need to take that into consideration.
Connectivity: As the app is designed to be used on-site within the university grounds, we can assume that users will have access to reliable connectivity, enabling them to use the app freely as they move around. If there are specific buildings or areas that are shielded, then we will need to take that into consideration too.
As you can see, there are many factors to consider. Part of the Thoughtworks process is a Discovery exercise that helps us uncover issues such as these and help us draw up a clear and detailed picture of what a client needs.
Chapter 3: Applying real-world context
Bridging gaps between the digital and physical world
One of the biggest challenges when designing and deploying AR experiences is bridging the gap between the digital and physical worlds without diminishing the experiences of people who aren’t moving between them.
For example, you shouldn’t design an entire retail store around your AR app because not every customer will have — or want — that app. Instead, the two need to seamlessly complement one another in ways that feel natural without being too obvious.
Fortunately, visual recognition can make that simple. By using detectable floor or display markings, you can easily anchor digital information and assets to physical locations in ways that enhance user experiences. They’re there for those who want them, without coming at the expense of those who don’t.
Understanding your virtual mapping choices
As you mark and map your environment with digital anchors, it’s important to consider the wider context that your app will be used in, so you can make informed connectivity choices. For example, if you’re mapping a physical store, then the context that store exists within is unlikely to change significantly. But, let’s say that you’re mapping the interior layout of a cruise ship. While its interior landscape isn’t going to change, its physical location most certainly is, which will affect whether you choose beacon-based connectivity or GPS-based methods.
Think about the environment you’re mapping from both a connectivity and flexibility point of view. In terms of connectivity, how reliable is cellular or wifi connectivity within your environment?
From a flexibility perspective, consider how much the environment you’re mapping is likely to change. Thinking about retail stores again, their physical location may not change, but their internal layout likely changes to at least some degree as seasonal items become available and special sales occur. In the case of a construction site, the physical environment will change dramatically over time.
Making XR a natural part of your ecosystem
Throughout your XR development and deployment journey, it’s essential to keep in mind that, above all else, your ultimate goal is to make XR experiences a coherent and complementary part of your overall ecosystem. Therefore, they should enhance experiences in ways that work for your customers and staff and naturally align with how those people behave, rather than forcing them to change their behaviors.
To do that, you need to think creatively about the best ways that XR can help solve the challenges faced by those users. For example, if you’re creating an app for finding products in a large store. Rather than assuming that users will hold their phones up as they walk around, you could use AR to lead people in the right direction, then use gyroscopes in mobile devices to sense if they have put the phone down, and cause the phone — or the user’s smartwatch — to vibrate when they get to the correct aisle.
It’s also important to recognize how behaviors and experiences can differ across user groups. For example, if you were creating an app for marathons, it could be used by runners, spectators, and staff — all of whom need different information, delivered in different ways. The runners need to know where the water and first aid stations are, while their families want to find them in the crowd, and the event staff needs to know where everyone is. The runners will likely favor watch-based, vibration, or audio cues over needing to pull their phones out. Spectators may want to take photos as their family member comes around the corner so the phone would be preferable for them.
Chapter 4: Ensuring XR has the right impact on your business
Great results start with strong onboarding
If you’ve built an XR app around user needs, made informed mapping decisions, and chosen connectivity options that align with the real-world context your app will be deployed in, you should be well on your way to achieving your XR business goals. But, one more step also needs to be carefully considered — onboarding and user enablement.
For your app to deliver great user experiences, those users need to understand exactly how to engage with it. If your app uses specialist hardware like headsets, users may need guidance on how to use that hardware. Then, once they’re in the app, they may need help getting to grips with the new experience and engaging with everything it has to offer. And in highly immersive use cases like VR, they may even need offboarding support to ensure a steady return to the real world.
It’s up to you to ensure that support and information are available. If you’re creating an app for internal teams, a whole-team training session may tick a lot of the boxes, but you’ll still need to make technical support available. For customers, they’ll look to whichever member of staff is closest for that support — turning your entire team into technical support. Those employees will need adequate training so that customers can get the information they need whenever they need it.
Adapting workflows to your augmented world
During the XR development and deployment process, most of your attention will naturally go on exactly that — development and deployment. But you can’t lose sight of what comes next.
Once an app is deployed, it’ll represent a change for both you and your users. You’ll need to consider how your app will change the processes, touchpoints, journeys and experiences that sit around it. For example:
If an app supports a single specific workflow like stock management, how will training and processes within the warehouse need to evolve alongside it?
Who will be responsible for rolling out mapping across locations if you’re creating something like an AR shopping map?
Are the experiences consistent across locations, unique to each location, or a combination?
If each location — e.g., each store in a chain — can modify the experience, what skills do they need? How will this impact other parts of your ecosystem (e.g., sales and stock tracking)?
Who is responsible for rolling out all necessary hardware and digitally onboarding employees? And can that hardware be remotely updated and secured?
How can your team use your XR app to create new opportunities for upselling, cross-selling, and increasing customer engagement? Do they have the power to drive those things directly through the app?
It’s a complex and multi-faceted part of the journey. But at its core, it’s just a matter of looking at the new augmented normal you’re creating, and ensuring that you align your workflows with it, so everyone can get the most from what you’ve created.
Mastering ongoing management
The last piece of the puzzle that you’ll need to consider is how your XR app and any associated hardware and software will be managed. It’s worth taking the time to plan out how you’ll:
Keep track of your physical devices to prevent loss, theft, or breakages
Roll out software updates — especially if your apps won’t be delivered through an app store
Scalable your infrastructure up if you need to support larger numbers of concurrent users
Manage security, and how things like new cameras or updated mapping of locations could impact security and privacy
Seize your chance to define this bold new world
While XR experiences are becoming an increasingly common part of our digital and physical lives, they’re still evolving at a rapid rate. It’s a whole new world of user interaction, and frankly, we’ve still got a lot to figure out. People are conditioned to click blue links or tap icons, but how we interact with virtual objects is still an active area of research.
Unfortunately, that means there’s no single “right” way to apply XR within your organization today. But, for those willing to jump in and use deep knowledge of their customers and users, there’s a huge opportunity to have a massive positive impact.
It’s a space that’s still being defined. However, if you have a creative idea to apply XR in your organization, the technology represents one of the few current opportunities to deliver truly differentiated and innovative experiences.