Hands-On (Heads-In!) Assembly with Mixed Reality!

Earlier this summer, I had the opportunity to attend AWE USA 2024 in Long Beach. AWE events never fail to immerse attendees in all aspects of XR: from industry thought-leadership to best-in-class ideas, products, and services. This year, in particular, there was an overwhelming sense of community and history of XR, as AWE celebrated the pioneers in XR among a plethora of engagements, activations, and awards.

An Engaging Workshop

One engagement at AWE that captured a lot of attention and excitement was an interactive workshop that I facilitated as part of AWE’s exclusive enterprise “Focus” program. There, I featured Arkio collaboration software and Meta display hardware to pit Mixed Reality (MR) against various other media for construction and assembly instructions in a fast-paced showdown.

The intent was to discover and learn which medium was the most effective in supporting complex, accurate, and fast assembly, not unlike what happens every day in the construction or manufacturing industries. To do so, about twenty business executives and professionals formed teams to compete against others using MR, architectural construction documents, LEGO-like assembly instructions, physical models, and verbal instructions.

We formed six teams, and each team was given a set of 50 LEGO pieces and a brief opportunity to see a sample model of the target final build: a complex “architectural” sculpture including hidden interior pieces and an operable, hinged floor. Each team was given a set of assembly instructions, and 15 minutes to build the assembly.

Two of the teams used Arkio in Meta Quest headsets (a mix of Quest 3 and Quest Pro). One of those teams was presented with a virtual model of the complete target build, while the other saw the build in twelve incremental phases, each step displayed side-by-side. Likewise, of the four teams using analog media, two of them (2-D architectural documents and physical model) referenced a completely assembled prototype, while the other two (LEGO instructions and verbal instructions) referenced as step-by-step set of instructions in the same twelve steps as one of the teams in MR.

Teams hypothesized which assembly instruction media would enable the fastest and most accurate builds, and after a brief orientation to basic features and tools in Arkio (navigation, scaling, teleporting, views, and sections), we started the 15-minute timer. Teams started furiously assembling LEGO pieces!

Which team do you think was the fastest? Who was the most accurate? And why?

The Results Are In…

Two of the teams finished their assemblies quickly: the team following LEGO instructions, followed closely by the team referencing a physical copy of the final build. In third place came the team using MR with the model broken down into twelve assembly phases.

The remaining teams did not finish within the 15-minute timeslot, but the MR team referencing a virtual model of the final build was about 90% complete when time expired. The teams using architectural drawings and verbal instructions were each about 50% complete in the same time.

“Team Procedural,” referencing LEGO-like step-by-step assembly instructions, narrowly took 1st place, and bragging rights to take home with them!

“Team Tactile,” referencing a complexly built physical sample assembly, finished in a close 2nd place.

“Seal Team 6,” using Mixed Reality to reference the assembly digitally, with step-by-step phases, came in 3rd place.

“Team Spatial,” using Mixed Reality to reference a completely built digital assembly, came in 4th place.

“Team Verbal,” referencing 100% written instructions, tied for last

“Team Visual,” referencing 2-D architectural drawings of the completed design intent, tied for last

Participants then went around to other teams’ tables to compare (and laugh at!) the assemblies their competitors built, and then we sat down to analyze and discuss some lessons learned. Teams identified the core strengths and weaknesses of each medium; they also noted which other capabilities and additional information they wish they had access to.

Lessons Learned

The discussion elicited several key insights:

In retrospect, the fact that the team using LEGO-like instructions performed the best for an assembly that consisted of LEGO pieces should come as no surprise. It’s an instruction format well-designed and validated by a best-in-class company! Those who develop assembly instructions of any kind – whether for toys or for heavy construction – would do well to consider emulating their precedent.

A team’s self-organizational capabilities and social dynamic was at least as important a factor as any strengths/weaknesses inherent in the assembly media. Case in point: a team comprised of seasoned professionals who had all worked together clearly outperformed a team of younger professionals who had never met before the workshop.

Teams using MR needed extra time to get oriented and onboarded to a new software tool, to the detriment of their performance during the time trial. It was widely accepted that, with the experience of their first experience with the software under their belts, they would have performed significantly better in future challenges. Like in the real world, any process change, and particularly one from analog to digital, requires a realistic investment in training and onboarding before productivity gains can be expected.

Overall, teams referencing assembly instructions in phases outperformed teams using complete assemblies as reference material. This has dramatic implications for the differences between traditional architectural construction documents (which depict completed “design intent” of a building) versus design for manufacturing and assembly (DfMA) documents and construction work packages (which acknowledge construction means and methods). That is, documents that describe how an assembly goes together outperform those that describe what the final build should look like.

While many participants presumed that the teams using MR would wildly outperform those using traditional media, that simply did not happen. This was surprising to the participants in a workshop featured within the context of an XR conference, where it was presumed that the scenario was biased to promote MR.

The consensus among the professionals, however, was that if this scenario had more closely reflected the “real world” challenges and characteristics of modern-day global enterprises, that MR would have ultimately outperformed other media. Those characteristics included:

Remote teams – if teams were working remotely on parts and sub-assemblies in a global supply chain, there was simply no way physical atoms could be transferred as quickly as digital bits, and those working in MR would have easily outperformed the others

Design/assembly changes – there are almost always changes in production of any assembly in manufacturing or construction, and teams relying on digital media and MR would have the advantage over teams with analog instructions.

Digital prototyping – if the task had been expanded to include design of a prototype assembly first, followed by production of multiple physical instances, design in a digital medium like MR would become the obvious tool of choice for developing that initial prototype

Finally, when participants were given the hypothetical opportunity to combine their media with any one other team, they overwhelmingly chose the MR teams as desired collaborators. The obvious take-away: MR has a place in assembly, whether in manufacturing or construction, and would ideally be implemented in a scenario where integrated with other, analog communication media to take best advantage what each offer.

Put Insights into Action

Overall, the workshop was highly successful: participants were highly engaged, and simply didn’t want to put their work down when the time expired. Additionally, everyone came away with a renewed appreciation for both Mixed Reality, and insights and ideas about how to successfully implement MR to support real-world assembly in enterprise manufacturing and construction.

If you missed the opportunity to attend AWE in Long Beach, it’s not too late to learn about the impact XR can have on your enterprise. You can join me at AWE EU this fall in Vienna, where I’ll be the program chair for the Enterprise VIP program – contact me for a registration discount code! Along with other “Enterprise Experts,” I would love to hear about your needs, vision, and plans for harnessing XR to make more effective decisions in your business.