Technological progress increasingly asks workers to adopt tools they had no part in choosing. As tasks shift from manual to computerised processes and organisations standardise around shared platforms, employees are often left with limited discretion over whether, or how, they engage with a given system [1]. This is a widely documented condition of modern work, extending well beyond XR or any single technology.
MOTIVATE XR operates squarely within this reality and treats it as something worth studying directly. Workers across the project’s five industrial pilots don’t opt into XR training. It arrives folded into certification requirements, onboarding procedures, or safety protocols, whether in aerospace, aluminium manufacturing, energy distribution, home appliances, or human-robot hybrid manufacturing. The authority to require it sits elsewhere: with regulators, professional standards, or an employer, rarely with the trainee putting on the headset.
This arrangement sits somewhat uneasily with much of what is known about why people accept new tools. The models built to explain technology adoption were designed with a voluntary decision in mind, one where perceived usefulness and ease of use shape whether someone chooses to engage at all, and a poor impression shows up directly as low or no usage [1]. Take away that choice, and the relationship no longer works the same way. What’s worth asking about design and evaluation shifts along with it.
A Different Kind of Use
This distinction has a history in information systems research, where it appears under several closely related labels: non-discretionary use, mandated use, and mandatory use all describe the same underlying condition, a system employees must use as part of their job rather than by choice. Hartwick and Barki frame voluntariness not as binary but as a continuum [2]. Venkatesh and Davis extend the Technology Acceptance Model specifically to account for this condition, showing across four longitudinal field studies that voluntariness itself changes which factors drive a system’s acceptance [3]. Brown, Massey, Montoya-Weiss, and Burkman push the point further, studying what they term mandated technology use directly, and showing that the relationships linking usefulness, ease of use, and adoption intention shift once choice is removed from the equation [4].
One consequence follows immediately: usage numbers lose their diagnostic value in a mandatory setting. Low usage is a signal when use is voluntary, a warning that something isn’t working. Once use is no longer a matter of choice, usage stays high almost by definition, and the signals worth watching move elsewhere: comfort, engagement while training, how much of what’s learned survives past the headset. It’s part of why TU Delft, as MOTIVATE XR’s partner leading this evaluation strand, tracks perceptions with the Technology Acceptance Model 3 (TAM3) both before training and immediately afterward, within the cross-pilot evaluation run under MOTIVATE XR’s Task 7.8 [5], with a longer-term follow-up planned as the evaluation moves into its next phase.
Two Roles, Two Degrees of Choice
A related distinction deserves equal attention. As with most XR training systems, MOTIVATE XR’s platform splits into an authoring layer, used by trainers and instructional designers to build modules, and an experiencing layer, used by trainees to complete them. The project is explicitly aware of this split and of the different needs it creates.
This split carries a difference in agency, not just functioning. Building a module leaves a trainer with real control over content and structure. A trainee assigned that module rarely has the same latitude and lacks the exit option that classic accounts of organisational dissatisfaction treat as one of two available channels, alongside voice, for expressing discontent [6]. A trainer who dislikes how a module is built can rebuild it; a trainee has, in practice, only feedback as an outlet, since simply not using the system isn’t an option. So, the non-voluntary character of the platform applies asymmetrically, depending on which side of the tool you are standing on. Designing for the person authoring a module is a different problem than designing for the person required to complete it, even when an evaluation framework treats both under a single label: “the user.”
Five Pilots, Five Different Challenges
What mandatory use means shifts depending on what the XR tool is built to do, and across MOTIVATE XR’s five industrial pilots, that purpose varies considerably.
- Aerospace uses XR for specialised, safety-critical training inside tightly regulated, accuracy-sensitive workflows. The tools lean heavily toward the experience side here: structured, step-by-step immersive scenarios worked through in focused sessions, with little room to improvise.
- Aluminium relies on XR mainly to make visible what’s otherwise hard or unsafe to observe up close during live operations, favouring visualisation over strict task-by-task guidance.
- Energy Distribution turns to XR for scenario-based training around situations that are rare, hazardous, or both. Here the authoring side plays a central role, since a credible hazardous scenario depends on trainers translating expert judgment about failure modes that may rarely occur in the field.
- Home Appliance operates inside production-line workflows shaped by efficiency pressure and short training windows, where fitting XR into an existing routine without slowing throughput matters more than the sophistication of the experience itself.
- Human-Robot Hybrid Manufacturing uses XR to build understanding of both a technical system and the dynamics of working alongside it, with XR functioning as one component within a broader training ecosystem rather than standing alone.
Set against the authoring-experiencing split, these five contexts make clear that “the user” of a non-discretionary XR system rarely means one person in one fixed relationship to the tool. Designing and evaluating the platform well means holding onto that difference rather than flattening five pilots, or both sides of the tool, into a single case.
A Harmonised, Evidence-Based Evaluation Framework
Because the platform doesn’t mean the same thing in any two of these settings, comparing results across pilots calls for one framework applied consistently. TU Delft’s cross-pilot evaluation brings together three instruments; each aimed at a different piece of the mandatory-use problem described above.
Technology Readiness Level (TRL) assessment runs at baseline and again at the close of the beta phase, anchoring the platform’s technical maturity so user feedback gets read against what the system can realistically do at that stage rather than against the standard of a finished product [7, 8]. That anchor matters because use isn’t discretionary here: without it, a trainee’s frustration with an early-stage feature could easily be mistaken for a design flaw rather than a maturity gap that later development will close.
Usability and user experience come through the System Usability Scale (SUS) and the User Experience Questionnaire (UEQ), both administered right after training [9, 10]. Since usage figures carry no real signal when use is required rather than chosen, these two instruments surface friction directly, asking how the experience felt rather than inferring anything from whether people kept using the system.
Acceptance is tracked with TAM3 at two points so far, before any hands-on contact and immediately after training [5], with a delayed measurement after a period of practical use planned as the evaluation moves into its next phase. That staged structure, once complete, will let the evaluation tell a first impression apart from a judgment that has survived contact with the job.
What Comes Next
With beta-phase data collection now complete across all five pilots, MOTIVATE XR is moving into the next phase of both the platform and its evaluation, as it transitions from beta toward the final release. Measurement will widen from system usability to the project’s core operational targets, training time and error rate in particular. A longitudinal survey, again built on TAM3, will follow how acceptance holds up after a stretch of sustained use and platform maturation, rather than resting on judgments formed shortly after first contact.
Trainers will also enter the evaluation more directly in this phase. The beta phase focused on the trainee’s side of things; the next round looks more closely at the training experience itself, comparing sessions run with XR against sessions run without it. Given the authoring-experiencing split laid out earlier, this is also a chance to evaluate both sides of the platform on their own terms, rather than folding trainer and trainee feedback into one undifferentiated measure.
Conclusion
Designing mandatory, non-discretionary use isn’t a minor footnote to Human-Centred Design in industrial XR; it’s a condition that changes what HCD has to track. Take away the option of simply stopping, and usefulness and ease of use don’t stop mattering; they just stop registering as usage statistics and start showing up somewhere else: in comfort, in engagement, in whether training carries over to the job. Add the split between those who build the training and those who complete it, and the design and evaluation brief sharpens further still: two different users, with two different degrees of say in how it turns out, both relying on the same platform.
References
[1] Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340.
[2] Hartwick, J., & Barki, H. (1994). Explaining the role of user participation in information system use. Management Science, 40(4), 440–465.
[3] Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186–204.
[4] Brown, S. A., Massey, A. P., Montoya-Weiss, M. M., & Burkman, J. R. (2002). Do I really have to? User acceptance of mandated technology. European Journal of Information Systems, 11(4), 283–295. https://doi.org/10.1057/palgrave.ejis.3000438
[5] Venkatesh, V., & Bala, H. (2008). Technology acceptance model 3 and a research agenda on interventions. Decision Sciences, 39(2), 273–315. https://doi.org/10.1111/j.1540-5915.2008.00192.x
[6] Hirschman, A. O. (1970). Exit, Voice, and Loyalty: Responses to Decline in Firms, Organizations, and States. Harvard University Press.
[7] Mankins, J. C. (1995). Technology readiness levels: A white paper. NASA, Office of Space Access and Technology.
[8] European Commission. (2014). Technology Readiness Levels (TRL). HORIZON 2020 – Work Programme 2014-2015.
[9] Brooke, J. (1996). SUS: A quick and dirty usability scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester, & I. L. McClelland (Eds.), Usability Evaluation in Industry (pp. 189–194). Taylor & Francis.
[10] Schrepp, M. (2015). User Experience Questionnaire Handbook. https://doi.org/10.13140/RG.2.1.2815.0245
Author
Delft University of Technology
Nicola Franciulli is a PhD Candidate at the Delft Centre for Entrepreneurship, within the Faculty of Technology, Policy and Management, TU Delft. His research examines how Human-Centred Design can improve the acceptance and effective use of new technologies in mandatory adoption contexts, where use is required rather than voluntary. He approaches technology adoption as a value design problem, bridging organisational goals with users lived experiences.
