The future of machine design is coalescing around a number of trends, including self-driving vehicles, teleoperation of robots, and remote human assistance, across essentially all industrial domains.
Indeed, intelligent autonomous and robotic system technology is transforming the way we work, and smart systems are taking over the worksite. That’s why the interaction between the human and the machine is now at the center of design: with autonomous systems working alongside humans, the man-machine interface has become safety-critical. Not only that, but people’s expectations are that products must go beyond looking great, to also working flawlessly, with superb ergonomics.
It’s no longer enough to simply make a powerful or efficient or useful mechanical object — everything also needs to work intuitively. That’s why today, what’s delivering the highest value for OEMS is electronics and control systems that are fully integrated with mechanical design.
Driver-in-the-loop assessments are critical to understanding human-machine performance, and both novice and skilled operators are a vital part of the test validation process. This is even truer with advanced driver assistance systems (ADAS), autonomous equipment, or remote controlled equipment, where the nature of the human interaction with the machine is fundamentally changing.
In fact, human-centric design is crucial in a number of application spaces, including:
- operational-level system validation
- allowing you to test the mechatronic systems of tomorrow
- generating simulation-based data for deep learning
- autonomous control system development
- building simulation-enabled digital twins for better decision making in product engineering and maintenance
- creating simulators to engage clients in immersive product demonstrations and operator training
Of course, if you are an equipment OEM, you already know that product success hinges on the emotional reactions of real people.
Considering the ever increasing complexity of today’s equipment and on-board systems, putting real people into contact with the virtual equipment is a key part of successful and efficient product design, and should be done early and often in the design process.
Driver-in-the-loop simulation is one excellent way to accomplish this. However, not all driver-in-the-loop simulations provide the necessary capabilities. They must be able to create environments in which operators can interact with virtual machines as if they’re interacting with the real equipment, so they can offer precise and relevant feedback.
In fact, OEMs are increasingly turning toward using engineering-grade simulation and digital twins for decision-making around good product design, for a variety of reasons:
- High-fidelity simulations make it possible to test edge cases that can be impossible or unsafe to test on a real equipment – indeed, simulation can be more realistic than limited testing on proving grounds. It’s also better controlled and more repeatable.
- Engineering-grade simulations elicit realistic responses and behaviors from human operators. Real operators — in other words, the user that’s best poised to be your ultimate evangelist — are put into direct contact with test systems much earlier in the design cycle.
- Engineers can deploy a fail-fast, fail-well strategy, with evaluate/accept/reject models for rapid prototyping, and move confidently and cost-effectively to the next stages of development.
- Simulators can also be used to test new control concepts as well as ADAS or operator assistance systems.
- With AI in the loop of some autonomous control systems, there’s also a real risk of operators dropping out of the loop during periods of inactivity, so handover between autonomous and manual control is also an issue that can be assessed in driver-in-the-loop simulators.
- Testing systems with distracted or fatigued operators, functional safety testing, and even controller tuning can also be performed using simulators.
The main components of a driver-in-the-loop simulator can be broadly separated between software and hardware:
First, a real-time multi-domain simulation of the machine or equipment is needed. It’s essential that the simulations be engineering-grade mechanical dynamics models, rather than game-based. It may even be necessary to translate models from existing in-house multibody applications.
Secondly, it must be possible to simulate interaction between the machine and its operating environment. That includes contact between the machine and the objects of the worksite, as well as track and tire modeling for construction and agricultural equipment that can include real-time soil mechanics models for implements that are digging or plowing the soil, sand, or rock.
Finally, the software must be capable of simulating the complete worksite, up to and including workers, other machines and traffic, and even bad weather.
From a hardware perspective, a simulator incorporates a visual and an audio system of course. That system can range from very simple VR headsets on the desktop to multiple TVs to even immersive surrounded warped and blended projection systems.
Motion is also often a key component in driver-in-the-loop simulators, and we believe that both motion and haptic control for steering and other controls are important in creating a high-fidelity assessment. However, neither motion nor haptics can accurately render that correct behavior if the simulation that drives them is not engineering-grade.
Finally, driver-in-the-loop solutions can include a mock-up of the operator controls and HMIs. These can optionally integrate even the actual electronic control units (ECUs) and control hardware from the real machine, or you can simulate them. Mock-ups can of course range from very simple desktop configurations to complete cab replicas.
A couple of examples from our clients will help concretize the use case for human-in-the-loop simulation for OEMs.
The first is from a major forestry OEM that was prototyping the use of inverse kinematics (IK) for forwarding operations. The idea was to use IK to make this operation easier to learn and use.
By first using simulation, they eliminated the need for a physical prototype at that early control design stage, and enabled the engineers to tune the controls. Once that initial control concept was proven in software the physical control hardware controls could be integrated into the driver version.
The result: a powerful new control feature for the OEM’s log forwarder product, which resulted in their customers seeing improved efficiency. Not only did that mean a single operator could load a lot more trucks every year, it also meant it was easier to use, and to train operators on, since the operation was simpler.
It is important to note that there’s a significant skills shortage in the forestry industry — logs often cannot be brought from forest to mill due to these shortages, with mills even closing operations during the year as a result. That’s why making machines easier to use and more productive is key.
The next example is from a major construction equipment OEM. Of course, earth moving equipment relies on sophisticated hydraulic systems comprised of valves and pumps and hoses as well as control system and sensors. These hydraulic systems must be matched with an appropriately sized engine, and then optimized for performance and fuel economy essentially by tuning control parameters.
This tuning is very costly on physical prototypes, especially at remote proving grounds. The OEM’s engineers in this case already had run the hydraulic simulation tools, but they needed to develop a full-scale simulator of a complete machine incorporating hydraulic, mechanical, electrical, or electronic subsystems.
The team needed tools that would enable them to model all those subsystems in a single environment, to perform real-time simulations of the complete system. This approach would let them make informed design trade-offs that took into account not only performance and efficiency but also feedback from the operator actually driving the machine.
A modeling and simulation environment that supported multiple domains was essential to the company’s concurrent engineering approach, in which teams develop various subsystems of the final product in parallel and then converge on an optimal solution.
Using their existing real-time hydraulics models, they were able to connect that to a mechanical dynamic simulation of the machine as well as soil mechanics simulations using Vortex Studio.
All of this was combined with a multi-channel synchronized 24-projector-based visual system driven by Vortex Studio, as well as a 6 DOF motion platform with a complete cab and control mock-up.
The result for the OEM was a 30% reduction in physical prototyping, with considerable cost savings as a result. It also allowed them to reduce new product development lead time and improve product quality, since more test scenarios are considered.
Currently, it allows engineers to replicate any issue that may arise in the field and troubleshoot problems and quickly find solutions, so the impact has been quite significant in their design process, and we continue to work with this OEM.
Well before the pandemic, economies were shifting from providing goods and services to a total experience package, where the experience itself is becoming the product. Naturally, customer expectations as well as product design are changing like never before in this age of experience.
Many industrial equipment manufacturers are now in a race for solutions to meet these expectations, which brings into focus the human factor — the combination of physiological and psychological elements that defines the total operating experience.
Fundamentally, it is this experience that captivates humans and motivates them to purchase and even evangelize the machines that they work with, so this all comes back to the core principle, which is that the interaction between the human and the machine is at the center of equipment design.
In the end, designing, building, integrating, and supporting driver-in-the-loop simulators is a multidisciplinary effort, and CM Labs has specialists and partners who are available to assist you with special projects in the area of driver-in-the-loop simulation, including simulation modeling, software integration, simulator design and audio-visual design. We also have a network of partners globally to provide local support in the Americas and in Europe and the Asia Pacific. Simply get in touch with us to speak to one of our experts, who can guide you in the early stages of defining and evaluating your project
