A closed-loop hybrid-signal brain-machine interface robotic arm control method based on augmented reality by WiMi enhances visual feedback for the user by combining BMI based on electroencephalogram (EEG) signals and eye tracking techniques to achieve closed-loop control during the control of the robotic arm.

WiMi Hologram Cloud Inc. (NASDAQ: WIMI) ("WiMi" or the "Company"), a leading global Hologram Augmented Reality ("AR") Technology provider, today announced that an AR-guided assistance method is outlined to provide enhanced visual feedback to the user for closed-loop control using a hybrid BMI that combines EEG signals and eye tracking for intuitive and effective control of the robotic arm.

A closed-loop hybrid-signal brain-machine interface robotic arm control method based on augmented reality by WiMi enhances visual feedback for the user by combining BMI based on electroencephalogram (EEG) signals and eye tracking techniques to achieve closed-loop control during the control of the robotic arm. The system integrates the functions of BMI, eye tracking, image processing, automation, and AR interface to allow the user to perform object manipulation tasks. Image processing is used to segment all potential rectangular objects from the image of the workspace. The user can use eye tracking to select the segmented objects. The output decoded from the BMI is used to (1) confirm the user-selected object, (2) switch the sequence of actions, and (3) continuously control the aperture and height of the fixture during gripping and lifting. The user-selected object and the status of the gripping and lifting operations are fed back to the user in real-time via a computer screen using AR technology. Finally, the robotic arm performs stretching, grasping, lifting, delivering and releasing tasks based on the output of the hybrid BMI decoding. The system device equipment includes an eye tracker, EEG headset, computer, robotic arm, and USB camera. Interaction between the user and the system is performed using hybrid gaze BMI and AR-enhanced visual feedback.

A closed-loop hybrid-signal BMI robotic arm control system based on AR of WiMi includes:

Augmented Reality Control Interface: Through AR technology, an intuitive and easy-to-use control interface is designed so that users can control the movement of the robotic arm through gestures or voice.

Acquisition of EEG Signals: The electrical signals from the user's brain are captured using EEG equipment and transmitted to a computer.

Processing EEG signals: Using signal processing algorithms, EEG signals are filtered, features are extracted, and other operations are performed to extract the commands for the movement of the robotic arm that the user wants to control.

Generate Control Signals: Based on the user's EEG signals, control signals are generated to control the movement of the robotic arm.

Robotic Arm Control: the control system of the robotic arm, including the selection and construction of hardware devices such as motor drivers, position sensors, and control chips.

Realization of Closed-loop Control: The collected feedback information on the position of the robotic arm is transmitted back to the computer to realize closed-loop control and ensure the precise and stable movement of the robotic arm.

Real-time Control: real-time transmission of user control signals and robotic arm position feedback information to the robotic arm control system, realizing real-time control.

WiMi tested the developed closed-loop system (with AR feedback) against the current conventional open-loop system (with visual inspection only). The results show that AR feedback significantly reduces the trigger commands for grasping and lifting objects compared to tests using only normal visual inspection. In addition, the height clearance of the fixture during lifting was reduced. Hybrid BMI users benefited from the information provided by the AR interface, which increased efficiency and reduced cognitive load during the gripping and lifting process. The closed-loop system of AR feedback provides a novel and effective way for users to control the robotic arm using hybrid BMI, further improving the conventional control system by integrating more advanced image processing and machine learning algorithms to enhance the segmentation of the object in the workspace and BMI signal decoding.

Technology is constantly progressing and developing, and the application scenarios of robotics are becoming more and more extensive. As an important application field of robotics technology, the robotic arm control system has a very broad market prospect. Especially in the manufacturing industry, robotic arm control system has become an indispensable tool.WiMi's closed-loop hybrid-signal brain-computer interface robotic arm control system based on augmented reality control adopts the augmented reality technology and the brain-computer interface technology, which can improve the accuracy and speed of the operation, and at the same time ensure the safety of the operation, and it has a very wide market prospect. WiMi will also further refine and improve the system to make it applicable to the real market demand for the current industrial production environment.

About WIMI Hologram Cloud

WIMI Hologram Cloud, Inc. (NASDAQ:WIMI) is a holographic cloud comprehensive technical solution provider that focuses on professional areas including holographic AR automotive HUD software, 3D holographic pulse LiDAR, head-mounted light field holographic equipment, holographic semiconductor, holographic cloud software, holographic car navigation and others. Its services and holographic AR technologies include holographic AR automotive application, 3D holographic pulse LiDAR technology, holographic vision semiconductor technology, holographic software development, holographic AR advertising technology, holographic AR entertainment technology, holographic ARSDK payment, interactive holographic communication and other holographic AR technologies.