Smart Immersive Modeling Lab

Home People Research Publications Teaching YouTube Sponsors Memories Contact
Research Objectives:

Robots are expected to have a more significant role in numerous aspects of our everyday lives, such as health care, education, entertainment, defense, and security. Societies are undergoing numerous changes due to the rapid expansion of the robotics industry. While industrial robots can save energy, time, and resources by optimizing processes, service robots can change social constructs. Due to the undeniable co-existence of humans and robots in the coming years and potential exposure of untrained end-users, robots should be enabled to deliver context-aware tasks. While Industry 4.0 focused on optimizing factories through promoting autonomy by relying on cyber-physical systems as well as internet of things and systems, Industry 5.0 is refining the interaction between humans and machines by refocusing on the human element. Many countries have offered strategic initiatives to implement Industry 4.0, and significant research efforts have been made to further develop Industry 4.0 concepts.

Industry 5.0 is defined to complement these efforts by using the creativity of human specialists in cooperation with effective, intelligent, and precise machines and robots, which further emphasizes the need to deliver appropriate, optimal, and efficient human-robot interactions. Context-aware robotics is built upon the modeling of human behavior in human-robot interactions (HRI), as it has been shown that people's behavior changes while interacting with robots. Robots must have precise knowledge about humans in different tasks to provide safer more efficient interactions. Having better understanding of human behavior in an industrial setup can help us to create more reliable and effective collaborative robotic environments. Intent prediction is a growing field HRI. It has been proven humans can extrapolate intentions from observed actions from an early age. Given the advancement of artificial intelligence and computing, our team is focusing to study and model human behavior in to successfully design and develop spatial-temporal algorithms for robots to predict a human’s intention from his/her body gesture and movement patterns. To secure a safe data collection environment, all collaborative scenarios are and will continue to be conducted using immersive virtual reality via HTC Vive technology. The location and rotation data of the head, hands, elbows, knees, back and feet of immersed participants are and will be collected through HTC Vive trackers and Leap Motion sensors. The observed bodily movements will be utilized to analyze the spatiotemporal patterns behind those movements and underling intentions. The immersive environment and virtual robots have been already developed and initial data collection is completed. The team will focus on using the collected data and additional data to be collected in parallel to analyze and design recurrent and convolutional graph neural networks models to classify the underling intentions. While the recurrent network structures will study the temporal dependencies of gestures, the convolutional graph structures will explore the concurrent spatial relations among different body parts (e.g., hands and feet).


Working papers:

  • Kamali Mohammdzadeh, A., Rafieian, A., Masoud, S., Intention Prediction and Trajectory Tracking for Manufacturing Cobots: An Extended Reality Approach, IEEE Transactions on Automation Science and Engineering, Working Paper.

  • Rafieian, A., Kamali Mohammdzadeh, A., Masoud, S., Unsupervised classification for Task Recognition in Human-Robot Interaction, Computers and Industril Engineering, Working Paper.