Grant: Discovery Grants Program - Individual

Visual servoing, using the vision information for motion control, has improved the dexterity and accuracy of industrial robots in the past decade. Although a lot of research work has been carried out on the visual servoing of industrial robots, both the position-based visual servoing (PBVS) and image-based visual servoing (IBVS) strategies have its own advantages and disadvantages which need to be addressed to meet the industrial requirements. For example, PBVS requires a 3D CAD object model, a precise knowledge of the robot kinematics model and camera calibration model. While IBVS suffers the inherit drawbacks such as features leaving field of view (FOV), slow convergence and local minima, etc. The success of current PBVS and IBVS largely hinges on the prior information or availability of appropriate image features or the object model, effective visual servoing control algorithms and learning mechanism. Nowadays, the rapid development of advanced information technologies and artificial intelligence (AI) has inspired the robotics community to develop various intelligent robots. The AI technologies have the great potential to make the industrial robots work in the unstructured environment and deal with above mentioned issues for both PBVS and IBVS. Incorporating the AI technologies into the visual servoing, i.e., AI-driven visual servoing aims at equipping the industrial robots with the flexibility and learning abilities, improving the manufacturing performance and reducing the production costs. The challenges of AI-driven visual servoing lie in fusing all available sensory data such as encoder, camera, force sensor, optical coordinate measurement machines (CMM) and 3D scanner, selecting and reconstructing image features, and formulating control strategy that can interact with the unknown environment and can improve the performance through learning, etc. In the next five years, various AI technologies such as artificial neural network, reinforcement learning, and machine learning will be explored to meet the aforementioned challenges in both PBVS and IBVS. The project aims at developing AI-driven visual servoing of industrial robotic manufacturing systems with intelligence, robustness and adaptability, which can realize the seamless transfer of files from design to manufacturing in the real environment. The ultimate objective is to develop a systematic framework of embedding the learning skills via AI into industrial robotic manufacturing systems.