NON-EXPERT PRACTICAL APPLICATION OF AI VISION SYSTEMS IN DESIGN ENGINEERING PROJECTS
Year: 2023
Editor: Buck, Lyndon; Grierson, Hilary; Bohemia, Erik
Author: Garland, Nigel Patrick; Wade, Russell; Palmer, Sarah
Series: E&PDE
Institution: Bournemouth University, United Kingdom
Section: The effect that design and engineering have on global co-habitation
DOI number: 10.35199/EPDE.2023.2
ISBN: 978-1-912254-19-4
Abstract
Design projects units are an essential element for Design Engineering students at our University and represent 20 ECTS credits at level-5 and 30 ECTS credits at level-6. Students integrate and apply knowledge from a range of taught units and subjects they may be unfamiliar with through self-directed learning. Students also demonstrate they meet specific elements of the Engineering Council’s learning outcomes for accredited programmes. For level-5, students work towards individual and group projects: level-6 students work on a single individual project of their own proposal. Recently, level-6 students have proposed and designed projects requiring AI vision control-systems. These projects presented a problem for supervision, especially during the pandemic, as the department lacked technical expertise, equipment, and experience in application. Students therefore treated these AI subsystems as “black-box” exercises. To address this issue, technical requirements were compiled from typical use-cases and combined with accessibility of learning material, extent of ecosystem, usability, and compatibility. A range of AI technologies were evaluated before selecting the Nvidia Jetson Nano: these provide a complete on-board workflow of deep-neural-network (DNN) retraining and deployment. From the existing literature, a streamlined training-programme was developed to introduce the technology to both level-5 and level-6 project students. This provided hands on experience through familiarization with the interface and pretrained DNN models for image classification, object detection, semantic segmentation, and pose estimation. Level-5 project students were assigned a group project to design and build a part-sorting technical demonstrator utilizing AI object-detection integrated to PLC control. The AI workflow was executed entirely on-board with the Jetson Nano. Students collected and annotated images of scratched and unscratched plate components to create a dataset (ground-truth) before retraining an existing DNN (SSD-mobilenet v2) using Pytorch. Students compiled a simple python script to call the DNN within the device’s DetectNET framework and provide signaling over GPIO to the PLC when detecting scratches and plates. Students also designed the electronic interface and programmed the PLC using ladder-logic to provide electrical control of their sorting machine’s electro-mechanical functions. Level-6 project students were able to integrate the technology into projects where appropriate and two students chose to do so. One project utilized a similar object-detect workflow to check if chili peppers are ripe for harvest: the only change was to off-board the image capture with ground-truth annotation through an alternative software package (CVAT). The second student used an existing semantic segmentation network (Multi-Human-Parsing) to identify people trapped under collapsed buildings, with GPIO controlling alarms when a threshold is reached for particular classes. level-5 and level-6 students gained valuable knowledge in the practical application of AI in control systems. The level of learning suitable for proving a design through the use of technical demonstrator rather than at a production level. Three technical issues were identified through this work: Specific human-errors in the annotation process were only identifiable once exported to another annotation package such as CVAT. CVAT became unsupported after sanctions were imposed on Russia and Intel withdrew operations. Like many semiconductor dependent resources, Jetson Nano’s became difficult to source.
Keywords: Artificial Intelligence, Machine Learning, Design, Engineering, Projects