Note: The content and the pictures in this article are contributed by the author. The opinions expressed by contributors are their own and not those of PCBWay. If there is any infringement of content or pictures, please contact our editor (steven@pcbway.com) for deleting.
Written by Vincent Kok
In this article, I will be making a voice-activated Robot Car with the in-built microphone on Wio Terminal to recognize the go, stop and background noise by TinyML.
Trained a wake-up word recognition model using Codecraft that is powered by Edge Impulse!
In this detailed blog post, I will cover the following:
What's UART Serial Communication?
UART Serial Communication between Wio Terminal and uKit Explore
In layman terms, UART allows an embedded device such as Arduino to send data over to another Arduino via the TX (transmit) and RX (receive) line as shown below.
Image source: https://learn.sparkfun.com/tutorials/serial-communication/all
Let me give you an example case study.
Well, Arduino UNO has no built-in WiFi and hence it's not possible to do IOT related projects. With the understanding of the basic UART serial communication, I was able to leverage the ESP8266/ESP32 as the co-processor to the Arduino UNO so that the data collected from the sensors attached to Arduino UNO will be sent over to ESP8266/ESP32 to send over to the cloud platform such as web server, Blynk or FAVORIOT.
The TX and RX pins for Wio Terminal is available on pin 8 and 10.
TXD - Pin 8
RXD - Pin 10
Whereas for uKit Explore, there are also TX/RX pins available on the pin D0 and D1. You can check the full uKit Explore pinout here: https://ubtechedu.gitbook.io/ukit-explore/v/english/ukit-explore-quick-start/pinout
RX0 - D0
TX0 - D1
For the hardware connection, basically, you will just need to connect as below:
Wio Terminal (Pin 8) to uKit Explore (pin 0) - TXD to RXD
Wio Terminal (Pin 10) to uKit Explore(pin 1) - RXD to TXD
I will explain more on the software programming how it is done on 3.0 Program the Robot Car (uKit Explore - Arduino Mega 2560 based).
In this first section, our goal is to create an embedded machine learning model (voice recognition) using Codecraft platform.
There are 4 steps involved to train an embedded machine learning model using Codecraft.
1.Create the "Wake-Up Words Recognition (built-in microphone)" model
2.Data Acquisition (on-board)
3.Training and Deployment
4.Programming & Model Usage
STEP 1 Create the "Wake-Up Words Recognition (built-in microphone)" model
Go to https://ide.tinkergen.com/. Select "(for TinyML) Wio Terminal".
Click on "Model Creation" on the embedded machine learning box on the middle left. Then select the "Wake-Up Words Recognition (built-in microphone)" as shown below. Enter the name of the model according to the requirements.
Click Ok and the window will automatically switch to the "Data Acquisition" interface.
STEP 2: Data Acquisition (on-board)
There are 3 default labels (hi wio, background, and other words) that are automatically created for you.
You can use it without any changes unless you want to have different names for your labels. For my case, I changed two of the default labels as below:
hi wio changed to go
other words changed to stop
IMPORTANT: Now, you have to remember to change the labels on the default data acquisition program to reflect the correct modified labels:
Connect Wio Terminal and Upload Data Acquisition Program.
Note: You need to download “Codecraft Assistant” to be able to Connect and upload code on Codecraft online IDE.
Data Acquisition: In the upper right hyperlink, you will find a step-by-step introduction to data acquisition. Follow the instructions to collect data accordingly to your modified labels.
STEP 3: Training and Deployment
· Click on “Training & Deployment”, and you will be seeing the model training interface as shown below.
Select neural network and parameters. Select the suitable neural network size: small, medium, and large
Set parameters:
number of training cycles (positive integer),
learning rate (number from 0 to 1)
minimum confidence rating (number from 0 to 1)
The interface provides default parameter values of training cycles of 50, however, the accuracy was not very good. Hence I changed the training cycles to 100.
Click “Start training”. When you click “Start training”, the windows will display “Loading..”! Wait for the training to be done!
Observe the model performance to select the ideal model. In the “Model Training Report” window, you can observe the training result including the accuracy, loss, and performance of the model.
· In the “Model Training Report” window, click on “Model Deployment”. Once the deployment is completed, click “Ok” to go the “Programming” windows which is the last step before we deploy the model to the Wio Terminal.
STEP 4: Programming & Model Usage
Alright, so now we done trained the model and the fun part of integrating Artificial Intelligence (Machine Learning in this case) with robotics (a Robo Car) using the UART communication protocol.
This is the sample program that created from the block programming interface:
We use the if-else conditional statements to evaluate the confidence of the labels.
If the confidence of "go" is greater than 0.8 (80%), I will print "1" on the serial terminal.
If the confidence of "stop" is greater than 0.8 (80%), I will print "2" on the serial terminal.
Otherwise, if the confidence of "background" is greater than 0.8 (80%), I will print "0" on the serial terminal.
Ok, so for now, just remember 3 different conditions:
"go" > 0.8, the command is '1'
"stop" > 0.8, the command is '2'
"background". the command is '0'
Alright, so take note on the 2 important findings as below as it will be the critical part for our project!
If we have a look at the text coding for the corresponding blocks code, you will notice that the Serial.print is NOT using the Serial1 line. Hence, this leads us to our second step which is to continue our coding on Arduino IDE for customization.
Toggle to the Text Code area & Copy The Text Code
On the text code area, copy all the code by pressing on CTRL + A to select all the code.
Open Arduino IDE, create a new file, paste the code into the empty sketch by pressing on CTRL+V. Proceed to save the sketch with the desired name.
Copy the Edge Impulse TinyML Arduino Library
Navigate to C:\Users\<User_Name>\AppData\Local\Programs\cc-assistant\resources\compilers\Arduino\contents\libraries
Find the folder name that has the same number with the Edge Impulse header file on top of your Arduino text code (in my case it will be 47606).
Copy the entire ei-project_47606 folder and paste it onto C:\Users\<User_Name>\Documents\Arduino\libraries\
Modification on the Serial.println function
Modify the Serial.println function to Serial1.println instead.
Upload the Code
Make sure you have installed the Wio Terminal board support package. If not, please refer to the "Get Started with Wio Terminal" guide on Seeed's Wiki.
Make sure you selected the correct board and COM port before you upload the code to Wio Terminal.
We will be programming the Robot Car that is built on the uKit Explore.
Introduction to uKit Explore
Let me give you some understanding of what's this uKit Explore about.
uKit Explore is a robotic kit manufactured by UBTECH Robotics and it is based on the Arduino ecosystem (the chipset is Arduino 2560).
"The uKit Explore has a main control box equipped with Arduino open-source platform which offers an incredible amount of electronics and programming learning resources from contributions of users worldwide."
Program uKit Explore to Read UART Data
Right now, we will need to program the Robo Car to act accordingly to the different conditions as it receives the command '1', '2', and '0' from Wio Terminal.
The key concept here is that we will need to get the uKit Explore to keep listening to the serial line if it contains any data. And if it does contain serial data, serial read the data and assigned it to a variable. The final step is that we will compare the variable and have different actions.
For our case, when uKit Explore
Upload the Code
Finally, we will upload the code onto the board. As always, please make sure you selected the correct board and COM port before the upload.
Check out the video for the Robot Car in action!
As you can see, the Robo Car can be activated via voice 'go, go, go', and stop by either 'stopppps....' or the 'background'. We have achieve the objective we wanted to!
The Codecraft is a very easy tool to train a simple embedded machine learning model and integrate it with your own robotics system powered by technology enabled by Edge Impulse and Wio Terminal. Please let me know if you have suggestions on the article. Thanks!
GitHub Source Code: https://github.com/VincentK16/voicerecognitionrobocar
