Shulou(Shulou.com)06/01 Report--

What is I2S speech recognition in Zephyr and TensorFlow Lite? I believe many inexperienced people don't know what to do about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

Machine learning is still mainly done in the cloud, which in many cases can lead to unnecessary delays, excessive power consumption, and reliance on the availability of wireless connections. Thanks to the latest developments in machine learning in microcontrollers and FPGA (an area in which Antmicro is currently heavily involved), small devices have become powerful enough to perform machine learning tasks locally, such as image or voice recognition.

Machine Learning and FPGA

Our experience in machine learning includes creating complex artificial intelligence algorithms, building ML/AI accelerators, designing specialized drivers, and providing tools to simplify ML development. While much of our artificial intelligence work is on high-end platforms such as NVIDIA Xavier NX, Google Coral or Xilinx UltraScale+, more and more customers are also considering implementing ML on smaller devices.

In a recent collaboration with Google, we enabled their TensorFlow Lite machine learning framework to run on the FPGA platform based on the soft SoC generation framework LiteX. The project introduced TF Lite to FPGA for the first time, which means that a whole new set of embedded and Internet of things devices can now benefit from the features of the Google Framework, on which developers can deploy ML models for gesture and speech recognition, keyword detection, and so on. It also opens the door for further optimization of ML applications using dedicated hardware accelerators in FPGA. In an article on the TF Lite blog, we described this work and demonstrated running and testing the ML framework in the open source simulator Renode. Importantly, as part of this effort, we integrated TF Lite Micro with the Zephyr real-time operating system. With Google and Facebook becoming platinum members, the RTOS is going through an unprecedented period of growth. Integration with Zephyr makes it possible to add new platforms and applications relatively quickly, as shown in this article.

Overview of setup and Architecture

This development makes it possible to perform speech recognition in systems running LiteX-based soft SoC.

The settings used as examples include an Arty A7 board and Pmod I2S2 from Digilent.

In the demonstration (instructions can be found on GitHub), the ADC (analog-to-digital converter) chip on Pmod collects analog signals, samples them, converts them into digital signals, and then sends them to FPGA via I ²S, which our extended LiteX-based IP core receives in PCM format. The Zephyr driver then reads the data from the FIFO buffer in real time and provides it to the TF Lite application that performs speech recognition.

LiteX gets the sound through Zephyr.

To achieve the results described above, we must first extend the I ²S interface support in LiteX to configure it as master, which allows the interface not only to play sound, but also to capture it. Next, we developed a Zephyr driver that enables I ²S to communicate with Pmod and allows CPU to process the received data. We also wrote a software interface in the TF Lite speech recognition demonstration to extract the sound from the Zephyr driver and pass it to the neural network.

The original LiteX-based FPGA IP core only supports 24-bit stereo data per sample, so as the final work, we extend it to the format required for speech recognition demonstrations, such as mono 16-bit. The neural network can recognize the words "yes" and "no", which is the focus of the demonstration and validates our FPGA/ software design.

After reading the above, have you mastered the method of I2S speech recognition in Zephyr and TensorFlow Lite? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.