Skip to content

Blog

NeuralSPOT now supports Opus Audio Encoding

NeuralSPOT now support Opus audio encoding. With this feature, the size of audio data can be reduced by roughly 75% while maintaining good audio fidelity. This is useful for storing and moving audio data between an EVB and external devices, where it can be played and visualized.

** NOTE**: AI models tend to work best with 'raw' (non-encoded) audio, so this feature is not really for AI. Instead, it is intended to ease the creation of applications that move or store audio.

To demonstrate how the Audio Codec works, we've added a new example, audio_codec, which uses the codec to sample audio and send it to a remote player, either via RPC-over-USB, or via BLE.

Audio over BLE

This example implements a dead-simple audio-over-BLE protocol: it creates a "notify read" characteristic and starts dumping Opus frames into it. Frame size is hardcoded at 320 PCM samples, and the Opus frame size is 80 bytes.

This example pairs with a web dashboard which is configured to look for the service created by this example. Once paired, the web dashboard will play the audio and show corresponding waveforms and spectrograms. This dashboard relies on WebBLE and WebAudio, which are not supported by every browser - it has been tested on recent Chrome browser versions.

image-20231106120630313

Audio over RPC

When configured for RPC, this example acts as an RPC client, sending Opus packets over USB to an RPC server using neuralSPOT's generic data interface. To receive, decode, and record this audio to a WAV file, you can use the opus_receive.py script.

python3 -m opus_receive -t /dev/tty.usbmodem1234561 -o myaudio.wav

NeuralSPOT's EdgeImpulse Examples Have a New Home

We've moved our EdgeImpulse examples to their own repo.

NeuralSPOT has long included a set of EdgeImpulse examples, including anomaly detection and audio use cases. Previously these were located in neuralSPOT's project directory because EdgeImpulse's C++ blob has its own tensorflow and CMSIS instances which conflict with neuralSPOT.

The examples were created using neuralSPOT's Nest makefile capability along with neuralSPOT-specific porting code - the process is documented here, and is a good example of how to integrate neuralSPOT applications with external code using Nests.

These example directories were getting a little big, and include a lot of small and replicated source files, so we decided to give them their own repository. The examples are standalone, and can be compiled without neuralSPOT (they have all the neuralSPOT components they need, exported via Nests). The porting code now exists in two places - the original projects location (for ease of development of new examples) and the new repo (to make it standalone).

  • in General
  • 1 min read

First!

Now that we've moved our documentation to Github Pages we can add a blog. So we did!

Expect to see ongoing discussion on new features, new use cases, and interesting applications built using neuralSPOT.