Esp32 ADF - Audio Dev Framework - Please Help

https://github.com/espressif/esp-adf/tree/master/examples/recorder/pipeline_wav_sdcard records MIC to a WAV on the SD card; although you’ll need an older version of the main.c, maybe this.

None of these “pipeline” methods seem to give you access to the actual audio data buffers. I need those for my project. The i2s_read function in ESP-IDF is what I want to use. Unfortunately the i2S configuration in those examples only seems to support the I2S_MODE_ADC_BUILT_IN - I am not sure how to get access to the ADC in the ESP8388 chip in this fashion. The closest example I found is the VAD detector - I want to try using that.

If you look at

Then the WAV encoder pipeline does nothing but read and forward the data, so it’s the simplest form of procesising. The code for that is implemented in

and the processing happens in

So by writing your encoder / decoder you can have access to the audio buffers in the “pipeline” way ESP-ADF uses.

I found that in Speech Recognition there is a Voice Activity Detector which receives buffers, downsamples them and sends it to the VAD. I am using that. I will test it.

This example worked for me - I am able to capture data buffers and process them.

1 Like

On the LyraT board I want to output through the headphones a single audio buffer containing 1024 samples at 48 kHz. I have connected the headphone output to the AUX IN of the Lyra board with a wire and now I want to capture the waveform. From the comments in the esp-adf driver code it appears that AUXIN corresponds to audio_hal_codec_cfg.adc_input = AUDIO_HAL_ADC_INPUT_LINE2. Using the “pipeline”, even though my headphone output is as desired ( I can listen to it) just some noise is captured by the ADC.

Implementing this using i2swrite and i2sread was very easy. Any suggestions. Thanks.

UPDATE:

It turns out that once the Codec in the Lyra Board is initialized and an I2S stream is created ( as reader or writer), one can use simple i2swrite and i2sread to communicate with the codec’s DAC and ADC. There is no need for a pipeline, listener etc. For anyone interested the AUX input on the board corresponds to LINE2 of the ESP8388 ADC. Another observation is to use LEFT and RIGHT for ADC and DAC. Using ONLY_LEFT or ONLY_RIGHT seems to introduce a clock jitter.

Back to the original topic of esp-adf. Can the same thing be done to get esp-mdf working in visual studio code? I’m going to give it a go but if you see this and want to give it a shot and help me set it up I’d appreciate it.

Thanks!

Sure, same techniques apply. I’ll release a proper ESP-MDF example with the components as seperate libraries as well.

Excellent! Thanks Max!

Up at GitHub - maxgerhardt/pio-mdf-example: An example on how to use ESP-MDF with PlatformIO, work-in-progress but it compiles.

1 Like

Ok I downloaded it and it compiled for me as well. Thank you!

My next question is, how do I force it to use the same snapshot of esp-idf that came with the esp-mdf libraries? Can I just copy the esp-idf folder into the lib folder and include it as a dependency the same way you did with the other mdf libraries?

I am also having some trouble trying to use certain libraries that are arduino based. I have a file that is trying to include esp32-hal.h which appears to be part of the arduino SDK

I tried changing the framework to arduino from espidf and that broke everything

That should have worked…any idea why it doesn’t?

Nevermind, I removed all dependencies to arduino and it compiles again!

By normal means you only have the stable release versions that come with the espressif32 platform (Releases · platformio/platform-espressif32 · GitHub), which is currently 3.2.0, which is the same version number used by ESP-MDF at the moment but not the exact commit. (ESP-MDF: Feb 01, 2019, 3.2.0 release from PIO: Apr 01, 2019). This should still be good enough. If you’d need an exact commit you’d have to overwrite files within your framework-espidf fodler or fork a new platform-espressif32 to realize it; shouldn’t be necessary.

No, framework = espidf makes the platform compile ESP-IDF already.

ESP-MDF is targeted at ESP-IDF, I wouldn’t attempt to hack Arduino code into it.

I actually got everything working on my end. Thank you again for all your help! I will follow your updates though in case there is a better way to set things up then your initial check-in.

Thank you for sharing, worked nicely!

Many thanks @jeffdc1979 and @maxgerhardt. your information is very helpful. @jeffdc1979 can you share the platformIO.ini file, I got the problem with it so I also cannot run. The picture attached. Many thanks for your help.

What exact project are you running? Do you have a github for reproduction?

Dear @maxgerhardt, many thanks for your reply. My project is try to using A2DP & BLE at the same time, also support SD card. The first step is how to streaming the music from SD card first. My board is lyrat_v4.3. I already cloned your github and try to modify, remove something in platformIO.ini. But still not success like the picture. I will share the link on github. If you have free time please take a look. Respect your help. ^^
P/s: My github (has just created) for the project: GitHub - kokonomutsu/ESP32_Coex_Demo

Problem solved. Thanks all.
Adding the library for platformIO.ini:
" -llibesp-mp3
-llibesp_processing
-llibesp-ogg-container
-llibesp-opus
-llibesp-tremor"

1 Like